2024-05-10T03:41:19.144 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-10T03:41:19.144 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505 --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iozone}} -- /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505/orig.config.yaml 2024-05-10T03:41:19.198 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T03:41:19.275 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505 branch: wip-pdonnell-testing-20240509.171335-reef-debug description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iozone}} email: pdonnell@redhat.com first_in_suite: false job_id: '7700505' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20240509.171335-reef-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 50174015a7e79f951b988949abe236791f52d4da subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 50174015a7e79f951b988949abe236791f52d4da kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240509.171335-reef-debug sha1: 50174015a7e79f951b988949abe236791f52d4da owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4933 sha1: 50174015a7e79f951b988949abe236791f52d4da sleep_before_teardown: 0 subset: 420/512 suite: fs suite_branch: wip-pdonnell-testing-20240509.171335-reef-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_50174015a7e79f951b988949abe236791f52d4da/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 50174015a7e79f951b988949abe236791f52d4da syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPJyftkxLjErqvMULWu1aB2DKZHxxGrQgIJ/k6TWbOEHR/EB4sq4iMrnx2zs38yovKnQZAB0p99HHt3+EGQWLo4= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLdjM5ZNuDtBwZ3Bhlap+HCSvEVTBzAPd2kiBRps4ls77OL+SAPq086bnvTk0s72YwczwCnMG7wKM2U7Zqz01pE= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0/ckGF0yydCFiwBZj3nKIRTAMN71IWQtb8eL+djXqC4HQo1p7GuRjaO/F+wNUy8yOOIYBYI0Yzf0fGhb5zZP4= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-10_02:33:39 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-10T03:41:19.275 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_50174015a7e79f951b988949abe236791f52d4da/qa; will attempt to use it 2024-05-10T03:41:19.276 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_50174015a7e79f951b988949abe236791f52d4da/qa/tasks 2024-05-10T03:41:19.276 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-10T03:41:19.278 INFO:teuthology.task.internal:Checking packages... 2024-05-10T03:41:19.301 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '50174015a7e79f951b988949abe236791f52d4da' 2024-05-10T03:41:19.301 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-10T03:41:19.301 INFO:teuthology.packaging:ref: None 2024-05-10T03:41:19.301 INFO:teuthology.packaging:tag: None 2024-05-10T03:41:19.301 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240509.171335-reef-debug 2024-05-10T03:41:19.301 INFO:teuthology.packaging:sha1: 50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:41:19.301 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20240509.171335-reef-debug 2024-05-10T03:41:19.521 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1330.g50174015 2024-05-10T03:41:19.523 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-10T03:41:19.530 INFO:teuthology.task.internal:no buildpackages task found 2024-05-10T03:41:19.531 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-10T03:41:19.575 INFO:teuthology.task.internal:Saving configuration 2024-05-10T03:41:19.597 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-10T03:41:19.608 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T03:41:19.626 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 03:35:57.624236', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPJyftkxLjErqvMULWu1aB2DKZHxxGrQgIJ/k6TWbOEHR/EB4sq4iMrnx2zs38yovKnQZAB0p99HHt3+EGQWLo4='} 2024-05-10T03:41:19.649 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 03:35:57.622722', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLdjM5ZNuDtBwZ3Bhlap+HCSvEVTBzAPd2kiBRps4ls77OL+SAPq086bnvTk0s72YwczwCnMG7wKM2U7Zqz01pE='} 2024-05-10T03:41:19.671 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 03:35:57.621133', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0/ckGF0yydCFiwBZj3nKIRTAMN71IWQtb8eL+djXqC4HQo1p7GuRjaO/F+wNUy8yOOIYBYI0Yzf0fGhb5zZP4='} 2024-05-10T03:41:19.671 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-10T03:41:19.678 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-10T03:41:19.678 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-10T03:41:19.678 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-10T03:41:19.678 INFO:teuthology.run_tasks:Running task console_log... 2024-05-10T03:41:19.862 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff4b4f920d0>, signals=[15]) 2024-05-10T03:41:19.862 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-10T03:41:19.869 INFO:teuthology.task.internal:Opening connections... 2024-05-10T03:41:19.869 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T03:41:19.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:41:19.947 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-05-10T03:41:19.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:41:20.022 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2024-05-10T03:41:20.022 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:41:20.095 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-10T03:41:20.101 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-05-10T03:41:20.122 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-05-10T03:41:20.122 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-05-10T03:41:20.182 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="Red Hat Enterprise Linux" 2024-05-10T03:41:20.182 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="8.6 (Ootpa)" 2024-05-10T03:41:20.182 INFO:teuthology.orchestra.run.smithi150.stdout:ID="rhel" 2024-05-10T03:41:20.182 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="fedora" 2024-05-10T03:41:20.182 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="8.6" 2024-05-10T03:41:20.182 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el8" 2024-05-10T03:41:20.182 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-10T03:41:20.182 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-05-10T03:41:20.182 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-10T03:41:20.182 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://www.redhat.com/" 2024-05-10T03:41:20.182 INFO:teuthology.orchestra.run.smithi150.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-10T03:41:20.182 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T03:41:20.183 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:41:20.183 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-10T03:41:20.183 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-10T03:41:20.183 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-10T03:41:20.183 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-10T03:41:20.183 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-05-10T03:41:20.203 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-05-10T03:41:20.224 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-05-10T03:41:20.225 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-05-10T03:41:20.282 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Red Hat Enterprise Linux" 2024-05-10T03:41:20.282 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8.6 (Ootpa)" 2024-05-10T03:41:20.282 INFO:teuthology.orchestra.run.smithi161.stdout:ID="rhel" 2024-05-10T03:41:20.282 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="fedora" 2024-05-10T03:41:20.282 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8.6" 2024-05-10T03:41:20.282 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8" 2024-05-10T03:41:20.283 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-10T03:41:20.283 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-05-10T03:41:20.283 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-10T03:41:20.283 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.redhat.com/" 2024-05-10T03:41:20.283 INFO:teuthology.orchestra.run.smithi161.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-10T03:41:20.283 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T03:41:20.283 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T03:41:20.283 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-10T03:41:20.283 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-10T03:41:20.283 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-10T03:41:20.284 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-10T03:41:20.284 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-05-10T03:41:20.303 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2024-05-10T03:41:20.326 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2024-05-10T03:41:20.326 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2024-05-10T03:41:20.386 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Red Hat Enterprise Linux" 2024-05-10T03:41:20.386 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8.6 (Ootpa)" 2024-05-10T03:41:20.386 INFO:teuthology.orchestra.run.smithi181.stdout:ID="rhel" 2024-05-10T03:41:20.386 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="fedora" 2024-05-10T03:41:20.386 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8.6" 2024-05-10T03:41:20.386 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2024-05-10T03:41:20.386 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-10T03:41:20.386 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2024-05-10T03:41:20.387 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-10T03:41:20.387 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.redhat.com/" 2024-05-10T03:41:20.387 INFO:teuthology.orchestra.run.smithi181.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-10T03:41:20.387 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T03:41:20.387 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:41:20.387 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-10T03:41:20.387 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-10T03:41:20.387 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-10T03:41:20.387 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-10T03:41:20.387 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2024-05-10T03:41:20.411 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-10T03:41:20.423 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-10T03:41:20.473 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-10T03:41:20.473 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-05-10T03:41:20.476 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-05-10T03:41:20.478 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2024-05-10T03:41:20.497 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-10T03:41:20.523 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-10T03:41:20.524 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-05-10T03:41:20.537 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-05-10T03:41:20.539 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2024-05-10T03:41:20.568 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-10T03:41:20.648 INFO:teuthology.run_tasks:Running task kernel... 2024-05-10T03:41:20.662 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-10T03:41:20.662 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-10T03:41:20.663 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-10T03:41:20.663 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T03:41:20.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:41:20.682 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-05-10T03:41:20.746 INFO:teuthology.orchestra.run.smithi150.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-10T03:41:20.746 INFO:teuthology.task.kernel:Running kernel on smithi150: 4.18.0-513.9.1.el8_9.x86_64 2024-05-10T03:41:20.746 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum install -y kernel 2024-05-10T03:41:22.164 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T03:41:22.165 INFO:teuthology.orchestra.run.smithi150.stdout:Unable to read consumer identity 2024-05-10T03:41:22.459 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:41:22.460 INFO:teuthology.orchestra.run.smithi150.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-10T03:41:22.460 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:41:24.181 INFO:teuthology.orchestra.run.smithi150.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-05-10T03:41:25.223 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 15 MB/s | 14 MB 00:00 2024-05-10T03:41:28.573 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 400 kB/s | 24 kB 00:00 2024-05-10T03:41:29.328 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-10T03:41:29.328 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-10T03:41:29.404 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-05-10T03:41:29.405 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-05-10T03:41:29.405 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-05-10T03:41:29.550 DEBUG:teuthology.orchestra.run.smithi150:> echo no | sudo yum reinstall kernel || true 2024-05-10T03:41:29.884 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T03:41:29.884 INFO:teuthology.orchestra.run.smithi150.stdout:Unable to read consumer identity 2024-05-10T03:41:30.113 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:41:30.113 INFO:teuthology.orchestra.run.smithi150.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-10T03:41:30.113 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:41:30.424 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:02 ago on Fri 10 May 2024 03:41:28 AM UTC. 2024-05-10T03:41:30.507 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-10T03:41:30.507 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-10T03:41:30.579 INFO:teuthology.orchestra.run.smithi150.stderr:Error: No packages marked for reinstall. 2024-05-10T03:41:30.626 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum reinstall -y kernel || true 2024-05-10T03:41:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T03:41:30.960 INFO:teuthology.orchestra.run.smithi150.stdout:Unable to read consumer identity 2024-05-10T03:41:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:41:31.169 INFO:teuthology.orchestra.run.smithi150.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-10T03:41:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:41:31.470 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:03 ago on Fri 10 May 2024 03:41:28 AM UTC. 2024-05-10T03:41:31.539 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-10T03:41:31.539 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-10T03:41:31.567 INFO:teuthology.orchestra.run.smithi150.stderr:Error: No packages marked for reinstall. 2024-05-10T03:41:31.621 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T03:41:31.787 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-10T03:41:31.787 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-10T03:41:31.787 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-10T03:41:31.788 DEBUG:teuthology.parallel:result is None 2024-05-10T03:41:31.788 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-10T03:41:31.799 INFO:teuthology.task.internal:Creating test directory... 2024-05-10T03:41:31.799 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T03:41:31.802 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T03:41:31.805 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T03:41:31.827 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-10T03:41:31.849 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-10T03:41:31.883 INFO:teuthology.task.internal:Creating archive directory... 2024-05-10T03:41:31.883 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T03:41:31.885 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T03:41:31.887 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T03:41:31.924 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-10T03:41:31.933 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-10T03:41:31.934 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T03:41:31.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:41:31.977 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T03:41:31.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:41:31.997 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T03:41:31.999 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T03:41:32.019 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T03:41:32.039 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T03:41:32.060 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T03:41:32.063 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T03:41:32.075 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T03:41:32.082 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T03:41:32.092 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T03:41:32.094 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-10T03:41:32.102 INFO:teuthology.task.internal:Configuring sudo... 2024-05-10T03:41:32.102 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T03:41:32.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T03:41:32.107 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T03:41:32.164 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-10T03:41:32.173 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-10T03:41:32.174 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T03:41:32.176 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T03:41:32.178 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T03:41:32.222 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T03:41:32.277 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T03:41:32.365 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T03:41:32.410 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T03:41:32.449 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:41:32.449 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T03:41:32.515 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T03:41:32.561 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T03:41:32.615 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T03:41:32.701 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T03:41:32.743 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T03:41:32.744 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T03:41:32.812 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T03:41:32.858 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T03:41:32.916 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T03:41:33.004 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T03:41:33.047 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T03:41:33.048 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T03:41:33.116 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-05-10T03:41:33.119 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-05-10T03:41:33.121 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2024-05-10T03:41:33.162 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T03:41:33.186 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T03:41:33.214 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T03:41:33.504 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-10T03:41:33.513 INFO:teuthology.task.internal:Starting timer... 2024-05-10T03:41:33.513 INFO:teuthology.run_tasks:Running task pcp... 2024-05-10T03:41:33.562 INFO:teuthology.run_tasks:Running task selinux... 2024-05-10T03:41:33.609 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2024-05-10T03:41:33.610 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-05-10T03:41:33.685 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-05-10T03:41:33.687 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-05-10T03:41:33.754 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-05-10T03:41:33.755 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2024-05-10T03:41:33.817 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2024-05-10T03:41:33.819 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-10T03:41:33.819 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-05-10T03:41:33.845 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-05-10T03:41:33.845 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-05-10T03:41:33.866 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-05-10T03:41:33.866 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2024-05-10T03:41:33.890 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2024-05-10T03:41:33.891 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi150.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive'} 2024-05-10T03:41:33.891 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T03:41:33.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:41:33.919 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T03:41:33.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:41:33.946 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T03:41:33.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:41:33.973 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-10T03:41:33.973 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-05-10T03:41:34.015 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-05-10T03:41:34.042 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2024-05-10T03:41:34.071 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-10T03:41:34.085 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-10T03:41:34.102 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-10T03:41:34.105 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi150.front.sepia.ceph.com,smithi161.front.sepia.ceph.com,smithi181.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-10T03:52:00.365 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi150.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com')] 2024-05-10T03:52:00.367 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-10T03:52:00.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:52:00.446 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-05-10T03:52:00.512 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-10T03:52:00.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-10T03:52:00.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:52:00.590 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-05-10T03:52:00.664 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-10T03:52:00.665 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-05-10T03:52:00.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:52:00.736 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-05-10T03:52:00.808 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-05-10T03:52:00.809 INFO:teuthology.run_tasks:Running task clock... 2024-05-10T03:52:00.820 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-10T03:52:00.820 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T03:52:00.821 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T03:52:00.823 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T03:52:00.824 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T03:52:00.826 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T03:52:00.826 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T03:52:00.857 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T03:52:00.861 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T03:52:00.873 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T03:52:00.878 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T03:52:00.880 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T03:52:00.895 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T03:52:00.944 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-05-10T03:52:00.956 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-05-10T03:52:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-05-10T03:52:00.961 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2024-05-10T03:52:00.971 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T03:52:00.973 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2024-05-10T03:52:00.973 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-05-10T03:52:00.986 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T03:52:00.987 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T03:52:00.989 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T03:52:01.001 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T03:52:01.003 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T03:52:01.051 INFO:teuthology.orchestra.run.smithi150.stderr:bash: ntpq: command not found 2024-05-10T03:52:01.056 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T03:52:01.056 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-05-10T03:52:01.056 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:52:01.056 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:52:01.056 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:52:01.056 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:52:01.070 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2024-05-10T03:52:01.073 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T03:52:01.074 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-05-10T03:52:01.074 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:52:01.074 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:52:01.074 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:52:01.074 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:52:01.076 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2024-05-10T03:52:01.079 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T03:52:01.079 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-05-10T03:52:01.079 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:52:01.080 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:52:01.080 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:52:01.080 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:52:01.081 INFO:teuthology.run_tasks:Running task pexec... 2024-05-10T03:52:01.089 INFO:teuthology.task.pexec:Executing custom commands... 2024-05-10T03:52:01.090 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T03:52:01.090 DEBUG:teuthology.orchestra.run.smithi150:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-10T03:52:01.090 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi161.front.sepia.ceph.com 2024-05-10T03:52:01.090 DEBUG:teuthology.orchestra.run.smithi161:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-10T03:52:01.091 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi181.front.sepia.ceph.com 2024-05-10T03:52:01.091 DEBUG:teuthology.orchestra.run.smithi181:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-10T03:52:01.478 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T03:52:01.491 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2024-05-10T03:52:01.500 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2024-05-10T03:52:03.387 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-05-10T03:52:03.411 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 kB/s | 4.1 kB 00:00 2024-05-10T03:52:03.606 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-05-10T03:52:03.833 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-05-10T03:52:03.984 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-05-10T03:52:04.072 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-05-10T03:52:06.965 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-05-10T03:52:06.966 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-05-10T03:52:06.966 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-05-10T03:52:06.966 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-05-10T03:52:06.967 INFO:teuthology.orchestra.run.smithi150.stdout:Resetting modules: 2024-05-10T03:52:06.967 INFO:teuthology.orchestra.run.smithi150.stdout: container-tools 2024-05-10T03:52:06.967 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:52:06.967 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-05-10T03:52:06.967 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-05-10T03:52:06.967 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:52:07.135 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-05-10T03:52:07.137 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-05-10T03:52:07.137 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2024-05-10T03:52:07.137 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-05-10T03:52:07.137 INFO:teuthology.orchestra.run.smithi161.stdout:Resetting modules: 2024-05-10T03:52:07.137 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools 2024-05-10T03:52:07.137 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T03:52:07.137 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-05-10T03:52:07.137 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-05-10T03:52:07.137 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T03:52:07.138 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-05-10T03:52:07.139 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-05-10T03:52:07.139 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2024-05-10T03:52:07.139 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-05-10T03:52:07.139 INFO:teuthology.orchestra.run.smithi181.stdout:Resetting modules: 2024-05-10T03:52:07.139 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools 2024-05-10T03:52:07.139 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:52:07.139 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-05-10T03:52:07.140 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-05-10T03:52:07.140 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:52:16.789 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-05-10T03:52:17.228 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-05-10T03:52:17.279 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2024-05-10T03:52:17.714 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2024-05-10T03:52:19.225 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-05-10T03:52:19.569 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 25 kB/s | 4.1 kB 00:00 2024-05-10T03:52:19.655 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-05-10T03:52:19.962 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 kB/s | 4.5 kB 00:00 2024-05-10T03:52:20.706 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-05-10T03:52:21.181 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T03:52:22.729 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-05-10T03:52:22.892 INFO:teuthology.orchestra.run.smithi181.stdout:========================================================================================================== 2024-05-10T03:52:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-05-10T03:52:22.893 INFO:teuthology.orchestra.run.smithi181.stdout:========================================================================================================== 2024-05-10T03:52:22.893 INFO:teuthology.orchestra.run.smithi181.stdout:Installing group/module packages: 2024-05-10T03:52:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-10T03:52:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-10T03:52:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-10T03:52:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-10T03:52:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T03:52:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-10T03:52:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout:Installing weak dependencies: 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout:Installing module profiles: 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools/common 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling module streams: 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools rhel8 2024-05-10T03:52:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:52:22.895 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-05-10T03:52:22.895 INFO:teuthology.orchestra.run.smithi181.stdout:========================================================================================================== 2024-05-10T03:52:22.895 INFO:teuthology.orchestra.run.smithi181.stdout:Install 13 Packages 2024-05-10T03:52:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:52:22.895 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 26 M 2024-05-10T03:52:22.896 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 87 M 2024-05-10T03:52:22.896 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-05-10T03:52:22.952 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 kB/s | 4.5 kB 00:00 2024-05-10T03:52:23.085 INFO:teuthology.orchestra.run.smithi181.stdout:(1/13): criu-libs-3.18-4.module+el8.9.0+20326+3 208 kB/s | 39 kB 00:00 2024-05-10T03:52:23.089 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-05-10T03:52:23.110 INFO:teuthology.orchestra.run.smithi181.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 116 kB/s | 25 kB 00:00 2024-05-10T03:52:23.135 INFO:teuthology.orchestra.run.smithi181.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 392 kB/s | 94 kB 00:00 2024-05-10T03:52:23.257 INFO:teuthology.orchestra.run.smithi161.stdout:========================================================================================================== 2024-05-10T03:52:23.257 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-05-10T03:52:23.258 INFO:teuthology.orchestra.run.smithi161.stdout:========================================================================================================== 2024-05-10T03:52:23.258 INFO:teuthology.orchestra.run.smithi161.stdout:Installing group/module packages: 2024-05-10T03:52:23.258 INFO:teuthology.orchestra.run.smithi161.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-10T03:52:23.258 INFO:teuthology.orchestra.run.smithi161.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-10T03:52:23.258 INFO:teuthology.orchestra.run.smithi161.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-10T03:52:23.258 INFO:teuthology.orchestra.run.smithi161.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-10T03:52:23.259 INFO:teuthology.orchestra.run.smithi161.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T03:52:23.259 INFO:teuthology.orchestra.run.smithi161.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-10T03:52:23.259 INFO:teuthology.orchestra.run.smithi161.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-10T03:52:23.259 INFO:teuthology.orchestra.run.smithi161.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-10T03:52:23.259 INFO:teuthology.orchestra.run.smithi161.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-10T03:52:23.259 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-05-10T03:52:23.260 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-10T03:52:23.260 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-10T03:52:23.260 INFO:teuthology.orchestra.run.smithi161.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-10T03:52:23.260 INFO:teuthology.orchestra.run.smithi161.stdout:Installing weak dependencies: 2024-05-10T03:52:23.260 INFO:teuthology.orchestra.run.smithi161.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T03:52:23.260 INFO:teuthology.orchestra.run.smithi161.stdout:Installing module profiles: 2024-05-10T03:52:23.260 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools/common 2024-05-10T03:52:23.261 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T03:52:23.261 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling module streams: 2024-05-10T03:52:23.261 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools rhel8 2024-05-10T03:52:23.261 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T03:52:23.261 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-05-10T03:52:23.261 INFO:teuthology.orchestra.run.smithi161.stdout:========================================================================================================== 2024-05-10T03:52:23.261 INFO:teuthology.orchestra.run.smithi161.stdout:Install 13 Packages 2024-05-10T03:52:23.262 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T03:52:23.263 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 26 M 2024-05-10T03:52:23.263 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 87 M 2024-05-10T03:52:23.263 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-05-10T03:52:23.319 INFO:teuthology.orchestra.run.smithi181.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 175 kB/s | 41 kB 00:00 2024-05-10T03:52:23.430 INFO:teuthology.orchestra.run.smithi161.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 554 kB/s | 94 kB 00:00 2024-05-10T03:52:23.446 INFO:teuthology.orchestra.run.smithi181.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 3.0 MB/s | 1.0 MB 00:00 2024-05-10T03:52:23.455 INFO:teuthology.orchestra.run.smithi161.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 127 kB/s | 25 kB 00:00 2024-05-10T03:52:23.480 INFO:teuthology.orchestra.run.smithi161.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 179 kB/s | 39 kB 00:00 2024-05-10T03:52:23.572 INFO:teuthology.orchestra.run.smithi161.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 288 kB/s | 41 kB 00:00 2024-05-10T03:52:23.620 INFO:teuthology.orchestra.run.smithi181.stdout:(6/13): buildah-1.31.5-1.module+el8.9.0+21697+6 18 MB/s | 8.8 MB 00:00 2024-05-10T03:52:23.661 INFO:teuthology.orchestra.run.smithi181.stdout:(7/13): cockpit-podman-75-1.module+el8.9.0+2169 2.1 MB/s | 739 kB 00:00 2024-05-10T03:52:23.681 INFO:teuthology.orchestra.run.smithi161.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 4.5 MB/s | 1.0 MB 00:00 2024-05-10T03:52:23.740 INFO:teuthology.orchestra.run.smithi161.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 4.3 MB/s | 739 kB 00:00 2024-05-10T03:52:23.779 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 14 kB/s | 4.1 kB 00:00 2024-05-10T03:52:23.853 INFO:teuthology.orchestra.run.smithi181.stdout:(8/13): python3-podman-4.6.0-2.module+el8.9.0+2 801 kB/s | 154 kB 00:00 2024-05-10T03:52:23.878 INFO:teuthology.orchestra.run.smithi181.stdout:(9/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 554 kB/s | 239 kB 00:00 2024-05-10T03:52:23.915 INFO:teuthology.orchestra.run.smithi161.stdout:(7/13): buildah-1.31.5-1.module+el8.9.0+21697+6 20 MB/s | 8.8 MB 00:00 2024-05-10T03:52:23.940 INFO:teuthology.orchestra.run.smithi161.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 924 kB/s | 239 kB 00:00 2024-05-10T03:52:23.962 INFO:teuthology.orchestra.run.smithi181.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 11 MB/s | 3.7 MB 00:00 2024-05-10T03:52:24.024 INFO:teuthology.orchestra.run.smithi161.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21697+6 13 MB/s | 3.7 MB 00:00 2024-05-10T03:52:24.082 INFO:teuthology.orchestra.run.smithi161.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 920 kB/s | 154 kB 00:00 2024-05-10T03:52:24.129 INFO:teuthology.orchestra.run.smithi181.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 10 MB/s | 2.5 MB 00:00 2024-05-10T03:52:24.154 INFO:teuthology.orchestra.run.smithi181.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 258 kB/s | 49 kB 00:00 2024-05-10T03:52:24.157 INFO:teuthology.orchestra.run.smithi161.stdout:(11/13): udica-0.2.6-20.module+el8.9.0+21697+6a 658 kB/s | 49 kB 00:00 2024-05-10T03:52:24.341 INFO:teuthology.orchestra.run.smithi161.stdout:(12/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 8.0 MB/s | 2.5 MB 00:00 2024-05-10T03:52:24.388 INFO:teuthology.orchestra.run.smithi181.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 15 MB/s | 8.1 MB 00:00 2024-05-10T03:52:24.389 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-05-10T03:52:24.389 INFO:teuthology.orchestra.run.smithi181.stdout:Total 17 MB/s | 26 MB 00:01 2024-05-10T03:52:24.499 INFO:teuthology.orchestra.run.smithi161.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 15 MB/s | 8.1 MB 00:00 2024-05-10T03:52:24.500 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-05-10T03:52:24.500 INFO:teuthology.orchestra.run.smithi161.stdout:Total 21 MB/s | 26 MB 00:01 2024-05-10T03:52:24.666 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-05-10T03:52:24.687 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-05-10T03:52:24.688 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-05-10T03:52:24.781 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-05-10T03:52:24.803 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-05-10T03:52:24.803 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-05-10T03:52:24.916 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-05-10T03:52:24.926 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-05-10T03:52:25.036 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-05-10T03:52:25.045 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-05-10T03:52:25.698 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-05-10T03:52:25.826 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-05-10T03:52:26.203 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-10T03:52:26.379 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-10T03:52:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-10T03:52:26.585 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-10T03:52:26.600 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-10T03:52:26.622 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-05-10T03:52:26.752 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-10T03:52:26.777 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-10T03:52:26.785 INFO:teuthology.orchestra.run.smithi150.stdout:========================================================================================================== 2024-05-10T03:52:26.785 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-05-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi150.stdout:========================================================================================================== 2024-05-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi150.stdout:Installing group/module packages: 2024-05-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi150.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi150.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi150.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi150.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi150.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi150.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi150.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi150.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi150.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-05-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout:Installing module profiles: 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout: container-tools/common 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout:Enabling module streams: 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout: container-tools rhel8 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout:========================================================================================================== 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout:Install 13 Packages 2024-05-10T03:52:26.787 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:52:26.788 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 26 M 2024-05-10T03:52:26.788 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 87 M 2024-05-10T03:52:26.788 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-05-10T03:52:26.924 INFO:teuthology.orchestra.run.smithi150.stdout:(1/13): criu-libs-3.18-4.module+el8.9.0+20326+3 290 kB/s | 39 kB 00:00 2024-05-10T03:52:26.941 INFO:teuthology.orchestra.run.smithi150.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 162 kB/s | 25 kB 00:00 2024-05-10T03:52:26.952 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-10T03:52:26.967 INFO:teuthology.orchestra.run.smithi150.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 528 kB/s | 94 kB 00:00 2024-05-10T03:52:26.980 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-10T03:52:27.058 INFO:teuthology.orchestra.run.smithi150.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 306 kB/s | 41 kB 00:00 2024-05-10T03:52:27.109 INFO:teuthology.orchestra.run.smithi150.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 6.0 MB/s | 1.0 MB 00:00 2024-05-10T03:52:27.141 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-10T03:52:27.198 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-10T03:52:27.276 INFO:teuthology.orchestra.run.smithi150.stdout:(6/13): buildah-1.31.5-1.module+el8.9.0+21697+6 29 MB/s | 8.8 MB 00:00 2024-05-10T03:52:27.301 INFO:teuthology.orchestra.run.smithi150.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.2 MB/s | 239 kB 00:00 2024-05-10T03:52:27.334 INFO:teuthology.orchestra.run.smithi150.stdout:(8/13): cockpit-podman-75-1.module+el8.9.0+2169 2.6 MB/s | 739 kB 00:00 2024-05-10T03:52:27.371 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-10T03:52:27.425 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-10T03:52:27.551 INFO:teuthology.orchestra.run.smithi150.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21697+6 13 MB/s | 3.7 MB 00:00 2024-05-10T03:52:27.577 INFO:teuthology.orchestra.run.smithi150.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 557 kB/s | 154 kB 00:00 2024-05-10T03:52:27.664 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-10T03:52:27.702 INFO:teuthology.orchestra.run.smithi150.stdout:(11/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 22 MB/s | 8.1 MB 00:00 2024-05-10T03:52:27.727 INFO:teuthology.orchestra.run.smithi150.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 329 kB/s | 49 kB 00:00 2024-05-10T03:52:27.786 INFO:teuthology.orchestra.run.smithi150.stdout:(13/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 11 MB/s | 2.5 MB 00:00 2024-05-10T03:52:27.786 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-05-10T03:52:27.786 INFO:teuthology.orchestra.run.smithi150.stdout:Total 26 MB/s | 26 MB 00:00 2024-05-10T03:52:28.066 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-05-10T03:52:28.087 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-05-10T03:52:28.088 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-05-10T03:52:28.317 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-05-10T03:52:28.326 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-05-10T03:52:28.327 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T03:52:28.545 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T03:52:28.568 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T03:52:28.791 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T03:52:28.955 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-10T03:52:29.087 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-05-10T03:52:29.178 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-10T03:52:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-10T03:52:29.748 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-10T03:52:29.842 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-10T03:52:29.890 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-10T03:52:30.054 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-10T03:52:30.070 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-10T03:52:30.074 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-10T03:52:30.220 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T03:52:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-10T03:52:30.298 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-10T03:52:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-10T03:52:30.472 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T03:52:30.631 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T03:52:30.631 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-10T03:52:30.631 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-10T03:52:30.631 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-10T03:52:30.632 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-10T03:52:30.632 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-10T03:52:30.632 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-10T03:52:30.632 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-10T03:52:30.632 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T03:52:30.632 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T03:52:30.632 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-10T03:52:30.632 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-10T03:52:30.632 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-10T03:52:30.681 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-10T03:52:30.880 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T03:52:30.881 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-10T03:52:30.881 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-10T03:52:30.881 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-10T03:52:30.881 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-10T03:52:30.881 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-10T03:52:30.881 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-10T03:52:30.881 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-10T03:52:30.881 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T03:52:30.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T03:52:30.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-10T03:52:30.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-10T03:52:30.882 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-10T03:52:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T03:52:31.754 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T03:52:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-10T03:52:33.009 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-10T03:52:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-10T03:52:33.352 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T03:52:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T03:52:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-10T03:52:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-10T03:52:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-10T03:52:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-10T03:52:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-10T03:52:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-10T03:52:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-10T03:52:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T03:52:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T03:52:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-10T03:52:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-10T03:52:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-10T03:52:40.311 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-10T03:52:40.311 INFO:teuthology.orchestra.run.smithi181.stdout:Installed products updated. 2024-05-10T03:52:40.867 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:52:40.867 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-05-10T03:52:40.867 INFO:teuthology.orchestra.run.smithi181.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:40.867 INFO:teuthology.orchestra.run.smithi181.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:40.867 INFO:teuthology.orchestra.run.smithi181.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T03:52:40.867 INFO:teuthology.orchestra.run.smithi181.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-10T03:52:40.868 INFO:teuthology.orchestra.run.smithi181.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:40.868 INFO:teuthology.orchestra.run.smithi181.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:40.868 INFO:teuthology.orchestra.run.smithi181.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T03:52:40.868 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-10T03:52:40.868 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-10T03:52:40.868 INFO:teuthology.orchestra.run.smithi181.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:40.868 INFO:teuthology.orchestra.run.smithi181.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:40.868 INFO:teuthology.orchestra.run.smithi181.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T03:52:40.868 INFO:teuthology.orchestra.run.smithi181.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-10T03:52:40.868 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:52:40.868 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-05-10T03:52:41.034 DEBUG:teuthology.parallel:result is None 2024-05-10T03:52:44.802 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-10T03:52:44.802 INFO:teuthology.orchestra.run.smithi161.stdout:Installed products updated. 2024-05-10T03:52:45.379 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T03:52:45.379 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-05-10T03:52:45.379 INFO:teuthology.orchestra.run.smithi161.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:45.379 INFO:teuthology.orchestra.run.smithi161.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:45.379 INFO:teuthology.orchestra.run.smithi161.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T03:52:45.380 INFO:teuthology.orchestra.run.smithi161.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-10T03:52:45.380 INFO:teuthology.orchestra.run.smithi161.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:45.380 INFO:teuthology.orchestra.run.smithi161.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:45.380 INFO:teuthology.orchestra.run.smithi161.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T03:52:45.380 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-10T03:52:45.380 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-10T03:52:45.380 INFO:teuthology.orchestra.run.smithi161.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:45.380 INFO:teuthology.orchestra.run.smithi161.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:45.380 INFO:teuthology.orchestra.run.smithi161.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T03:52:45.380 INFO:teuthology.orchestra.run.smithi161.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-10T03:52:45.380 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T03:52:45.380 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-05-10T03:52:45.584 DEBUG:teuthology.parallel:result is None 2024-05-10T03:52:48.417 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-10T03:52:48.418 INFO:teuthology.orchestra.run.smithi150.stdout:Installed products updated. 2024-05-10T03:52:48.983 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:52:48.983 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-05-10T03:52:48.983 INFO:teuthology.orchestra.run.smithi150.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:48.983 INFO:teuthology.orchestra.run.smithi150.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:48.983 INFO:teuthology.orchestra.run.smithi150.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T03:52:48.983 INFO:teuthology.orchestra.run.smithi150.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-10T03:52:48.983 INFO:teuthology.orchestra.run.smithi150.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:48.983 INFO:teuthology.orchestra.run.smithi150.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:48.983 INFO:teuthology.orchestra.run.smithi150.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T03:52:48.983 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-10T03:52:48.983 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-10T03:52:48.983 INFO:teuthology.orchestra.run.smithi150.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:48.984 INFO:teuthology.orchestra.run.smithi150.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T03:52:48.984 INFO:teuthology.orchestra.run.smithi150.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T03:52:48.984 INFO:teuthology.orchestra.run.smithi150.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-10T03:52:48.984 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:52:48.984 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-05-10T03:52:49.143 DEBUG:teuthology.parallel:result is None 2024-05-10T03:52:49.143 INFO:teuthology.run_tasks:Running task install... 2024-05-10T03:52:49.152 DEBUG:teuthology.task.install:project ceph 2024-05-10T03:52:49.152 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '50174015a7e79f951b988949abe236791f52d4da'}} 2024-05-10T03:52:49.152 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '50174015a7e79f951b988949abe236791f52d4da'} 2024-05-10T03:52:49.152 INFO:teuthology.task.install:Using flavor: default 2024-05-10T03:52:49.160 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-10T03:52:49.160 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-10T03:52:49.161 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'tag': None, 'wait_for_package': False} 2024-05-10T03:52:49.161 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:52:49.163 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'tag': None, 'wait_for_package': False} 2024-05-10T03:52:49.163 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:52:49.164 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'tag': None, 'wait_for_package': False} 2024-05-10T03:52:49.164 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:52:49.311 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/ 2024-05-10T03:52:49.311 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1330.g50174015 2024-05-10T03:52:49.315 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/ 2024-05-10T03:52:49.315 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1330.g50174015 2024-05-10T03:52:49.388 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/ 2024-05-10T03:52:49.388 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1330.g50174015 2024-05-10T03:52:49.530 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T03:52:49.530 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T03:52:49.530 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T03:52:49.536 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T03:52:49.536 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T03:52:49.536 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T03:52:49.564 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T03:52:49.564 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50174015a7e79f951b988949abe236791f52d4da/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T03:52:49.569 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T03:52:49.570 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50174015a7e79f951b988949abe236791f52d4da/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T03:52:49.621 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T03:52:49.622 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:52:49.622 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T03:52:49.635 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T03:52:49.642 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T03:52:49.650 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T03:52:49.650 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50174015a7e79f951b988949abe236791f52d4da/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T03:52:49.717 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T03:52:49.718 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T03:52:49.723 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T03:52:49.786 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-05-10T03:52:49.788 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-05-10T03:52:49.791 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2024-05-10T03:52:49.792 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2024-05-10T03:52:49.796 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T03:52:49.863 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-05-10T03:52:49.864 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-05-10T03:52:50.170 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2024-05-10T03:52:50.171 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2024-05-10T03:52:50.246 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T03:52:51.735 INFO:teuthology.orchestra.run.smithi150.stdout:61 files removed 2024-05-10T03:52:51.747 INFO:teuthology.orchestra.run.smithi181.stdout:61 files removed 2024-05-10T03:52:51.751 INFO:teuthology.orchestra.run.smithi161.stdout:61 files removed 2024-05-10T03:52:51.788 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T03:52:51.799 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T03:52:51.803 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T03:52:52.126 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T03:52:52.134 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2024-05-10T03:52:52.142 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2024-05-10T03:52:54.100 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 167 kB/s | 80 kB 00:00 2024-05-10T03:52:54.109 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 187 kB/s | 80 kB 00:00 2024-05-10T03:52:54.229 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 155 kB/s | 80 kB 00:00 2024-05-10T03:52:54.601 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 24 kB/s | 11 kB 00:00 2024-05-10T03:52:54.635 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 23 kB/s | 11 kB 00:00 2024-05-10T03:52:54.657 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-05-10T03:52:55.010 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 3.9 kB/s | 1.5 kB 00:00 2024-05-10T03:52:55.055 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 4.0 kB/s | 1.5 kB 00:00 2024-05-10T03:52:55.144 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 3.1 kB/s | 1.5 kB 00:00 2024-05-10T03:52:55.261 INFO:teuthology.orchestra.run.smithi150.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2024-05-10T03:52:55.331 INFO:teuthology.orchestra.run.smithi161.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2024-05-10T03:52:55.395 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2024-05-10T03:52:56.998 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 14 MB 00:01 2024-05-10T03:52:57.140 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 7.9 MB/s | 14 MB 00:01 2024-05-10T03:52:57.436 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 14 MB 00:02 2024-05-10T03:53:00.316 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 418 kB/s | 24 kB 00:00 2024-05-10T03:53:00.505 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 435 kB/s | 24 kB 00:00 2024-05-10T03:53:00.745 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 412 kB/s | 24 kB 00:00 2024-05-10T03:53:01.726 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 67 MB 00:01 2024-05-10T03:53:01.873 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 52 MB/s | 67 MB 00:01 2024-05-10T03:53:02.055 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 49 MB/s | 61 MB 00:01 2024-05-10T03:53:12.073 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-05-10T03:53:12.668 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 61 MB 00:01 2024-05-10T03:53:12.939 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-05-10T03:53:15.042 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 48 MB/s | 61 MB 00:01 2024-05-10T03:53:15.874 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 51 MB/s | 67 MB 00:01 2024-05-10T03:53:23.451 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 15 MB/s | 8.8 MB 00:00 2024-05-10T03:53:30.682 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T03:53:30.683 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T03:53:30.701 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-10T03:53:30.702 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-10T03:53:30.703 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-10T03:53:30.705 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-10T03:53:30.707 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-10T03:53:30.711 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-10T03:53:30.713 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-10T03:53:30.927 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-05-10T03:53:30.941 INFO:teuthology.orchestra.run.smithi161.stdout:=================================================================================================================================== 2024-05-10T03:53:30.942 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-05-10T03:53:30.942 INFO:teuthology.orchestra.run.smithi161.stdout:=================================================================================================================================== 2024-05-10T03:53:30.942 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-05-10T03:53:30.942 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-10T03:53:30.942 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-10T03:53:30.942 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:18.2.2-1330.g50174015.el8 ceph 6.4 k 2024-05-10T03:53:30.942 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.7 M 2024-05-10T03:53:30.942 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 829 k 2024-05-10T03:53:30.942 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1330.g50174015.el8 ceph 157 k 2024-05-10T03:53:30.943 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:18.2.2-1330.g50174015.el8 ceph 1.5 M 2024-05-10T03:53:30.943 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 138 k 2024-05-10T03:53:30.943 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 3.5 M 2024-05-10T03:53:30.943 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 7.4 M 2024-05-10T03:53:30.943 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 49 k 2024-05-10T03:53:30.943 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 14 M 2024-05-10T03:53:30.943 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:18.2.2-1330.g50174015.el8 ceph 49 M 2024-05-10T03:53:30.943 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 263 k 2024-05-10T03:53:30.943 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 173 k 2024-05-10T03:53:30.943 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 246 k 2024-05-10T03:53:30.943 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 18 k 2024-05-10T03:53:30.943 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-10T03:53:30.944 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-10T03:53:30.944 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-10T03:53:30.944 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 31 k 2024-05-10T03:53:30.944 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 756 k 2024-05-10T03:53:30.944 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T03:53:30.944 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-10T03:53:30.944 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-10T03:53:30.944 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-10T03:53:30.944 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-10T03:53:30.944 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-10T03:53:30.944 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-10T03:53:30.945 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:18.2.2-1330.g50174015.el8 ceph 197 k 2024-05-10T03:53:30.945 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:18.2.2-1330.g50174015.el8 ceph 371 k 2024-05-10T03:53:30.945 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 364 k 2024-05-10T03:53:30.945 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 107 k 2024-05-10T03:53:30.945 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 91 k 2024-05-10T03:53:30.945 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.3 M 2024-05-10T03:53:30.945 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 179 k 2024-05-10T03:53:30.945 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-10T03:53:30.946 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-05-10T03:53:30.946 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-10T03:53:30.946 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-10T03:53:30.946 INFO:teuthology.orchestra.run.smithi161.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-10T03:53:30.946 INFO:teuthology.orchestra.run.smithi161.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-10T03:53:30.946 INFO:teuthology.orchestra.run.smithi161.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-10T03:53:30.946 INFO:teuthology.orchestra.run.smithi161.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-10T03:53:30.946 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T03:53:30.946 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T03:53:30.946 INFO:teuthology.orchestra.run.smithi161.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-10T03:53:30.946 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-10T03:53:30.947 INFO:teuthology.orchestra.run.smithi161.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-10T03:53:30.947 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-10T03:53:30.947 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-10T03:53:30.947 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-10T03:53:30.947 INFO:teuthology.orchestra.run.smithi161.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-10T03:53:30.947 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-10T03:53:30.947 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-05-10T03:53:30.947 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-10T03:53:30.947 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 23 M 2024-05-10T03:53:30.947 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 24 k 2024-05-10T03:53:30.947 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:18.2.2-1330.g50174015.el8 ceph 2.2 M 2024-05-10T03:53:30.947 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 246 k 2024-05-10T03:53:30.947 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.1 M 2024-05-10T03:53:30.948 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:18.2.2-1330.g50174015.el8 ceph 19 M 2024-05-10T03:53:30.948 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 16 k 2024-05-10T03:53:30.948 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:18.2.2-1330.g50174015.el8 ceph 24 k 2024-05-10T03:53:30.948 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-10T03:53:30.948 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-10T03:53:30.948 INFO:teuthology.orchestra.run.smithi161.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-10T03:53:30.948 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-10T03:53:30.948 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-10T03:53:30.948 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T03:53:30.948 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-10T03:53:30.948 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-10T03:53:30.948 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-10T03:53:30.949 INFO:teuthology.orchestra.run.smithi161.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-10T03:53:30.949 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-10T03:53:30.949 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-10T03:53:30.949 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 461 k 2024-05-10T03:53:30.949 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-10T03:53:30.949 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 7.2 M 2024-05-10T03:53:30.949 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-10T03:53:30.949 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-10T03:53:30.949 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-10T03:53:30.949 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-10T03:53:30.949 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-10T03:53:30.949 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-10T03:53:30.949 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-10T03:53:30.950 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-10T03:53:30.950 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-10T03:53:30.950 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-10T03:53:30.950 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-10T03:53:30.950 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-10T03:53:30.950 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-10T03:53:30.950 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-10T03:53:30.950 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-10T03:53:30.950 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-10T03:53:30.950 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-10T03:53:30.950 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:18.2.2-1330.g50174015.el8 ceph 45 k 2024-05-10T03:53:30.950 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 129 k 2024-05-10T03:53:30.951 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-10T03:53:30.951 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-10T03:53:30.951 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-10T03:53:30.951 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-10T03:53:30.951 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-10T03:53:30.951 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-10T03:53:30.951 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-10T03:53:30.951 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-10T03:53:30.951 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-10T03:53:30.951 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-10T03:53:30.951 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-10T03:53:30.951 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T03:53:30.952 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-10T03:53:30.952 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-10T03:53:30.952 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-10T03:53:30.952 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T03:53:30.952 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-10T03:53:30.952 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-10T03:53:30.952 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T03:53:30.952 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-10T03:53:30.952 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-10T03:53:30.952 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-10T03:53:30.952 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-10T03:53:30.952 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-10T03:53:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-10T03:53:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-10T03:53:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-10T03:53:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-10T03:53:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-10T03:53:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-10T03:53:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-10T03:53:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T03:53:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-10T03:53:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-10T03:53:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-10T03:53:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-10T03:53:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-10T03:53:30.954 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-10T03:53:30.954 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-10T03:53:30.954 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-10T03:53:30.954 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-10T03:53:30.954 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-10T03:53:30.954 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-10T03:53:30.954 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-10T03:53:30.954 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-10T03:53:30.954 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-10T03:53:30.954 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-10T03:53:30.954 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-10T03:53:30.954 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-10T03:53:30.954 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-10T03:53:30.955 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling module streams: 2024-05-10T03:53:30.955 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql 10 2024-05-10T03:53:30.955 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T03:53:30.955 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-05-10T03:53:30.955 INFO:teuthology.orchestra.run.smithi161.stdout:=================================================================================================================================== 2024-05-10T03:53:30.955 INFO:teuthology.orchestra.run.smithi161.stdout:Install 130 Packages 2024-05-10T03:53:30.955 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 16 Packages 2024-05-10T03:53:30.955 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T03:53:30.956 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 221 M 2024-05-10T03:53:30.956 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-05-10T03:53:30.963 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T03:53:30.965 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T03:53:30.984 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-10T03:53:30.985 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-10T03:53:30.987 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-10T03:53:30.988 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-10T03:53:30.991 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-10T03:53:30.995 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-10T03:53:30.997 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-10T03:53:31.202 INFO:teuthology.orchestra.run.smithi161.stdout:(1/146): ceph-18.2.2-1330.g50174015.el8.x86_64. 35 kB/s | 6.4 kB 00:00 2024-05-10T03:53:31.205 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T03:53:31.206 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T03:53:31.219 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-05-10T03:53:31.224 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-10T03:53:31.226 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-10T03:53:31.227 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-10T03:53:31.229 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-10T03:53:31.231 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-10T03:53:31.233 INFO:teuthology.orchestra.run.smithi181.stdout:=================================================================================================================================== 2024-05-10T03:53:31.233 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-05-10T03:53:31.233 INFO:teuthology.orchestra.run.smithi181.stdout:=================================================================================================================================== 2024-05-10T03:53:31.233 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-05-10T03:53:31.233 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-10T03:53:31.233 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-10T03:53:31.233 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:18.2.2-1330.g50174015.el8 ceph 6.4 k 2024-05-10T03:53:31.233 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.7 M 2024-05-10T03:53:31.234 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 829 k 2024-05-10T03:53:31.234 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1330.g50174015.el8 ceph 157 k 2024-05-10T03:53:31.234 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:18.2.2-1330.g50174015.el8 ceph 1.5 M 2024-05-10T03:53:31.234 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 138 k 2024-05-10T03:53:31.234 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 3.5 M 2024-05-10T03:53:31.234 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 7.4 M 2024-05-10T03:53:31.234 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 49 k 2024-05-10T03:53:31.234 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 14 M 2024-05-10T03:53:31.234 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:18.2.2-1330.g50174015.el8 ceph 49 M 2024-05-10T03:53:31.234 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 263 k 2024-05-10T03:53:31.234 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 173 k 2024-05-10T03:53:31.234 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 246 k 2024-05-10T03:53:31.234 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 18 k 2024-05-10T03:53:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-10T03:53:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-10T03:53:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-10T03:53:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 31 k 2024-05-10T03:53:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 756 k 2024-05-10T03:53:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T03:53:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-10T03:53:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-10T03:53:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-10T03:53:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-10T03:53:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-10T03:53:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-10T03:53:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:18.2.2-1330.g50174015.el8 ceph 197 k 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:18.2.2-1330.g50174015.el8 ceph 371 k 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 364 k 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 107 k 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 91 k 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.3 M 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 179 k 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-10T03:53:31.236 INFO:teuthology.orchestra.run.smithi181.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-10T03:53:31.237 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T03:53:31.237 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T03:53:31.237 INFO:teuthology.orchestra.run.smithi181.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-10T03:53:31.237 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-10T03:53:31.237 INFO:teuthology.orchestra.run.smithi181.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-10T03:53:31.237 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-10T03:53:31.237 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-10T03:53:31.237 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-10T03:53:31.237 INFO:teuthology.orchestra.run.smithi181.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-10T03:53:31.237 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-10T03:53:31.237 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-05-10T03:53:31.237 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-10T03:53:31.237 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 23 M 2024-05-10T03:53:31.238 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 24 k 2024-05-10T03:53:31.238 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:18.2.2-1330.g50174015.el8 ceph 2.2 M 2024-05-10T03:53:31.238 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 246 k 2024-05-10T03:53:31.238 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.1 M 2024-05-10T03:53:31.238 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:18.2.2-1330.g50174015.el8 ceph 19 M 2024-05-10T03:53:31.238 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 16 k 2024-05-10T03:53:31.238 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:18.2.2-1330.g50174015.el8 ceph 24 k 2024-05-10T03:53:31.238 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-10T03:53:31.238 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-10T03:53:31.238 INFO:teuthology.orchestra.run.smithi181.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-10T03:53:31.238 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-10T03:53:31.238 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-10T03:53:31.238 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T03:53:31.239 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-10T03:53:31.239 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-10T03:53:31.239 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-10T03:53:31.239 INFO:teuthology.orchestra.run.smithi181.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-10T03:53:31.239 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-10T03:53:31.239 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-10T03:53:31.239 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 461 k 2024-05-10T03:53:31.239 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-10T03:53:31.239 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 7.2 M 2024-05-10T03:53:31.239 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-10T03:53:31.239 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-10T03:53:31.239 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-10T03:53:31.240 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-10T03:53:31.241 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:18.2.2-1330.g50174015.el8 ceph 45 k 2024-05-10T03:53:31.241 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 129 k 2024-05-10T03:53:31.241 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-10T03:53:31.241 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-10T03:53:31.241 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-10T03:53:31.241 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-10T03:53:31.241 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-10T03:53:31.241 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-10T03:53:31.241 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-10T03:53:31.241 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-10T03:53:31.241 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-10T03:53:31.241 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-10T03:53:31.241 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-10T03:53:31.242 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T03:53:31.242 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-10T03:53:31.242 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-10T03:53:31.242 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-10T03:53:31.242 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T03:53:31.242 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-10T03:53:31.242 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-10T03:53:31.242 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T03:53:31.242 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-10T03:53:31.242 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-10T03:53:31.242 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-10T03:53:31.242 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-10T03:53:31.242 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-10T03:53:31.243 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-10T03:53:31.243 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-10T03:53:31.243 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-10T03:53:31.243 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-10T03:53:31.243 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-10T03:53:31.243 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-10T03:53:31.243 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-10T03:53:31.243 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T03:53:31.243 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-10T03:53:31.243 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-10T03:53:31.243 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-10T03:53:31.243 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-10T03:53:31.244 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-10T03:53:31.245 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling module streams: 2024-05-10T03:53:31.245 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql 10 2024-05-10T03:53:31.245 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:53:31.245 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-05-10T03:53:31.245 INFO:teuthology.orchestra.run.smithi181.stdout:=================================================================================================================================== 2024-05-10T03:53:31.245 INFO:teuthology.orchestra.run.smithi181.stdout:Install 130 Packages 2024-05-10T03:53:31.245 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 16 Packages 2024-05-10T03:53:31.245 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:53:31.246 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-10T03:53:31.246 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-10T03:53:31.246 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 221 M 2024-05-10T03:53:31.246 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-05-10T03:53:31.416 INFO:teuthology.orchestra.run.smithi181.stdout:(1/146): ceph-18.2.2-1330.g50174015.el8.x86_64. 45 kB/s | 6.4 kB 00:00 2024-05-10T03:53:31.457 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-05-10T03:53:31.471 INFO:teuthology.orchestra.run.smithi150.stdout:=================================================================================================================================== 2024-05-10T03:53:31.471 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-05-10T03:53:31.471 INFO:teuthology.orchestra.run.smithi150.stdout:=================================================================================================================================== 2024-05-10T03:53:31.472 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-05-10T03:53:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-10T03:53:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-10T03:53:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:18.2.2-1330.g50174015.el8 ceph 6.4 k 2024-05-10T03:53:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.7 M 2024-05-10T03:53:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 829 k 2024-05-10T03:53:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1330.g50174015.el8 ceph 157 k 2024-05-10T03:53:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:18.2.2-1330.g50174015.el8 ceph 1.5 M 2024-05-10T03:53:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 138 k 2024-05-10T03:53:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 3.5 M 2024-05-10T03:53:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 7.4 M 2024-05-10T03:53:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 49 k 2024-05-10T03:53:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 14 M 2024-05-10T03:53:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:18.2.2-1330.g50174015.el8 ceph 49 M 2024-05-10T03:53:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 263 k 2024-05-10T03:53:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 173 k 2024-05-10T03:53:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 246 k 2024-05-10T03:53:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 18 k 2024-05-10T03:53:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-10T03:53:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-10T03:53:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-10T03:53:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 31 k 2024-05-10T03:53:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 756 k 2024-05-10T03:53:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T03:53:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-10T03:53:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-10T03:53:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-10T03:53:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-10T03:53:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-10T03:53:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-10T03:53:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:18.2.2-1330.g50174015.el8 ceph 197 k 2024-05-10T03:53:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:18.2.2-1330.g50174015.el8 ceph 371 k 2024-05-10T03:53:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 364 k 2024-05-10T03:53:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 107 k 2024-05-10T03:53:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 91 k 2024-05-10T03:53:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.3 M 2024-05-10T03:53:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 179 k 2024-05-10T03:53:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-10T03:53:31.475 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-05-10T03:53:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-10T03:53:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-10T03:53:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-10T03:53:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-10T03:53:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-10T03:53:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-10T03:53:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T03:53:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T03:53:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-10T03:53:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-10T03:53:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-10T03:53:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-10T03:53:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-10T03:53:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-10T03:53:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-10T03:53:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-10T03:53:31.476 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-05-10T03:53:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-10T03:53:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 23 M 2024-05-10T03:53:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 24 k 2024-05-10T03:53:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:18.2.2-1330.g50174015.el8 ceph 2.2 M 2024-05-10T03:53:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 246 k 2024-05-10T03:53:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.1 M 2024-05-10T03:53:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:18.2.2-1330.g50174015.el8 ceph 19 M 2024-05-10T03:53:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 16 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:18.2.2-1330.g50174015.el8 ceph 24 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-10T03:53:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 461 k 2024-05-10T03:53:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-10T03:53:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 7.2 M 2024-05-10T03:53:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-10T03:53:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-10T03:53:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-10T03:53:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-10T03:53:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-10T03:53:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-10T03:53:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-10T03:53:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-10T03:53:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-10T03:53:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-10T03:53:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-10T03:53:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-10T03:53:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-10T03:53:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-10T03:53:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-10T03:53:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-10T03:53:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-10T03:53:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:18.2.2-1330.g50174015.el8 ceph 45 k 2024-05-10T03:53:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 129 k 2024-05-10T03:53:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-10T03:53:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-10T03:53:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-10T03:53:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-10T03:53:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-10T03:53:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-10T03:53:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-10T03:53:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-10T03:53:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-10T03:53:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-10T03:53:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-10T03:53:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T03:53:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-10T03:53:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-10T03:53:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-10T03:53:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T03:53:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-10T03:53:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-10T03:53:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T03:53:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-10T03:53:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-10T03:53:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-10T03:53:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-10T03:53:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-10T03:53:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-10T03:53:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-10T03:53:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-10T03:53:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-10T03:53:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-10T03:53:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-10T03:53:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-10T03:53:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T03:53:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-10T03:53:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-10T03:53:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-10T03:53:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-10T03:53:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-10T03:53:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-10T03:53:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-10T03:53:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-10T03:53:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-10T03:53:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-10T03:53:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-10T03:53:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-10T03:53:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-10T03:53:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-10T03:53:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-10T03:53:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-10T03:53:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-10T03:53:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-10T03:53:31.484 INFO:teuthology.orchestra.run.smithi150.stdout:Enabling module streams: 2024-05-10T03:53:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql 10 2024-05-10T03:53:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:53:31.484 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-05-10T03:53:31.484 INFO:teuthology.orchestra.run.smithi150.stdout:=================================================================================================================================== 2024-05-10T03:53:31.484 INFO:teuthology.orchestra.run.smithi150.stdout:Install 130 Packages 2024-05-10T03:53:31.484 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 16 Packages 2024-05-10T03:53:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:53:31.485 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 221 M 2024-05-10T03:53:31.485 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-05-10T03:53:31.511 INFO:teuthology.orchestra.run.smithi161.stdout:(2/146): ceph-fuse-18.2.2-1330.g50174015.el8.x8 2.6 MB/s | 829 kB 00:00 2024-05-10T03:53:31.611 INFO:teuthology.orchestra.run.smithi161.stdout:(3/146): ceph-immutable-object-cache-18.2.2-133 1.5 MB/s | 157 kB 00:00 2024-05-10T03:53:31.982 INFO:teuthology.orchestra.run.smithi150.stdout:(1/146): ceph-18.2.2-1330.g50174015.el8.x86_64. 23 kB/s | 6.4 kB 00:00 2024-05-10T03:53:31.995 INFO:teuthology.orchestra.run.smithi161.stdout:(4/146): ceph-mds-18.2.2-1330.g50174015.el8.x86 5.6 MB/s | 2.2 MB 00:00 2024-05-10T03:53:32.091 INFO:teuthology.orchestra.run.smithi181.stdout:(2/146): ceph-fuse-18.2.2-1330.g50174015.el8.x8 1.2 MB/s | 829 kB 00:00 2024-05-10T03:53:32.253 INFO:teuthology.orchestra.run.smithi161.stdout:(5/146): ceph-base-18.2.2-1330.g50174015.el8.x8 4.6 MB/s | 5.7 MB 00:01 2024-05-10T03:53:32.275 INFO:teuthology.orchestra.run.smithi181.stdout:(3/146): ceph-immutable-object-cache-18.2.2-133 857 kB/s | 157 kB 00:00 2024-05-10T03:53:32.287 INFO:teuthology.orchestra.run.smithi161.stdout:(6/146): ceph-mgr-18.2.2-1330.g50174015.el8.x86 5.2 MB/s | 1.5 MB 00:00 2024-05-10T03:53:33.196 INFO:teuthology.orchestra.run.smithi161.stdout:(7/146): ceph-mon-18.2.2-1330.g50174015.el8.x86 5.4 MB/s | 5.1 MB 00:00 2024-05-10T03:53:33.249 INFO:teuthology.orchestra.run.smithi150.stdout:(2/146): ceph-fuse-18.2.2-1330.g50174015.el8.x8 655 kB/s | 829 kB 00:01 2024-05-10T03:53:33.533 INFO:teuthology.orchestra.run.smithi150.stdout:(3/146): ceph-immutable-object-cache-18.2.2-133 554 kB/s | 157 kB 00:00 2024-05-10T03:53:33.633 INFO:teuthology.orchestra.run.smithi181.stdout:(4/146): ceph-mds-18.2.2-1330.g50174015.el8.x86 1.6 MB/s | 2.2 MB 00:01 2024-05-10T03:53:34.121 INFO:teuthology.orchestra.run.smithi161.stdout:(8/146): ceph-common-18.2.2-1330.g50174015.el8. 7.4 MB/s | 23 MB 00:03 2024-05-10T03:53:34.221 INFO:teuthology.orchestra.run.smithi161.stdout:(9/146): ceph-selinux-18.2.2-1330.g50174015.el8 244 kB/s | 24 kB 00:00 2024-05-10T03:53:34.408 INFO:teuthology.orchestra.run.smithi150.stdout:(4/146): ceph-base-18.2.2-1330.g50174015.el8.x8 2.1 MB/s | 5.7 MB 00:02 2024-05-10T03:53:34.517 INFO:teuthology.orchestra.run.smithi181.stdout:(5/146): ceph-mgr-18.2.2-1330.g50174015.el8.x86 1.7 MB/s | 1.5 MB 00:00 2024-05-10T03:53:35.029 INFO:teuthology.orchestra.run.smithi161.stdout:(10/146): ceph-radosgw-18.2.2-1330.g50174015.el 7.9 MB/s | 14 MB 00:01 2024-05-10T03:53:35.130 INFO:teuthology.orchestra.run.smithi161.stdout:(11/146): cephfs-mirror-18.2.2-1330.g50174015.e 2.4 MB/s | 246 kB 00:00 2024-05-10T03:53:35.263 INFO:teuthology.orchestra.run.smithi161.stdout:(12/146): libcephfs-devel-18.2.2-1330.g50174015 232 kB/s | 31 kB 00:00 2024-05-10T03:53:35.308 INFO:teuthology.orchestra.run.smithi150.stdout:(5/146): ceph-mgr-18.2.2-1330.g50174015.el8.x86 1.7 MB/s | 1.5 MB 00:00 2024-05-10T03:53:35.389 INFO:teuthology.orchestra.run.smithi161.stdout:(13/146): libcephfs2-18.2.2-1330.g50174015.el8. 5.9 MB/s | 756 kB 00:00 2024-05-10T03:53:35.489 INFO:teuthology.orchestra.run.smithi161.stdout:(14/146): libcephsqlite-18.2.2-1330.g50174015.e 1.4 MB/s | 139 kB 00:00 2024-05-10T03:53:35.581 INFO:teuthology.orchestra.run.smithi161.stdout:(15/146): librados-devel-18.2.2-1330.g50174015. 1.5 MB/s | 139 kB 00:00 2024-05-10T03:53:35.716 INFO:teuthology.orchestra.run.smithi161.stdout:(16/146): libradosstriper1-18.2.2-1330.g5017401 3.3 MB/s | 461 kB 00:00 2024-05-10T03:53:35.875 INFO:teuthology.orchestra.run.smithi150.stdout:(6/146): ceph-common-18.2.2-1330.g50174015.el8. 5.5 MB/s | 23 MB 00:04 2024-05-10T03:53:36.000 INFO:teuthology.orchestra.run.smithi150.stdout:(7/146): ceph-mds-18.2.2-1330.g50174015.el8.x86 897 kB/s | 2.2 MB 00:02 2024-05-10T03:53:36.350 INFO:teuthology.orchestra.run.smithi181.stdout:(6/146): ceph-base-18.2.2-1330.g50174015.el8.x8 1.1 MB/s | 5.7 MB 00:05 2024-05-10T03:53:36.778 INFO:teuthology.orchestra.run.smithi161.stdout:(17/146): librgw2-18.2.2-1330.g50174015.el8.x86 6.8 MB/s | 7.2 MB 00:01 2024-05-10T03:53:36.873 INFO:teuthology.orchestra.run.smithi161.stdout:(18/146): python3-ceph-argparse-18.2.2-1330.g50 470 kB/s | 45 kB 00:00 2024-05-10T03:53:36.951 INFO:teuthology.orchestra.run.smithi181.stdout:(7/146): ceph-mon-18.2.2-1330.g50174015.el8.x86 2.1 MB/s | 5.1 MB 00:02 2024-05-10T03:53:36.965 INFO:teuthology.orchestra.run.smithi161.stdout:(19/146): python3-ceph-common-18.2.2-1330.g5017 1.4 MB/s | 129 kB 00:00 2024-05-10T03:53:37.073 INFO:teuthology.orchestra.run.smithi161.stdout:(20/146): python3-cephfs-18.2.2-1330.g50174015. 1.8 MB/s | 197 kB 00:00 2024-05-10T03:53:37.190 INFO:teuthology.orchestra.run.smithi161.stdout:(21/146): python3-rados-18.2.2-1330.g50174015.e 3.1 MB/s | 371 kB 00:00 2024-05-10T03:53:37.299 INFO:teuthology.orchestra.run.smithi161.stdout:(22/146): python3-rbd-18.2.2-1330.g50174015.el8 3.3 MB/s | 364 kB 00:00 2024-05-10T03:53:37.399 INFO:teuthology.orchestra.run.smithi161.stdout:(23/146): python3-rgw-18.2.2-1330.g50174015.el8 1.0 MB/s | 107 kB 00:00 2024-05-10T03:53:37.499 INFO:teuthology.orchestra.run.smithi161.stdout:(24/146): rbd-fuse-18.2.2-1330.g50174015.el8.x8 915 kB/s | 91 kB 00:00 2024-05-10T03:53:37.917 INFO:teuthology.orchestra.run.smithi150.stdout:(8/146): ceph-mon-18.2.2-1330.g50174015.el8.x86 2.0 MB/s | 5.1 MB 00:02 2024-05-10T03:53:38.026 INFO:teuthology.orchestra.run.smithi150.stdout:(9/146): ceph-selinux-18.2.2-1330.g50174015.el8 225 kB/s | 24 kB 00:00 2024-05-10T03:53:38.074 INFO:teuthology.orchestra.run.smithi161.stdout:(25/146): rbd-mirror-18.2.2-1330.g50174015.el8. 5.8 MB/s | 3.3 MB 00:00 2024-05-10T03:53:38.208 INFO:teuthology.orchestra.run.smithi161.stdout:(26/146): rbd-nbd-18.2.2-1330.g50174015.el8.x86 1.3 MB/s | 179 kB 00:00 2024-05-10T03:53:38.300 INFO:teuthology.orchestra.run.smithi161.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1330.g 265 kB/s | 24 kB 00:00 2024-05-10T03:53:38.417 INFO:teuthology.orchestra.run.smithi161.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1330.g5017401 1.2 MB/s | 138 kB 00:00 2024-05-10T03:53:38.992 INFO:teuthology.orchestra.run.smithi161.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1330.g50174 6.1 MB/s | 3.5 MB 00:00 2024-05-10T03:53:39.160 INFO:teuthology.orchestra.run.smithi161.stdout:(30/146): ceph-osd-18.2.2-1330.g50174015.el8.x8 2.7 MB/s | 19 MB 00:06 2024-05-10T03:53:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1330.g50 1.2 MB/s | 246 kB 00:00 2024-05-10T03:53:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:(32/146): ceph-mgr-rook-18.2.2-1330.g50174015.e 537 kB/s | 49 kB 00:00 2024-05-10T03:53:39.551 INFO:teuthology.orchestra.run.smithi150.stdout:(10/146): ceph-osd-18.2.2-1330.g50174015.el8.x8 5.1 MB/s | 19 MB 00:03 2024-05-10T03:53:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1330.g5 131 kB/s | 16 kB 00:00 2024-05-10T03:53:39.618 INFO:teuthology.orchestra.run.smithi150.stdout:(11/146): cephfs-mirror-18.2.2-1330.g50174015.e 3.6 MB/s | 246 kB 00:00 2024-05-10T03:53:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:(12/146): libcephfs-devel-18.2.2-1330.g50174015 414 kB/s | 31 kB 00:00 2024-05-10T03:53:39.777 INFO:teuthology.orchestra.run.smithi161.stdout:(34/146): ceph-volume-18.2.2-1330.g50174015.el8 1.3 MB/s | 263 kB 00:00 2024-05-10T03:53:39.885 INFO:teuthology.orchestra.run.smithi150.stdout:(13/146): libcephfs2-18.2.2-1330.g50174015.el8. 3.9 MB/s | 756 kB 00:00 2024-05-10T03:53:39.886 INFO:teuthology.orchestra.run.smithi161.stdout:(35/146): cephadm-18.2.2-1330.g50174015.el8.noa 1.6 MB/s | 173 kB 00:00 2024-05-10T03:53:39.960 INFO:teuthology.orchestra.run.smithi150.stdout:(14/146): libcephsqlite-18.2.2-1330.g50174015.e 1.8 MB/s | 139 kB 00:00 2024-05-10T03:53:40.003 INFO:teuthology.orchestra.run.smithi161.stdout:(36/146): cephfs-top-18.2.2-1330.g50174015.el8. 155 kB/s | 18 kB 00:00 2024-05-10T03:53:40.027 INFO:teuthology.orchestra.run.smithi150.stdout:(15/146): librados-devel-18.2.2-1330.g50174015. 2.0 MB/s | 139 kB 00:00 2024-05-10T03:53:40.086 INFO:teuthology.orchestra.run.smithi161.stdout:(37/146): ceph-mgr-diskprediction-local-18.2.2- 6.7 MB/s | 7.4 MB 00:01 2024-05-10T03:53:40.127 INFO:teuthology.orchestra.run.smithi150.stdout:(16/146): libradosstriper1-18.2.2-1330.g5017401 4.5 MB/s | 461 kB 00:00 2024-05-10T03:53:40.587 INFO:teuthology.orchestra.run.smithi161.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 118 kB/s | 59 kB 00:00 2024-05-10T03:53:40.620 INFO:teuthology.orchestra.run.smithi161.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 496 kB/s | 306 kB 00:00 2024-05-10T03:53:40.776 INFO:teuthology.orchestra.run.smithi161.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 398 kB/s | 75 kB 00:00 2024-05-10T03:53:40.784 INFO:teuthology.orchestra.run.smithi181.stdout:(8/146): ceph-common-18.2.2-1330.g50174015.el8. 2.4 MB/s | 23 MB 00:09 2024-05-10T03:53:40.812 INFO:teuthology.orchestra.run.smithi161.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.4 MB/s | 476 kB 00:00 2024-05-10T03:53:40.859 INFO:teuthology.orchestra.run.smithi181.stdout:(9/146): ceph-selinux-18.2.2-1330.g50174015.el8 326 kB/s | 24 kB 00:00 2024-05-10T03:53:40.863 INFO:teuthology.orchestra.run.smithi161.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 503 kB/s | 44 kB 00:00 2024-05-10T03:53:40.913 INFO:teuthology.orchestra.run.smithi161.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2024-05-10T03:53:40.955 INFO:teuthology.orchestra.run.smithi161.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 359 kB/s | 33 kB 00:00 2024-05-10T03:53:41.005 INFO:teuthology.orchestra.run.smithi161.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 177 kB/s | 16 kB 00:00 2024-05-10T03:53:41.137 INFO:teuthology.orchestra.run.smithi161.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.9 MB/s | 384 kB 00:00 2024-05-10T03:53:41.164 INFO:teuthology.orchestra.run.smithi161.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 827 kB/s | 173 kB 00:00 2024-05-10T03:53:41.211 INFO:teuthology.orchestra.run.smithi150.stdout:(17/146): librgw2-18.2.2-1330.g50174015.el8.x86 6.7 MB/s | 7.2 MB 00:01 2024-05-10T03:53:41.223 INFO:teuthology.orchestra.run.smithi161.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 951 kB/s | 81 kB 00:00 2024-05-10T03:53:41.248 INFO:teuthology.orchestra.run.smithi161.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2024-05-10T03:53:41.278 INFO:teuthology.orchestra.run.smithi150.stdout:(18/146): python3-ceph-argparse-18.2.2-1330.g50 670 kB/s | 45 kB 00:00 2024-05-10T03:53:41.307 INFO:teuthology.orchestra.run.smithi161.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2024-05-10T03:53:41.345 INFO:teuthology.orchestra.run.smithi150.stdout:(19/146): python3-ceph-common-18.2.2-1330.g5017 1.9 MB/s | 129 kB 00:00 2024-05-10T03:53:41.412 INFO:teuthology.orchestra.run.smithi150.stdout:(20/146): python3-cephfs-18.2.2-1330.g50174015. 2.9 MB/s | 197 kB 00:00 2024-05-10T03:53:41.487 INFO:teuthology.orchestra.run.smithi150.stdout:(21/146): python3-rados-18.2.2-1330.g50174015.e 4.9 MB/s | 371 kB 00:00 2024-05-10T03:53:41.571 INFO:teuthology.orchestra.run.smithi150.stdout:(22/146): python3-rbd-18.2.2-1330.g50174015.el8 4.3 MB/s | 364 kB 00:00 2024-05-10T03:53:41.621 INFO:teuthology.orchestra.run.smithi150.stdout:(23/146): python3-rgw-18.2.2-1330.g50174015.el8 2.1 MB/s | 107 kB 00:00 2024-05-10T03:53:41.689 INFO:teuthology.orchestra.run.smithi150.stdout:(24/146): rbd-fuse-18.2.2-1330.g50174015.el8.x8 1.3 MB/s | 91 kB 00:00 2024-05-10T03:53:41.790 INFO:teuthology.orchestra.run.smithi161.stdout:(51/146): ceph-test-18.2.2-1330.g50174015.el8.x 6.5 MB/s | 49 MB 00:07 2024-05-10T03:53:41.807 INFO:teuthology.orchestra.run.smithi161.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 97 kB/s | 49 kB 00:00 2024-05-10T03:53:41.874 INFO:teuthology.orchestra.run.smithi161.stdout:(53/146): python3-kubernetes-11.0.0-6.el8.noarc 1.9 MB/s | 1.2 MB 00:00 2024-05-10T03:53:41.891 INFO:teuthology.orchestra.run.smithi161.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 721 kB/s | 60 kB 00:00 2024-05-10T03:53:41.958 INFO:teuthology.orchestra.run.smithi161.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2024-05-10T03:53:41.976 INFO:teuthology.orchestra.run.smithi161.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 187 kB/s | 16 kB 00:00 2024-05-10T03:53:42.017 INFO:teuthology.orchestra.run.smithi181.stdout:(10/146): ceph-radosgw-18.2.2-1330.g50174015.el 2.9 MB/s | 14 MB 00:05 2024-05-10T03:53:42.050 INFO:teuthology.orchestra.run.smithi161.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 366 kB/s | 34 kB 00:00 2024-05-10T03:53:42.055 INFO:teuthology.orchestra.run.smithi150.stdout:(25/146): rbd-mirror-18.2.2-1330.g50174015.el8. 9.1 MB/s | 3.3 MB 00:00 2024-05-10T03:53:42.076 INFO:teuthology.orchestra.run.smithi161.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2024-05-10T03:53:42.084 INFO:teuthology.orchestra.run.smithi181.stdout:(11/146): cephfs-mirror-18.2.2-1330.g50174015.e 3.6 MB/s | 246 kB 00:00 2024-05-10T03:53:42.105 INFO:teuthology.orchestra.run.smithi150.stdout:(26/146): rbd-nbd-18.2.2-1330.g50174015.el8.x86 3.5 MB/s | 179 kB 00:00 2024-05-10T03:53:42.109 INFO:teuthology.orchestra.run.smithi161.stdout:(59/146): python3-more-itertools-7.2.0-3.el8.no 184 kB/s | 59 kB 00:00 2024-05-10T03:53:42.126 INFO:teuthology.orchestra.run.smithi181.stdout:(12/146): libcephfs-devel-18.2.2-1330.g50174015 744 kB/s | 31 kB 00:00 2024-05-10T03:53:42.135 INFO:teuthology.orchestra.run.smithi161.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 763 kB/s | 64 kB 00:00 2024-05-10T03:53:42.147 INFO:teuthology.orchestra.run.smithi150.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1330.g 584 kB/s | 24 kB 00:00 2024-05-10T03:53:42.168 INFO:teuthology.orchestra.run.smithi161.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 203 kB/s | 19 kB 00:00 2024-05-10T03:53:42.227 INFO:teuthology.orchestra.run.smithi161.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 204 kB/s | 24 kB 00:00 2024-05-10T03:53:42.252 INFO:teuthology.orchestra.run.smithi161.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2024-05-10T03:53:42.277 INFO:teuthology.orchestra.run.smithi161.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 252 kB/s | 27 kB 00:00 2024-05-10T03:53:42.310 INFO:teuthology.orchestra.run.smithi181.stdout:(13/146): libcephfs2-18.2.2-1330.g50174015.el8. 4.0 MB/s | 756 kB 00:00 2024-05-10T03:53:42.344 INFO:teuthology.orchestra.run.smithi161.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2024-05-10T03:53:42.360 INFO:teuthology.orchestra.run.smithi181.stdout:(14/146): libcephsqlite-18.2.2-1330.g50174015.e 2.7 MB/s | 139 kB 00:00 2024-05-10T03:53:42.369 INFO:teuthology.orchestra.run.smithi161.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 667 kB/s | 61 kB 00:00 2024-05-10T03:53:42.410 INFO:teuthology.orchestra.run.smithi181.stdout:(15/146): librados-devel-18.2.2-1330.g50174015. 2.7 MB/s | 139 kB 00:00 2024-05-10T03:53:42.428 INFO:teuthology.orchestra.run.smithi161.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2024-05-10T03:53:42.462 INFO:teuthology.orchestra.run.smithi161.stdout:(68/146): python3-waitress-1.4.3-1.el8.noarch.r 1.1 MB/s | 255 kB 00:00 2024-05-10T03:53:42.465 INFO:teuthology.orchestra.run.smithi150.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1330.g5017401 436 kB/s | 138 kB 00:00 2024-05-10T03:53:42.487 INFO:teuthology.orchestra.run.smithi161.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2024-05-10T03:53:42.502 INFO:teuthology.orchestra.run.smithi181.stdout:(16/146): libradosstriper1-18.2.2-1330.g5017401 4.9 MB/s | 461 kB 00:00 2024-05-10T03:53:42.504 INFO:teuthology.orchestra.run.smithi161.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2024-05-10T03:53:42.629 INFO:teuthology.orchestra.run.smithi161.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 126 kB/s | 18 kB 00:00 2024-05-10T03:53:42.671 INFO:teuthology.orchestra.run.smithi161.stdout:(72/146): thrift-0.13.0-2.el8.x86_64.rpm 7.2 MB/s | 1.7 MB 00:00 2024-05-10T03:53:42.696 INFO:teuthology.orchestra.run.smithi161.stdout:(73/146): python3-jwt-1.6.1-2.el8.noarch.rpm 228 kB/s | 43 kB 00:00 2024-05-10T03:53:42.772 INFO:teuthology.orchestra.run.smithi161.stdout:(74/146): python3-oauthlib-2.1.0-1.el8.noarch.r 1.1 MB/s | 155 kB 00:00 2024-05-10T03:53:42.797 INFO:teuthology.orchestra.run.smithi161.stdout:(75/146): python3-requests-oauthlib-1.0.0-1.el8 451 kB/s | 43 kB 00:00 2024-05-10T03:53:42.822 INFO:teuthology.orchestra.run.smithi161.stdout:(76/146): python3-pycparser-2.14-14.el8.noarch. 724 kB/s | 109 kB 00:00 2024-05-10T03:53:42.873 INFO:teuthology.orchestra.run.smithi161.stdout:(77/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 584 kB/s | 44 kB 00:00 2024-05-10T03:53:42.918 INFO:teuthology.orchestra.run.smithi161.stdout:(78/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 128 kB/s | 19 kB 00:00 2024-05-10T03:53:42.956 INFO:teuthology.orchestra.run.smithi150.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1330.g50174 7.2 MB/s | 3.5 MB 00:00 2024-05-10T03:53:42.957 INFO:teuthology.orchestra.run.smithi161.stdout:(79/146): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2024-05-10T03:53:42.982 INFO:teuthology.orchestra.run.smithi161.stdout:(80/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2024-05-10T03:53:43.049 INFO:teuthology.orchestra.run.smithi161.stdout:(81/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2024-05-10T03:53:43.099 INFO:teuthology.orchestra.run.smithi161.stdout:(82/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.4 MB/s | 200 kB 00:00 2024-05-10T03:53:43.119 INFO:teuthology.orchestra.run.smithi181.stdout:(17/146): ceph-osd-18.2.2-1330.g50174015.el8.x8 2.8 MB/s | 19 MB 00:06 2024-05-10T03:53:43.132 INFO:teuthology.orchestra.run.smithi161.stdout:(83/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 319 kB/s | 48 kB 00:00 2024-05-10T03:53:43.178 INFO:teuthology.orchestra.run.smithi181.stdout:(18/146): python3-ceph-argparse-18.2.2-1330.g50 771 kB/s | 45 kB 00:00 2024-05-10T03:53:43.183 INFO:teuthology.orchestra.run.smithi161.stdout:(84/146): libsepol-devel-2.9-3.el8.x86_64.rpm 651 kB/s | 87 kB 00:00 2024-05-10T03:53:43.233 INFO:teuthology.orchestra.run.smithi161.stdout:(85/146): libcom_err-devel-1.45.6-4.el8.x86_64. 385 kB/s | 39 kB 00:00 2024-05-10T03:53:43.270 INFO:teuthology.orchestra.run.smithi181.stdout:(19/146): python3-ceph-common-18.2.2-1330.g5017 1.4 MB/s | 129 kB 00:00 2024-05-10T03:53:43.287 INFO:teuthology.orchestra.run.smithi161.stdout:(86/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 306 kB/s | 58 kB 00:00 2024-05-10T03:53:43.359 INFO:teuthology.orchestra.run.smithi161.stdout:(87/146): libgfortran-8.5.0-20.el8.x86_64.rpm 5.0 MB/s | 645 kB 00:00 2024-05-10T03:53:43.370 INFO:teuthology.orchestra.run.smithi181.stdout:(20/146): python3-cephfs-18.2.2-1330.g50174015. 1.9 MB/s | 197 kB 00:00 2024-05-10T03:53:43.400 INFO:teuthology.orchestra.run.smithi161.stdout:(88/146): fuse-2.9.7-17.el8.x86_64.rpm 382 kB/s | 83 kB 00:00 2024-05-10T03:53:43.476 INFO:teuthology.orchestra.run.smithi161.stdout:(89/146): libquadmath-8.5.0-20.el8.x86_64.rpm 918 kB/s | 172 kB 00:00 2024-05-10T03:53:43.509 INFO:teuthology.orchestra.run.smithi161.stdout:(90/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 667 kB/s | 73 kB 00:00 2024-05-10T03:53:43.512 INFO:teuthology.orchestra.run.smithi181.stdout:(21/146): python3-rados-18.2.2-1330.g50174015.e 2.6 MB/s | 371 kB 00:00 2024-05-10T03:53:43.535 INFO:teuthology.orchestra.run.smithi161.stdout:(91/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.3 MB/s | 238 kB 00:00 2024-05-10T03:53:43.585 INFO:teuthology.orchestra.run.smithi161.stdout:(92/146): elfutils-libelf-devel-0.189-3.el8.x86 566 kB/s | 62 kB 00:00 2024-05-10T03:53:43.612 INFO:teuthology.orchestra.run.smithi181.stdout:(22/146): python3-rbd-18.2.2-1330.g50174015.el8 3.6 MB/s | 364 kB 00:00 2024-05-10T03:53:43.662 INFO:teuthology.orchestra.run.smithi181.stdout:(23/146): python3-rgw-18.2.2-1330.g50174015.el8 2.1 MB/s | 107 kB 00:00 2024-05-10T03:53:43.685 INFO:teuthology.orchestra.run.smithi161.stdout:(93/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 3.1 MB/s | 562 kB 00:00 2024-05-10T03:53:43.690 INFO:teuthology.orchestra.run.smithi150.stdout:(30/146): ceph-radosgw-18.2.2-1330.g50174015.el 1.9 MB/s | 14 MB 00:07 2024-05-10T03:53:43.713 INFO:teuthology.orchestra.run.smithi181.stdout:(24/146): rbd-fuse-18.2.2-1330.g50174015.el8.x8 1.8 MB/s | 91 kB 00:00 2024-05-10T03:53:43.727 INFO:teuthology.orchestra.run.smithi161.stdout:(94/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 332 kB/s | 47 kB 00:00 2024-05-10T03:53:43.786 INFO:teuthology.orchestra.run.smithi161.stdout:(95/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 9.3 MB/s | 2.3 MB 00:00 2024-05-10T03:53:43.844 INFO:teuthology.orchestra.run.smithi161.stdout:(96/146): python3-cryptography-3.2.1-7.el8_9.x8 3.4 MB/s | 559 kB 00:00 2024-05-10T03:53:43.865 INFO:teuthology.orchestra.run.smithi150.stdout:(31/146): ceph-mgr-diskprediction-local-18.2.2- 8.1 MB/s | 7.4 MB 00:00 2024-05-10T03:53:43.878 INFO:teuthology.orchestra.run.smithi161.stdout:(97/146): NetworkManager-initscripts-updown-1.4 958 kB/s | 144 kB 00:00 2024-05-10T03:53:43.890 INFO:teuthology.orchestra.run.smithi150.stdout:(32/146): ceph-mgr-modules-core-18.2.2-1330.g50 1.2 MB/s | 246 kB 00:00 2024-05-10T03:53:43.906 INFO:teuthology.orchestra.run.smithi161.stdout:(98/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 852 kB/s | 100 kB 00:00 2024-05-10T03:53:43.915 INFO:teuthology.orchestra.run.smithi150.stdout:(33/146): ceph-mgr-rook-18.2.2-1330.g50174015.e 989 kB/s | 49 kB 00:00 2024-05-10T03:53:43.928 INFO:teuthology.orchestra.run.smithi161.stdout:(99/146): python3-prettytable-0.7.2-14.el8.noar 878 kB/s | 44 kB 00:00 2024-05-10T03:53:43.954 INFO:teuthology.orchestra.run.smithi161.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 183 kB/s | 20 kB 00:00 2024-05-10T03:53:43.957 INFO:teuthology.orchestra.run.smithi150.stdout:(34/146): ceph-prometheus-alerts-18.2.2-1330.g5 244 kB/s | 16 kB 00:00 2024-05-10T03:53:43.983 INFO:teuthology.orchestra.run.smithi150.stdout:(35/146): ceph-volume-18.2.2-1330.g50174015.el8 3.8 MB/s | 263 kB 00:00 2024-05-10T03:53:44.020 INFO:teuthology.orchestra.run.smithi161.stdout:(101/146): python3-pyasn1-modules-0.3.7-6.el8.n 939 kB/s | 110 kB 00:00 2024-05-10T03:53:44.033 INFO:teuthology.orchestra.run.smithi150.stdout:(36/146): cephadm-18.2.2-1330.g50174015.el8.noa 2.2 MB/s | 173 kB 00:00 2024-05-10T03:53:44.046 INFO:teuthology.orchestra.run.smithi161.stdout:(102/146): python3-jsonpatch-1.21-2.el8.noarch. 226 kB/s | 27 kB 00:00 2024-05-10T03:53:44.058 INFO:teuthology.orchestra.run.smithi150.stdout:(37/146): cephfs-top-18.2.2-1330.g50174015.el8. 241 kB/s | 18 kB 00:00 2024-05-10T03:53:44.080 INFO:teuthology.orchestra.run.smithi181.stdout:(25/146): librgw2-18.2.2-1330.g50174015.el8.x86 4.6 MB/s | 7.2 MB 00:01 2024-05-10T03:53:44.121 INFO:teuthology.orchestra.run.smithi161.stdout:(103/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 756 kB/s | 126 kB 00:00 2024-05-10T03:53:44.138 INFO:teuthology.orchestra.run.smithi181.stdout:(26/146): rbd-nbd-18.2.2-1330.g50174015.el8.x86 3.0 MB/s | 179 kB 00:00 2024-05-10T03:53:44.188 INFO:teuthology.orchestra.run.smithi161.stdout:(104/146): bison-3.0.4-10.el8.x86_64.rpm 4.7 MB/s | 688 kB 00:00 2024-05-10T03:53:44.206 INFO:teuthology.orchestra.run.smithi181.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1330.g 364 kB/s | 24 kB 00:00 2024-05-10T03:53:44.225 INFO:teuthology.orchestra.run.smithi150.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 353 kB/s | 59 kB 00:00 2024-05-10T03:53:44.263 INFO:teuthology.orchestra.run.smithi161.stdout:(105/146): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2024-05-10T03:53:44.264 INFO:teuthology.orchestra.run.smithi181.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1330.g5017401 2.3 MB/s | 138 kB 00:00 2024-05-10T03:53:44.301 INFO:teuthology.orchestra.run.smithi150.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2024-05-10T03:53:44.314 INFO:teuthology.orchestra.run.smithi161.stdout:(106/146): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2024-05-10T03:53:44.323 INFO:teuthology.orchestra.run.smithi181.stdout:(29/146): rbd-mirror-18.2.2-1330.g50174015.el8. 5.4 MB/s | 3.3 MB 00:00 2024-05-10T03:53:44.326 INFO:teuthology.orchestra.run.smithi150.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 749 kB/s | 75 kB 00:00 2024-05-10T03:53:44.364 INFO:teuthology.orchestra.run.smithi161.stdout:(107/146): python3-werkzeug-0.12.2-4.el8.noarch 1.3 MB/s | 457 kB 00:00 2024-05-10T03:53:44.393 INFO:teuthology.orchestra.run.smithi150.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 655 kB/s | 44 kB 00:00 2024-05-10T03:53:44.397 INFO:teuthology.orchestra.run.smithi161.stdout:(108/146): python3-markupsafe-0.23-19.el8.x86_6 290 kB/s | 39 kB 00:00 2024-05-10T03:53:44.418 INFO:teuthology.orchestra.run.smithi150.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2024-05-10T03:53:44.464 INFO:teuthology.orchestra.run.smithi161.stdout:(109/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.7 MB/s | 259 kB 00:00 2024-05-10T03:53:44.477 INFO:teuthology.orchestra.run.smithi150.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 565 kB/s | 33 kB 00:00 2024-05-10T03:53:44.490 INFO:teuthology.orchestra.run.smithi161.stdout:(110/146): uuid-1.6.2-43.el8.x86_64.rpm 510 kB/s | 64 kB 00:00 2024-05-10T03:53:44.502 INFO:teuthology.orchestra.run.smithi150.stdout:(44/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2024-05-10T03:53:44.536 INFO:teuthology.orchestra.run.smithi150.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 279 kB/s | 16 kB 00:00 2024-05-10T03:53:44.556 INFO:teuthology.orchestra.run.smithi161.stdout:(111/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 652 kB/s | 103 kB 00:00 2024-05-10T03:53:44.569 INFO:teuthology.orchestra.run.smithi150.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2024-05-10T03:53:44.611 INFO:teuthology.orchestra.run.smithi150.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 5.0 MB/s | 384 kB 00:00 2024-05-10T03:53:44.637 INFO:teuthology.orchestra.run.smithi150.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2024-05-10T03:53:44.670 INFO:teuthology.orchestra.run.smithi150.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2024-05-10T03:53:44.696 INFO:teuthology.orchestra.run.smithi150.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 300 kB/s | 18 kB 00:00 2024-05-10T03:53:44.748 INFO:teuthology.orchestra.run.smithi161.stdout:(112/146): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2024-05-10T03:53:44.754 INFO:teuthology.orchestra.run.smithi150.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 838 kB/s | 49 kB 00:00 2024-05-10T03:53:44.782 INFO:teuthology.orchestra.run.smithi161.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 1.8 MB/s | 538 kB 00:00 2024-05-10T03:53:44.788 INFO:teuthology.orchestra.run.smithi150.stdout:(52/146): python3-kubernetes-11.0.0-6.el8.noarc 10 MB/s | 1.2 MB 00:00 2024-05-10T03:53:44.798 INFO:teuthology.orchestra.run.smithi181.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1330.g50174 6.6 MB/s | 3.5 MB 00:00 2024-05-10T03:53:44.813 INFO:teuthology.orchestra.run.smithi150.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2024-05-10T03:53:44.847 INFO:teuthology.orchestra.run.smithi150.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2024-05-10T03:53:44.865 INFO:teuthology.orchestra.run.smithi181.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1330.g50 3.6 MB/s | 246 kB 00:00 2024-05-10T03:53:44.874 INFO:teuthology.orchestra.run.smithi161.stdout:(114/146): python3-numpy-1.14.3-10.el8.x86_64.r 12 MB/s | 3.7 MB 00:00 2024-05-10T03:53:44.889 INFO:teuthology.orchestra.run.smithi150.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2024-05-10T03:53:44.899 INFO:teuthology.orchestra.run.smithi161.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.1 MB/s | 323 kB 00:00 2024-05-10T03:53:44.914 INFO:teuthology.orchestra.run.smithi150.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2024-05-10T03:53:44.915 INFO:teuthology.orchestra.run.smithi181.stdout:(32/146): ceph-mgr-rook-18.2.2-1330.g50174015.e 985 kB/s | 49 kB 00:00 2024-05-10T03:53:44.924 INFO:teuthology.orchestra.run.smithi161.stdout:(116/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.5 MB/s | 225 kB 00:00 2024-05-10T03:53:44.948 INFO:teuthology.orchestra.run.smithi150.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 576 kB/s | 34 kB 00:00 2024-05-10T03:53:45.000 INFO:teuthology.orchestra.run.smithi181.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1330.g5 192 kB/s | 16 kB 00:00 2024-05-10T03:53:45.013 INFO:teuthology.orchestra.run.smithi150.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2024-05-10T03:53:45.025 INFO:teuthology.orchestra.run.smithi161.stdout:(117/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.3 MB/s | 354 kB 00:00 2024-05-10T03:53:45.040 INFO:teuthology.orchestra.run.smithi150.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 700 kB/s | 64 kB 00:00 2024-05-10T03:53:45.050 INFO:teuthology.orchestra.run.smithi161.stdout:(118/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2024-05-10T03:53:45.073 INFO:teuthology.orchestra.run.smithi150.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 311 kB/s | 19 kB 00:00 2024-05-10T03:53:45.074 INFO:teuthology.orchestra.run.smithi181.stdout:(34/146): ceph-volume-18.2.2-1330.g50174015.el8 3.5 MB/s | 263 kB 00:00 2024-05-10T03:53:45.099 INFO:teuthology.orchestra.run.smithi150.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 408 kB/s | 24 kB 00:00 2024-05-10T03:53:45.124 INFO:teuthology.orchestra.run.smithi150.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 583 kB/s | 29 kB 00:00 2024-05-10T03:53:45.141 INFO:teuthology.orchestra.run.smithi181.stdout:(35/146): cephadm-18.2.2-1330.g50174015.el8.noa 2.5 MB/s | 173 kB 00:00 2024-05-10T03:53:45.149 INFO:teuthology.orchestra.run.smithi150.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 550 kB/s | 27 kB 00:00 2024-05-10T03:53:45.183 INFO:teuthology.orchestra.run.smithi181.stdout:(36/146): cephfs-top-18.2.2-1330.g50174015.el8. 434 kB/s | 18 kB 00:00 2024-05-10T03:53:45.200 INFO:teuthology.orchestra.run.smithi150.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2024-05-10T03:53:45.217 INFO:teuthology.orchestra.run.smithi161.stdout:(119/146): openblas-threads-0.3.15-6.el8.x86_64 16 MB/s | 5.2 MB 00:00 2024-05-10T03:53:45.225 INFO:teuthology.orchestra.run.smithi150.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2024-05-10T03:53:45.250 INFO:teuthology.orchestra.run.smithi150.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2024-05-10T03:53:45.250 INFO:teuthology.orchestra.run.smithi161.stdout:(120/146): automake-1.16.1-8.el8.noarch.rpm 3.5 MB/s | 714 kB 00:00 2024-05-10T03:53:45.284 INFO:teuthology.orchestra.run.smithi150.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2024-05-10T03:53:45.301 INFO:teuthology.orchestra.run.smithi150.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 456 kB/s | 23 kB 00:00 2024-05-10T03:53:45.308 INFO:teuthology.orchestra.run.smithi181.stdout:(37/146): ceph-mgr-diskprediction-local-18.2.2- 7.5 MB/s | 7.4 MB 00:00 2024-05-10T03:53:45.326 INFO:teuthology.orchestra.run.smithi150.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-05-10T03:53:45.326 INFO:teuthology.orchestra.run.smithi161.stdout:(121/146): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2024-05-10T03:53:45.360 INFO:teuthology.orchestra.run.smithi161.stdout:(122/146): libpq-13.11-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2024-05-10T03:53:45.385 INFO:teuthology.orchestra.run.smithi161.stdout:(123/146): jq-1.6-7.el8.x86_64.rpm 1.5 MB/s | 202 kB 00:00 2024-05-10T03:53:45.393 INFO:teuthology.orchestra.run.smithi150.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 16 MB/s | 1.7 MB 00:00 2024-05-10T03:53:45.460 INFO:teuthology.orchestra.run.smithi150.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 150 kB/s | 20 kB 00:00 2024-05-10T03:53:45.477 INFO:teuthology.orchestra.run.smithi161.stdout:(124/146): python3-pytz-2017.2-11.el8.noarch.rp 361 kB/s | 54 kB 00:00 2024-05-10T03:53:45.510 INFO:teuthology.orchestra.run.smithi161.stdout:(125/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-10T03:53:45.543 INFO:teuthology.orchestra.run.smithi150.stdout:(72/146): python3-prettytable-0.7.2-14.el8.noar 293 kB/s | 44 kB 00:00 2024-05-10T03:53:45.577 INFO:teuthology.orchestra.run.smithi150.stdout:(73/146): python3-pyasn1-modules-0.3.7-6.el8.no 942 kB/s | 110 kB 00:00 2024-05-10T03:53:45.627 INFO:teuthology.orchestra.run.smithi161.stdout:(126/146): postgresql-contrib-10.23-4.module+el 6.8 MB/s | 812 kB 00:00 2024-05-10T03:53:45.669 INFO:teuthology.orchestra.run.smithi161.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 7.8 MB/s | 1.5 MB 00:00 2024-05-10T03:53:45.670 INFO:teuthology.orchestra.run.smithi150.stdout:(74/146): python3-jsonpatch-1.21-2.el8.noarch.r 212 kB/s | 27 kB 00:00 2024-05-10T03:53:45.695 INFO:teuthology.orchestra.run.smithi150.stdout:(75/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2024-05-10T03:53:45.700 INFO:teuthology.orchestra.run.smithi181.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 150 kB/s | 59 kB 00:00 2024-05-10T03:53:45.820 INFO:teuthology.orchestra.run.smithi150.stdout:(76/146): bison-3.0.4-10.el8.x86_64.rpm 5.4 MB/s | 688 kB 00:00 2024-05-10T03:53:45.861 INFO:teuthology.orchestra.run.smithi161.stdout:(128/146): python3-scipy-1.0.0-21.module+el8.9. 27 MB/s | 14 MB 00:00 2024-05-10T03:53:45.895 INFO:teuthology.orchestra.run.smithi161.stdout:(129/146): python3-packaging-16.8-9.el8.noarch. 214 kB/s | 48 kB 00:00 2024-05-10T03:53:45.900 INFO:teuthology.orchestra.run.smithi181.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 375 kB/s | 75 kB 00:00 2024-05-10T03:53:45.929 INFO:teuthology.orchestra.run.smithi150.stdout:(77/146): python3-werkzeug-0.12.2-4.el8.noarch. 1.7 MB/s | 457 kB 00:00 2024-05-10T03:53:45.995 INFO:teuthology.orchestra.run.smithi150.stdout:(78/146): flex-2.6.1-9.el8.x86_64.rpm 1.8 MB/s | 320 kB 00:00 2024-05-10T03:53:46.078 INFO:teuthology.orchestra.run.smithi161.stdout:(130/146): postgresql-server-10.23-4.module+el8 11 MB/s | 5.1 MB 00:00 2024-05-10T03:53:46.104 INFO:teuthology.orchestra.run.smithi150.stdout:(79/146): libtool-2.4.6-25.el8.x86_64.rpm 4.0 MB/s | 709 kB 00:00 2024-05-10T03:53:46.129 INFO:teuthology.orchestra.run.smithi150.stdout:(80/146): python3-markupsafe-0.23-19.el8.x86_64 291 kB/s | 39 kB 00:00 2024-05-10T03:53:46.159 INFO:teuthology.orchestra.run.smithi181.stdout:(40/146): gperftools-libs-2.7-9.el8.x86_64.rpm 314 kB/s | 306 kB 00:00 2024-05-10T03:53:46.228 INFO:teuthology.orchestra.run.smithi161.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.5 MB/s | 232 kB 00:00 2024-05-10T03:53:46.230 INFO:teuthology.orchestra.run.smithi150.stdout:(81/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2024-05-10T03:53:46.276 INFO:teuthology.orchestra.run.smithi181.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 374 kB/s | 44 kB 00:00 2024-05-10T03:53:46.302 INFO:teuthology.orchestra.run.smithi150.stdout:(82/146): uuid-1.6.2-43.el8.x86_64.rpm 371 kB/s | 64 kB 00:00 2024-05-10T03:53:46.337 INFO:teuthology.orchestra.run.smithi161.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.7 MB/s | 303 kB 00:00 2024-05-10T03:53:46.372 INFO:teuthology.orchestra.run.smithi150.stdout:(83/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 730 kB/s | 103 kB 00:00 2024-05-10T03:53:46.437 INFO:teuthology.orchestra.run.smithi161.stdout:(133/146): librbd1-18.2.2-1330.g50174015.el8.x8 6.5 MB/s | 3.5 MB 00:00 2024-05-10T03:53:46.479 INFO:teuthology.orchestra.run.smithi161.stdout:(134/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.9 MB/s | 854 kB 00:00 2024-05-10T03:53:46.514 INFO:teuthology.orchestra.run.smithi150.stdout:(84/146): python3-jinja2-2.10.1-3.el8.noarch.rp 3.7 MB/s | 538 kB 00:00 2024-05-10T03:53:46.554 INFO:teuthology.orchestra.run.smithi161.stdout:(135/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 5.9 MB/s | 711 kB 00:00 2024-05-10T03:53:46.589 INFO:teuthology.orchestra.run.smithi150.stdout:(85/146): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2024-05-10T03:53:46.647 INFO:teuthology.orchestra.run.smithi161.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.8 MB/s | 1.5 MB 00:00 2024-05-10T03:53:46.668 INFO:teuthology.orchestra.run.smithi181.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 473 kB/s | 185 kB 00:00 2024-05-10T03:53:46.672 INFO:teuthology.orchestra.run.smithi161.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.8 MB/s | 333 kB 00:00 2024-05-10T03:53:46.689 INFO:teuthology.orchestra.run.smithi150.stdout:(86/146): socat-1.7.4.1-1.el8.x86_64.rpm 3.2 MB/s | 323 kB 00:00 2024-05-10T03:53:46.743 INFO:teuthology.orchestra.run.smithi181.stdout:(43/146): python3-asyncssh-2.7.0-2.el8.noarch.r 565 kB/s | 476 kB 00:00 2024-05-10T03:53:46.772 INFO:teuthology.orchestra.run.smithi161.stdout:(138/146): librados2-18.2.2-1330.g50174015.el8. 3.9 MB/s | 3.5 MB 00:00 2024-05-10T03:53:46.777 INFO:teuthology.orchestra.run.smithi181.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 304 kB/s | 33 kB 00:00 2024-05-10T03:53:46.790 INFO:teuthology.orchestra.run.smithi150.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2024-05-10T03:53:46.797 INFO:teuthology.orchestra.run.smithi161.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 221 kB 00:00 2024-05-10T03:53:46.815 INFO:teuthology.orchestra.run.smithi150.stdout:(88/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.8 MB/s | 225 kB 00:00 2024-05-10T03:53:46.852 INFO:teuthology.orchestra.run.smithi181.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2024-05-10T03:53:46.862 INFO:teuthology.orchestra.run.smithi161.stdout:(140/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.0 MB/s | 1.1 MB 00:00 2024-05-10T03:53:46.898 INFO:teuthology.orchestra.run.smithi161.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 2.0 MB/s | 254 kB 00:00 2024-05-10T03:53:46.930 INFO:teuthology.orchestra.run.smithi150.stdout:(89/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.5 MB/s | 354 kB 00:00 2024-05-10T03:53:46.974 INFO:teuthology.orchestra.run.smithi161.stdout:(142/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 1.3 MB/s | 99 kB 00:00 2024-05-10T03:53:46.998 INFO:teuthology.orchestra.run.smithi161.stdout:(143/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.2 MB/s | 237 kB 00:00 2024-05-10T03:53:47.024 INFO:teuthology.orchestra.run.smithi161.stdout:(144/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.1 MB/s | 180 kB 00:00 2024-05-10T03:53:47.049 INFO:teuthology.orchestra.run.smithi150.stdout:(90/146): openblas-threads-0.3.15-6.el8.x86_64. 22 MB/s | 5.2 MB 00:00 2024-05-10T03:53:47.078 INFO:teuthology.orchestra.run.smithi181.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 575 kB/s | 173 kB 00:00 2024-05-10T03:53:47.082 INFO:teuthology.orchestra.run.smithi150.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2024-05-10T03:53:47.132 INFO:teuthology.orchestra.run.smithi161.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 16 MB/s | 2.5 MB 00:00 2024-05-10T03:53:47.157 INFO:teuthology.orchestra.run.smithi161.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 648 kB/s | 103 kB 00:00 2024-05-10T03:53:47.160 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-05-10T03:53:47.160 INFO:teuthology.orchestra.run.smithi161.stdout:Total 14 MB/s | 221 MB 00:16 2024-05-10T03:53:47.199 INFO:teuthology.orchestra.run.smithi150.stdout:(92/146): openblas-0.3.15-6.el8.x86_64.rpm 34 MB/s | 5.0 MB 00:00 2024-05-10T03:53:47.249 INFO:teuthology.orchestra.run.smithi150.stdout:(93/146): automake-1.16.1-8.el8.noarch.rpm 4.2 MB/s | 714 kB 00:00 2024-05-10T03:53:47.286 INFO:teuthology.orchestra.run.smithi181.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 390 kB/s | 81 kB 00:00 2024-05-10T03:53:47.403 INFO:teuthology.orchestra.run.smithi181.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2024-05-10T03:53:47.512 INFO:teuthology.orchestra.run.smithi181.stdout:(49/146): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2024-05-10T03:53:47.537 INFO:teuthology.orchestra.run.smithi181.stdout:(50/146): python3-cherrypy-18.4.0-1.el8.noarch. 562 kB/s | 384 kB 00:00 2024-05-10T03:53:47.670 INFO:teuthology.orchestra.run.smithi181.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 365 kB/s | 49 kB 00:00 2024-05-10T03:53:47.717 INFO:teuthology.orchestra.run.smithi150.stdout:(94/146): ceph-test-18.2.2-1330.g50174015.el8.x 5.1 MB/s | 49 MB 00:09 2024-05-10T03:53:47.742 INFO:teuthology.orchestra.run.smithi150.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 365 kB/s | 198 kB 00:00 2024-05-10T03:53:47.775 INFO:teuthology.orchestra.run.smithi150.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 385 kB/s | 202 kB 00:00 2024-05-10T03:53:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 503 kB/s | 59 kB 00:00 2024-05-10T03:53:47.904 INFO:teuthology.orchestra.run.smithi181.stdout:(53/146): python3-natsort-7.1.1-2.el8.noarch.rp 517 kB/s | 60 kB 00:00 2024-05-10T03:53:47.934 INFO:teuthology.orchestra.run.smithi150.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 250 kB/s | 54 kB 00:00 2024-05-10T03:53:48.002 INFO:teuthology.orchestra.run.smithi150.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 836 kB/s | 188 kB 00:00 2024-05-10T03:53:48.168 INFO:teuthology.orchestra.run.smithi150.stdout:(99/146): python3-scipy-1.0.0-21.module+el8.9.0 33 MB/s | 14 MB 00:00 2024-05-10T03:53:48.174 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-05-10T03:53:48.218 INFO:teuthology.orchestra.run.smithi150.stdout:(100/146): postgresql-contrib-10.23-4.module+el 3.7 MB/s | 812 kB 00:00 2024-05-10T03:53:48.260 INFO:teuthology.orchestra.run.smithi150.stdout:(101/146): postgresql-10.23-4.module+el8.9.0+21 4.6 MB/s | 1.5 MB 00:00 2024-05-10T03:53:48.394 INFO:teuthology.orchestra.run.smithi150.stdout:(102/146): postgresql-server-10.23-4.module+el8 22 MB/s | 5.1 MB 00:00 2024-05-10T03:53:48.415 INFO:teuthology.orchestra.run.smithi181.stdout:(54/146): python3-pecan-1.3.2-9.el8.noarch.rpm 554 kB/s | 283 kB 00:00 2024-05-10T03:53:48.419 INFO:teuthology.orchestra.run.smithi150.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 113 kB/s | 18 kB 00:00 2024-05-10T03:53:48.444 INFO:teuthology.orchestra.run.smithi150.stdout:(104/146): python3-packaging-16.8-9.el8.noarch. 213 kB/s | 48 kB 00:00 2024-05-10T03:53:48.495 INFO:teuthology.orchestra.run.smithi150.stdout:(105/146): python3-oauthlib-2.1.0-1.el8.noarch. 2.0 MB/s | 155 kB 00:00 2024-05-10T03:53:48.530 INFO:teuthology.orchestra.run.smithi181.stdout:(55/146): python3-portend-2.6-1.el8.noarch.rpm 135 kB/s | 16 kB 00:00 2024-05-10T03:53:48.537 INFO:teuthology.orchestra.run.smithi150.stdout:(106/146): python3-jwt-1.6.1-2.el8.noarch.rpm 299 kB/s | 43 kB 00:00 2024-05-10T03:53:48.570 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-05-10T03:53:48.570 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-05-10T03:53:48.603 INFO:teuthology.orchestra.run.smithi150.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 687 kB/s | 109 kB 00:00 2024-05-10T03:53:48.637 INFO:teuthology.orchestra.run.smithi150.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 300 kB/s | 43 kB 00:00 2024-05-10T03:53:48.648 INFO:teuthology.orchestra.run.smithi181.stdout:(56/146): python3-repoze-lru-0.7-6.el8.noarch.r 288 kB/s | 34 kB 00:00 2024-05-10T03:53:48.721 INFO:teuthology.orchestra.run.smithi150.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 101 kB/s | 19 kB 00:00 2024-05-10T03:53:48.779 INFO:teuthology.orchestra.run.smithi150.stdout:(110/146): pcre2-devel-10.32-2.el8.x86_64.rpm 4.2 MB/s | 605 kB 00:00 2024-05-10T03:53:48.804 INFO:teuthology.orchestra.run.smithi150.stdout:(111/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 218 kB/s | 44 kB 00:00 2024-05-10T03:53:48.830 INFO:teuthology.orchestra.run.smithi150.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2024-05-10T03:53:48.897 INFO:teuthology.orchestra.run.smithi150.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2024-05-10T03:53:48.922 INFO:teuthology.orchestra.run.smithi150.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.7 MB/s | 200 kB 00:00 2024-05-10T03:53:48.988 INFO:teuthology.orchestra.run.smithi150.stdout:(115/146): keyutils-libs-devel-1.5.10-9.el8.x86 305 kB/s | 48 kB 00:00 2024-05-10T03:53:49.015 INFO:teuthology.orchestra.run.smithi150.stdout:(116/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 627 kB/s | 58 kB 00:00 2024-05-10T03:53:49.039 INFO:teuthology.orchestra.run.smithi181.stdout:(57/146): python3-routes-2.4.1-12.el8.noarch.rp 502 kB/s | 196 kB 00:00 2024-05-10T03:53:49.040 INFO:teuthology.orchestra.run.smithi150.stdout:(117/146): libsepol-devel-2.9-3.el8.x86_64.rpm 610 kB/s | 87 kB 00:00 2024-05-10T03:53:49.073 INFO:teuthology.orchestra.run.smithi150.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 453 kB/s | 39 kB 00:00 2024-05-10T03:53:49.149 INFO:teuthology.orchestra.run.smithi150.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 621 kB/s | 83 kB 00:00 2024-05-10T03:53:49.180 INFO:teuthology.orchestra.run.smithi181.stdout:(58/146): python3-rsa-4.9-2.el8.noarch.rpm 459 kB/s | 64 kB 00:00 2024-05-10T03:53:49.190 INFO:teuthology.orchestra.run.smithi150.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.2 MB/s | 645 kB 00:00 2024-05-10T03:53:49.215 INFO:teuthology.orchestra.run.smithi150.stdout:(121/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.2 MB/s | 172 kB 00:00 2024-05-10T03:53:49.275 INFO:teuthology.orchestra.run.smithi150.stdout:(122/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.9 MB/s | 238 kB 00:00 2024-05-10T03:53:49.308 INFO:teuthology.orchestra.run.smithi150.stdout:(123/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 620 kB/s | 73 kB 00:00 2024-05-10T03:53:49.350 INFO:teuthology.orchestra.run.smithi150.stdout:(124/146): elfutils-libelf-devel-0.189-3.el8.x8 461 kB/s | 62 kB 00:00 2024-05-10T03:53:49.383 INFO:teuthology.orchestra.run.smithi150.stdout:(125/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 5.1 MB/s | 562 kB 00:00 2024-05-10T03:53:49.459 INFO:teuthology.orchestra.run.smithi150.stdout:(126/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 434 kB/s | 47 kB 00:00 2024-05-10T03:53:49.542 INFO:teuthology.orchestra.run.smithi150.stdout:(127/146): openssl-devel-1.1.1k-12.el8_9.x86_64 9.9 MB/s | 2.3 MB 00:00 2024-05-10T03:53:49.584 INFO:teuthology.orchestra.run.smithi150.stdout:(128/146): python3-cryptography-3.2.1-7.el8_9.x 2.7 MB/s | 559 kB 00:00 2024-05-10T03:53:49.634 INFO:teuthology.orchestra.run.smithi150.stdout:(129/146): NetworkManager-initscripts-updown-1. 822 kB/s | 144 kB 00:00 2024-05-10T03:53:49.665 INFO:teuthology.orchestra.run.smithi181.stdout:(59/146): ceph-test-18.2.2-1330.g50174015.el8.x 5.6 MB/s | 49 MB 00:08 2024-05-10T03:53:49.676 INFO:teuthology.orchestra.run.smithi150.stdout:(130/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 746 kB/s | 100 kB 00:00 2024-05-10T03:53:49.698 INFO:teuthology.orchestra.run.smithi181.stdout:(60/146): python3-kubernetes-11.0.0-6.el8.noarc 546 kB/s | 1.2 MB 00:02 2024-05-10T03:53:49.724 INFO:teuthology.orchestra.run.smithi181.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 34 kB/s | 19 kB 00:00 2024-05-10T03:53:49.800 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-05-10T03:53:49.810 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-05-10T03:53:49.827 INFO:teuthology.orchestra.run.smithi150.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.5 MB/s | 232 kB 00:00 2024-05-10T03:53:49.965 INFO:teuthology.orchestra.run.smithi181.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 80 kB/s | 24 kB 00:00 2024-05-10T03:53:49.985 INFO:teuthology.orchestra.run.smithi150.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.9 MB/s | 303 kB 00:00 2024-05-10T03:53:50.077 INFO:teuthology.orchestra.run.smithi150.stdout:(133/146): librbd1-18.2.2-1330.g50174015.el8.x8 8.0 MB/s | 3.5 MB 00:00 2024-05-10T03:53:50.116 INFO:teuthology.orchestra.run.smithi181.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 75 kB/s | 29 kB 00:00 2024-05-10T03:53:50.136 INFO:teuthology.orchestra.run.smithi150.stdout:(134/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.6 MB/s | 854 kB 00:00 2024-05-10T03:53:50.141 INFO:teuthology.orchestra.run.smithi181.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 66 kB/s | 27 kB 00:00 2024-05-10T03:53:50.269 INFO:teuthology.orchestra.run.smithi150.stdout:(135/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.6 MB/s | 711 kB 00:00 2024-05-10T03:53:50.328 INFO:teuthology.orchestra.run.smithi150.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.7 MB/s | 1.5 MB 00:00 2024-05-10T03:53:50.387 INFO:teuthology.orchestra.run.smithi150.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.8 MB/s | 333 kB 00:00 2024-05-10T03:53:50.441 INFO:teuthology.orchestra.run.smithi181.stdout:(65/146): python3-websocket-client-0.56.0-5.el8 204 kB/s | 61 kB 00:00 2024-05-10T03:53:50.446 INFO:teuthology.orchestra.run.smithi150.stdout:(138/146): librados2-18.2.2-1330.g50174015.el8. 4.1 MB/s | 3.5 MB 00:00 2024-05-10T03:53:50.471 INFO:teuthology.orchestra.run.smithi150.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 2.6 MB/s | 221 kB 00:00 2024-05-10T03:53:50.513 INFO:teuthology.orchestra.run.smithi150.stdout:(140/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.8 MB/s | 1.1 MB 00:00 2024-05-10T03:53:50.533 INFO:teuthology.orchestra.run.smithi181.stdout:(66/146): python3-waitress-1.4.3-1.el8.noarch.r 450 kB/s | 255 kB 00:00 2024-05-10T03:53:50.580 INFO:teuthology.orchestra.run.smithi150.stdout:(141/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 2.1 MB/s | 237 kB 00:00 2024-05-10T03:53:50.613 INFO:teuthology.orchestra.run.smithi150.stdout:(142/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.5 MB/s | 254 kB 00:00 2024-05-10T03:53:50.650 INFO:teuthology.orchestra.run.smithi181.stdout:(67/146): python3-zc-lockfile-2.0-2.el8.noarch. 197 kB/s | 23 kB 00:00 2024-05-10T03:53:50.667 INFO:teuthology.orchestra.run.smithi181.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 379 kB/s | 85 kB 00:00 2024-05-10T03:53:50.697 INFO:teuthology.orchestra.run.smithi150.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 978 kB/s | 180 kB 00:00 2024-05-10T03:53:50.701 INFO:teuthology.orchestra.run.smithi181.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.1 MB/s | 70 kB 00:00 2024-05-10T03:53:50.714 INFO:teuthology.orchestra.run.smithi150.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 740 kB/s | 99 kB 00:00 2024-05-10T03:53:50.822 INFO:teuthology.orchestra.run.smithi150.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 822 kB/s | 103 kB 00:00 2024-05-10T03:53:50.826 INFO:teuthology.orchestra.run.smithi181.stdout:(70/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 143 kB/s | 18 kB 00:00 2024-05-10T03:53:50.896 INFO:teuthology.orchestra.run.smithi150.stdout:(146/146): util-linux-2.32.1-44.el8_9.1.x86_64. 8.8 MB/s | 2.5 MB 00:00 2024-05-10T03:53:50.900 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-05-10T03:53:50.900 INFO:teuthology.orchestra.run.smithi150.stdout:Total 11 MB/s | 221 MB 00:19 2024-05-10T03:53:50.951 INFO:teuthology.orchestra.run.smithi181.stdout:(71/146): python3-jwt-1.6.1-2.el8.noarch.rpm 341 kB/s | 43 kB 00:00 2024-05-10T03:53:50.976 INFO:teuthology.orchestra.run.smithi181.stdout:(72/146): python3-webob-1.8.5-1.el8.1.noarch.rp 292 kB/s | 251 kB 00:00 2024-05-10T03:53:51.102 INFO:teuthology.orchestra.run.smithi181.stdout:(73/146): python3-oauthlib-2.1.0-1.el8.noarch.r 1.0 MB/s | 155 kB 00:00 2024-05-10T03:53:51.235 INFO:teuthology.orchestra.run.smithi181.stdout:(74/146): python3-pycparser-2.14-14.el8.noarch. 422 kB/s | 109 kB 00:00 2024-05-10T03:53:51.261 INFO:teuthology.orchestra.run.smithi181.stdout:(75/146): python3-requests-oauthlib-1.0.0-1.el8 269 kB/s | 43 kB 00:00 2024-05-10T03:53:51.369 INFO:teuthology.orchestra.run.smithi181.stdout:(76/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 405 kB/s | 44 kB 00:00 2024-05-10T03:53:51.395 INFO:teuthology.orchestra.run.smithi181.stdout:(77/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 118 kB/s | 19 kB 00:00 2024-05-10T03:53:51.516 INFO:teuthology.orchestra.run.smithi181.stdout:(78/146): pcre2-devel-10.32-2.el8.x86_64.rpm 4.0 MB/s | 605 kB 00:00 2024-05-10T03:53:51.554 INFO:teuthology.orchestra.run.smithi181.stdout:(79/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.4 MB/s | 229 kB 00:00 2024-05-10T03:53:51.612 INFO:teuthology.orchestra.run.smithi181.stdout:(80/146): libselinux-devel-2.9-5.el8.x86_64.rpm 3.4 MB/s | 200 kB 00:00 2024-05-10T03:53:51.629 INFO:teuthology.orchestra.run.smithi181.stdout:(81/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.9 MB/s | 220 kB 00:00 2024-05-10T03:53:51.738 INFO:teuthology.orchestra.run.smithi181.stdout:(82/146): libsepol-devel-2.9-3.el8.x86_64.rpm 802 kB/s | 87 kB 00:00 2024-05-10T03:53:51.863 INFO:teuthology.orchestra.run.smithi181.stdout:(83/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 461 kB/s | 58 kB 00:00 2024-05-10T03:53:51.923 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-05-10T03:53:51.980 INFO:teuthology.orchestra.run.smithi181.stdout:(84/146): libcom_err-devel-1.45.6-4.el8.x86_64. 331 kB/s | 39 kB 00:00 2024-05-10T03:53:52.072 INFO:teuthology.orchestra.run.smithi181.stdout:(85/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 105 kB/s | 48 kB 00:00 2024-05-10T03:53:52.089 INFO:teuthology.orchestra.run.smithi181.stdout:(86/146): fuse-2.9.7-17.el8.x86_64.rpm 764 kB/s | 83 kB 00:00 2024-05-10T03:53:52.173 INFO:teuthology.orchestra.run.smithi181.stdout:(87/146): libgfortran-8.5.0-20.el8.x86_64.rpm 6.3 MB/s | 645 kB 00:00 2024-05-10T03:53:52.198 INFO:teuthology.orchestra.run.smithi181.stdout:(88/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.6 MB/s | 172 kB 00:00 2024-05-10T03:53:52.299 INFO:teuthology.orchestra.run.smithi181.stdout:(89/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.9 MB/s | 238 kB 00:00 2024-05-10T03:53:52.324 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-05-10T03:53:52.324 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-05-10T03:53:52.348 INFO:teuthology.orchestra.run.smithi181.stdout:(90/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 483 kB/s | 73 kB 00:00 2024-05-10T03:53:52.390 INFO:teuthology.orchestra.run.smithi181.stdout:(91/146): elfutils-libelf-devel-0.189-3.el8.x86 671 kB/s | 62 kB 00:00 2024-05-10T03:53:52.474 INFO:teuthology.orchestra.run.smithi181.stdout:(92/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 4.4 MB/s | 562 kB 00:00 2024-05-10T03:53:52.541 INFO:teuthology.orchestra.run.smithi181.stdout:(93/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 706 kB/s | 47 kB 00:00 2024-05-10T03:53:52.600 INFO:teuthology.orchestra.run.smithi181.stdout:(94/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-05-10T03:53:52.675 INFO:teuthology.orchestra.run.smithi181.stdout:(95/146): NetworkManager-initscripts-updown-1.4 1.9 MB/s | 144 kB 00:00 2024-05-10T03:53:52.709 INFO:teuthology.orchestra.run.smithi181.stdout:(96/146): python3-cryptography-3.2.1-7.el8_9.x8 3.3 MB/s | 559 kB 00:00 2024-05-10T03:53:52.784 INFO:teuthology.orchestra.run.smithi181.stdout:(97/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 919 kB/s | 100 kB 00:00 2024-05-10T03:53:52.859 INFO:teuthology.orchestra.run.smithi181.stdout:(98/146): python3-packaging-16.8-9.el8.noarch.r 321 kB/s | 48 kB 00:00 2024-05-10T03:53:52.926 INFO:teuthology.orchestra.run.smithi181.stdout:(99/146): python3-jsonpointer-1.10-11.el8.noarc 141 kB/s | 20 kB 00:00 2024-05-10T03:53:53.051 INFO:teuthology.orchestra.run.smithi181.stdout:(100/146): python3-pyasn1-modules-0.3.7-6.el8.n 882 kB/s | 110 kB 00:00 2024-05-10T03:53:53.076 INFO:teuthology.orchestra.run.smithi181.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 202 kB/s | 44 kB 00:00 2024-05-10T03:53:53.202 INFO:teuthology.orchestra.run.smithi181.stdout:(102/146): python3-jsonpatch-1.21-2.el8.noarch. 176 kB/s | 27 kB 00:00 2024-05-10T03:53:53.227 INFO:teuthology.orchestra.run.smithi181.stdout:(103/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 841 kB/s | 126 kB 00:00 2024-05-10T03:53:53.328 INFO:teuthology.orchestra.run.smithi181.stdout:(104/146): bison-3.0.4-10.el8.x86_64.rpm 6.7 MB/s | 688 kB 00:00 2024-05-10T03:53:53.361 INFO:teuthology.orchestra.run.smithi181.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 2.8 MB/s | 457 kB 00:00 2024-05-10T03:53:53.478 INFO:teuthology.orchestra.run.smithi181.stdout:(106/146): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2024-05-10T03:53:53.520 INFO:teuthology.orchestra.run.smithi181.stdout:(107/146): libtool-2.4.6-25.el8.x86_64.rpm 4.4 MB/s | 709 kB 00:00 2024-05-10T03:53:53.545 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-05-10T03:53:53.554 INFO:teuthology.orchestra.run.smithi181.stdout:(108/146): thrift-0.13.0-2.el8.x86_64.rpm 612 kB/s | 1.7 MB 00:02 2024-05-10T03:53:53.556 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-05-10T03:53:53.587 INFO:teuthology.orchestra.run.smithi181.stdout:(109/146): python3-markupsafe-0.23-19.el8.x86_6 357 kB/s | 39 kB 00:00 2024-05-10T03:53:53.621 INFO:teuthology.orchestra.run.smithi181.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.5 MB/s | 259 kB 00:00 2024-05-10T03:53:53.721 INFO:teuthology.orchestra.run.smithi181.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 382 kB/s | 64 kB 00:00 2024-05-10T03:53:53.747 INFO:teuthology.orchestra.run.smithi181.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 673 kB/s | 103 kB 00:00 2024-05-10T03:53:53.855 INFO:teuthology.orchestra.run.smithi181.stdout:(113/146): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2024-05-10T03:53:53.889 INFO:teuthology.orchestra.run.smithi181.stdout:(114/146): python3-jinja2-2.10.1-3.el8.noarch.r 3.1 MB/s | 538 kB 00:00 2024-05-10T03:53:53.973 INFO:teuthology.orchestra.run.smithi181.stdout:(115/146): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2024-05-10T03:53:53.998 INFO:teuthology.orchestra.run.smithi181.stdout:(116/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.2 MB/s | 323 kB 00:00 2024-05-10T03:53:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.6 MB/s | 225 kB 00:00 2024-05-10T03:53:54.115 INFO:teuthology.orchestra.run.smithi181.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.4 MB/s | 354 kB 00:00 2024-05-10T03:53:54.174 INFO:teuthology.orchestra.run.smithi181.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2024-05-10T03:53:54.299 INFO:teuthology.orchestra.run.smithi181.stdout:(120/146): openblas-threads-0.3.15-6.el8.x86_64 17 MB/s | 5.2 MB 00:00 2024-05-10T03:53:54.341 INFO:teuthology.orchestra.run.smithi181.stdout:(121/146): automake-1.16.1-8.el8.noarch.rpm 4.2 MB/s | 714 kB 00:00 2024-05-10T03:53:54.441 INFO:teuthology.orchestra.run.smithi181.stdout:(122/146): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2024-05-10T03:53:54.491 INFO:teuthology.orchestra.run.smithi181.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 1.0 MB/s | 198 kB 00:00 2024-05-10T03:53:54.558 INFO:teuthology.orchestra.run.smithi181.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 933 kB/s | 202 kB 00:00 2024-05-10T03:53:54.584 INFO:teuthology.orchestra.run.smithi181.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 381 kB/s | 54 kB 00:00 2024-05-10T03:53:54.717 INFO:teuthology.orchestra.run.smithi181.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-05-10T03:53:54.776 INFO:teuthology.orchestra.run.smithi181.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 7.8 MB/s | 1.5 MB 00:00 2024-05-10T03:53:54.959 INFO:teuthology.orchestra.run.smithi181.stdout:(128/146): python3-scipy-1.0.0-21.module+el8.9. 29 MB/s | 14 MB 00:00 2024-05-10T03:53:55.001 INFO:teuthology.orchestra.run.smithi181.stdout:(129/146): postgresql-contrib-10.23-4.module+el 2.8 MB/s | 812 kB 00:00 2024-05-10T03:53:55.127 INFO:teuthology.orchestra.run.smithi181.stdout:(130/146): postgresql-server-10.23-4.module+el8 14 MB/s | 5.1 MB 00:00 2024-05-10T03:53:55.302 INFO:teuthology.orchestra.run.smithi181.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.3 MB/s | 232 kB 00:00 2024-05-10T03:53:55.419 INFO:teuthology.orchestra.run.smithi181.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.5 MB/s | 303 kB 00:00 2024-05-10T03:53:55.477 INFO:teuthology.orchestra.run.smithi181.stdout:(133/146): librbd1-18.2.2-1330.g50174015.el8.x8 7.5 MB/s | 3.5 MB 00:00 2024-05-10T03:53:55.569 INFO:teuthology.orchestra.run.smithi181.stdout:(134/146): librados2-18.2.2-1330.g50174015.el8. 5.8 MB/s | 3.5 MB 00:00 2024-05-10T03:53:55.603 INFO:teuthology.orchestra.run.smithi181.stdout:(135/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 5.5 MB/s | 711 kB 00:00 2024-05-10T03:53:55.663 INFO:teuthology.orchestra.run.smithi181.stdout:(136/146): iproute-6.2.0-5.el8_9.x86_64.rpm 3.4 MB/s | 854 kB 00:00 2024-05-10T03:53:55.745 INFO:teuthology.orchestra.run.smithi181.stdout:(137/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.4 MB/s | 1.5 MB 00:00 2024-05-10T03:53:55.771 INFO:teuthology.orchestra.run.smithi181.stdout:(138/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.9 MB/s | 333 kB 00:00 2024-05-10T03:53:55.812 INFO:teuthology.orchestra.run.smithi181.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.2 MB/s | 1.1 MB 00:00 2024-05-10T03:53:55.838 INFO:teuthology.orchestra.run.smithi181.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 2.3 MB/s | 221 kB 00:00 2024-05-10T03:53:55.846 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-05-10T03:53:55.888 INFO:teuthology.orchestra.run.smithi181.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 2.1 MB/s | 254 kB 00:00 2024-05-10T03:53:55.938 INFO:teuthology.orchestra.run.smithi181.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.8 MB/s | 237 kB 00:00 2024-05-10T03:53:55.980 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-10T03:53:55.988 INFO:teuthology.orchestra.run.smithi181.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.2 MB/s | 180 kB 00:00 2024-05-10T03:53:56.056 INFO:teuthology.orchestra.run.smithi181.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 593 kB/s | 99 kB 00:00 2024-05-10T03:53:56.114 INFO:teuthology.orchestra.run.smithi181.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 14 MB/s | 2.5 MB 00:00 2024-05-10T03:53:56.139 INFO:teuthology.orchestra.run.smithi181.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 683 kB/s | 103 kB 00:00 2024-05-10T03:53:56.142 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-05-10T03:53:56.142 INFO:teuthology.orchestra.run.smithi181.stdout:Total 8.9 MB/s | 221 MB 00:24 2024-05-10T03:53:56.256 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T03:53:56.472 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T03:53:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-10T03:53:57.060 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T03:53:57.134 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T03:53:57.208 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-05-10T03:53:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T03:53:57.413 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T03:53:57.602 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-05-10T03:53:57.602 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-05-10T03:53:57.606 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-10T03:53:57.859 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-10T03:53:58.085 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-10T03:53:58.171 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-10T03:53:58.351 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T03:53:58.787 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T03:53:58.832 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-05-10T03:53:58.842 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-05-10T03:53:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-05-10T03:53:58.993 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T03:53:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-10T03:53:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T03:53:59.404 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T03:53:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T03:53:59.610 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T03:53:59.712 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-10T03:53:59.737 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T03:53:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T03:53:59.959 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T03:53:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T03:54:00.041 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T03:54:00.175 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T03:54:00.235 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T03:54:00.387 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T03:54:00.448 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-10T03:54:00.650 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-10T03:54:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T03:54:00.719 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-10T03:54:00.747 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T03:54:00.885 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-10T03:54:00.999 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-10T03:54:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T03:54:01.262 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-10T03:54:01.420 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-10T03:54:01.467 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T03:54:01.667 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T03:54:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-10T03:54:02.077 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T03:54:02.080 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-10T03:54:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T03:54:02.255 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T03:54:02.360 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T03:54:02.383 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T03:54:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T03:54:02.558 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T03:54:02.622 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T03:54:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T03:54:02.829 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-10T03:54:02.943 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T03:54:03.160 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-10T03:54:03.200 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T03:54:03.288 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T03:54:03.372 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-10T03:54:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-10T03:54:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-10T03:54:03.705 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-10T03:54:03.820 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-10T03:54:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-10T03:54:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-10T03:54:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-10T03:54:04.476 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-05-10T03:54:04.534 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-10T03:54:04.571 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-10T03:54:04.611 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-10T03:54:04.720 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-10T03:54:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T03:54:04.850 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T03:54:04.891 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T03:54:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T03:54:05.079 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T03:54:05.104 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T03:54:05.249 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-10T03:54:05.302 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-10T03:54:05.324 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-10T03:54:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-10T03:54:05.481 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T03:54:05.500 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-10T03:54:05.558 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T03:54:05.574 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-10T03:54:05.724 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T03:54:05.794 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T03:54:06.036 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-10T03:54:06.180 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-10T03:54:06.214 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-10T03:54:06.214 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-10T03:54:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-10T03:54:06.440 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-10T03:54:06.489 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-10T03:54:06.527 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-10T03:54:06.723 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-10T03:54:06.748 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T03:54:06.961 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-10T03:54:06.968 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-10T03:54:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-10T03:54:07.123 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 31/162 2024-05-10T03:54:07.150 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T03:54:07.261 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 32/162 2024-05-10T03:54:07.356 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T03:54:07.571 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 33/162 2024-05-10T03:54:07.767 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T03:54:07.794 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-10T03:54:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-10T03:54:07.965 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T03:54:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-10T03:54:08.093 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T03:54:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-10T03:54:08.298 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T03:54:08.382 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T03:54:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-10T03:54:08.675 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-10T03:54:08.683 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T03:54:08.940 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T03:54:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-10T03:54:09.055 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T03:54:09.184 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-10T03:54:09.254 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-10T03:54:09.331 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 31/162 2024-05-10T03:54:09.468 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 32/162 2024-05-10T03:54:09.545 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-10T03:54:09.552 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-10T03:54:09.709 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-10T03:54:09.768 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 33/162 2024-05-10T03:54:09.902 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-10T03:54:10.094 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-10T03:54:10.335 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-10T03:54:10.496 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T03:54:10.626 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T03:54:10.680 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-10T03:54:10.764 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T03:54:10.834 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-10T03:54:10.838 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T03:54:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-10T03:54:11.042 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-10T03:54:11.180 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-10T03:54:11.278 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-10T03:54:11.332 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-10T03:54:11.441 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-10T03:54:11.531 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-10T03:54:11.569 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-10T03:54:11.693 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-10T03:54:11.739 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-10T03:54:11.877 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-10T03:54:11.937 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-10T03:54:12.006 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-10T03:54:12.163 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 41/162 2024-05-10T03:54:12.350 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 42/162 2024-05-10T03:54:12.566 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T03:54:12.630 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T03:54:12.677 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-10T03:54:12.857 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T03:54:12.869 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-10T03:54:12.923 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T03:54:13.179 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-10T03:54:13.277 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-10T03:54:13.377 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-10T03:54:13.481 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-10T03:54:13.574 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T03:54:13.591 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-10T03:54:13.672 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-10T03:54:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-10T03:54:14.014 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-10T03:54:14.035 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T03:54:14.220 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 41/162 2024-05-10T03:54:14.333 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T03:54:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T03:54:14.367 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-10T03:54:14.389 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 42/162 2024-05-10T03:54:14.557 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T03:54:14.569 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-16.8-9.el8.noarch 49/162 2024-05-10T03:54:14.614 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T03:54:14.620 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-10T03:54:14.741 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 50/162 2024-05-10T03:54:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 50/162 2024-05-10T03:54:14.840 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T03:54:14.845 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-10T03:54:14.908 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T03:54:15.082 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-10T03:54:15.110 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-10T03:54:15.122 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : jq-1.6-7.el8.x86_64 51/162 2024-05-10T03:54:15.270 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 31/162 2024-05-10T03:54:15.308 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-10T03:54:15.374 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.1-8.el8.noarch 52/162 2024-05-10T03:54:15.399 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 32/162 2024-05-10T03:54:15.515 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T03:54:15.549 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-43.el8.x86_64 53/162 2024-05-10T03:54:15.631 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 53/162 2024-05-10T03:54:15.651 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 33/162 2024-05-10T03:54:15.820 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 54/162 2024-05-10T03:54:15.895 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-10T03:54:15.962 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T03:54:16.040 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 55/162 2024-05-10T03:54:16.209 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 56/162 2024-05-10T03:54:16.274 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T03:54:16.291 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T03:54:16.411 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-05-10T03:54:16.469 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-10T03:54:16.606 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-05-10T03:54:16.662 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-10T03:54:16.821 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-05-10T03:54:16.877 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-10T03:54:16.994 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-05-10T03:54:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-10T03:54:17.228 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-05-10T03:54:17.260 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-10T03:54:17.562 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-05-10T03:54:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-10T03:54:17.638 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-10T03:54:17.811 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-10T03:54:17.865 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-05-10T03:54:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-10T03:54:18.078 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-10T03:54:18.087 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-05-10T03:54:18.244 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-10T03:54:18.278 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-05-10T03:54:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-10T03:54:18.485 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-05-10T03:54:18.831 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-10T03:54:18.848 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-05-10T03:54:19.018 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-10T03:54:19.127 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-05-10T03:54:19.134 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-10T03:54:19.352 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-10T03:54:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-10T03:54:19.439 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-05-10T03:54:19.572 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-10T03:54:19.614 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-10T03:54:19.689 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-05-10T03:54:19.784 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-10T03:54:19.816 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-10T03:54:19.879 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-05-10T03:54:20.019 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-10T03:54:20.047 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-10T03:54:20.154 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-05-10T03:54:20.243 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-10T03:54:20.268 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 41/162 2024-05-10T03:54:20.411 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-05-10T03:54:20.429 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 42/162 2024-05-10T03:54:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-10T03:54:20.624 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-05-10T03:54:20.654 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T03:54:20.683 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-10T03:54:20.731 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T03:54:20.850 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-05-10T03:54:20.877 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-10T03:54:20.963 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T03:54:21.033 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T03:54:21.040 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-05-10T03:54:21.123 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-10T03:54:21.277 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-10T03:54:21.283 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-05-10T03:54:21.300 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-16.8-9.el8.noarch 70/162 2024-05-10T03:54:21.457 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-10T03:54:21.473 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-10T03:54:21.481 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-10T03:54:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-10T03:54:21.645 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-10T03:54:21.654 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T03:54:21.808 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-05-10T03:54:21.963 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-10T03:54:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-05-10T03:54:22.103 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T03:54:22.172 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-05-10T03:54:22.221 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-10T03:54:22.244 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-05-10T03:54:22.404 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-05-10T03:54:22.463 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T03:54:22.491 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T03:54:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-05-10T03:54:22.671 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-10T03:54:22.763 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-10T03:54:22.773 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-05-10T03:54:22.816 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-10T03:54:22.914 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-10T03:54:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-10T03:54:23.026 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-10T03:54:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-10T03:54:23.242 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-10T03:54:23.288 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-10T03:54:23.369 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-10T03:54:23.445 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-10T03:54:23.496 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-10T03:54:23.567 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-10T03:54:23.588 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-10T03:54:23.703 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-10T03:54:23.748 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-10T03:54:23.793 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-10T03:54:23.884 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-10T03:54:23.928 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-10T03:54:23.959 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-10T03:54:24.087 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-10T03:54:24.114 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-10T03:54:24.126 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-10T03:54:24.314 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-packaging-16.8-9.el8.noarch 56/162 2024-05-10T03:54:24.343 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-10T03:54:24.402 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-10T03:54:24.492 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-05-10T03:54:24.542 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-10T03:54:24.622 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-10T03:54:24.728 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-05-10T03:54:24.745 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-10T03:54:24.775 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-10T03:54:24.909 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-05-10T03:54:24.936 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-10T03:54:25.002 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-10T03:54:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-05-10T03:54:25.160 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-10T03:54:25.246 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-10T03:54:25.308 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-05-10T03:54:25.351 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-10T03:54:25.447 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-10T03:54:25.503 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-10T03:54:25.633 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-05-10T03:54:25.673 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-10T03:54:25.768 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-10T03:54:25.901 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-05-10T03:54:25.988 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-10T03:54:26.157 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-05-10T03:54:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-10T03:54:26.316 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-05-10T03:54:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-10T03:54:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-05-10T03:54:26.792 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-10T03:54:26.885 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-05-10T03:54:27.084 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-10T03:54:27.182 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-05-10T03:54:27.330 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-10T03:54:27.476 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-10T03:54:27.494 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-05-10T03:54:27.658 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-10T03:54:27.753 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-05-10T03:54:27.792 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-10T03:54:27.933 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-05-10T03:54:28.004 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-10T03:54:28.056 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-10T03:54:28.198 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-05-10T03:54:28.272 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-10T03:54:28.281 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-10T03:54:28.416 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-05-10T03:54:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-10T03:54:28.670 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-05-10T03:54:28.755 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-10T03:54:28.930 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-05-10T03:54:29.107 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-05-10T03:54:29.163 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-10T03:54:29.363 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T03:54:29.421 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-05-10T03:54:29.429 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T03:54:29.614 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 104/162 2024-05-10T03:54:29.644 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-10T03:54:29.686 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-10T03:54:29.814 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-10T03:54:29.888 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T03:54:29.966 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T03:54:30.129 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-10T03:54:30.130 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 104/162 2024-05-10T03:54:30.360 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-10T03:54:30.532 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-10T03:54:30.680 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-10T03:54:30.915 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-10T03:54:31.155 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-10T03:54:31.388 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-10T03:54:31.606 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-10T03:54:31.750 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-10T03:54:32.015 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-10T03:54:32.223 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-10T03:54:32.417 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-10T03:54:32.736 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-10T03:54:33.244 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-10T03:54:33.410 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-10T03:54:33.645 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T03:54:33.668 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-10T03:54:33.880 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T03:54:34.095 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T03:54:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T03:54:34.505 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T03:54:34.769 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-10T03:54:34.784 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T03:54:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T03:54:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T03:54:34.988 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T03:54:35.115 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-10T03:54:35.310 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-10T03:54:35.619 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-10T03:54:35.776 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T03:54:35.868 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T03:54:35.903 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-10T03:54:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T03:54:36.093 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-10T03:54:37.041 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-10T03:54:37.210 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T03:54:37.283 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T03:54:37.486 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 104/162 2024-05-10T03:54:41.998 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T03:54:42.237 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T03:54:42.860 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T03:54:43.177 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T03:54:43.266 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T03:54:43.376 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T03:54:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T03:54:54.902 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-05-10T03:54:54.903 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-05-10T03:54:54.903 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2024-05-10T03:54:54.903 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /run 2024-05-10T03:54:54.903 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-05-10T03:54:54.903 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-05-10T03:54:54.903 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-05-10T03:54:54.903 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-05-10T03:54:54.903 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2024-05-10T03:54:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T03:54:55.285 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T03:54:55.746 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T03:54:55.746 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-05-10T03:54:55.746 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-05-10T03:54:55.746 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /dev 2024-05-10T03:54:55.746 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /run 2024-05-10T03:54:55.746 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-05-10T03:54:55.746 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-05-10T03:54:55.746 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-05-10T03:54:55.746 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-05-10T03:54:55.747 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /dev 2024-05-10T03:54:55.747 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:54:56.135 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T03:54:57.029 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T03:54:57.214 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T03:54:57.433 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T03:54:57.601 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T03:54:57.859 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T03:54:58.081 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T03:54:58.124 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T03:54:58.284 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T03:54:58.302 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T03:54:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T03:54:58.468 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T03:54:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T03:54:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T03:54:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T03:55:00.184 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T03:55:00.349 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T03:55:00.523 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 113/162 2024-05-10T03:55:00.735 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 114/162 2024-05-10T03:55:00.967 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T03:55:01.005 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T03:55:01.175 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T03:55:01.341 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 113/162 2024-05-10T03:55:01.546 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 114/162 2024-05-10T03:55:01.759 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T03:55:02.003 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T03:55:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T03:55:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T03:55:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T03:55:03.202 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T03:55:03.383 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T03:55:03.477 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T03:55:03.843 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 118/162 2024-05-10T03:55:03.880 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T03:55:03.880 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2024-05-10T03:55:03.881 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2024-05-10T03:55:03.881 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2024-05-10T03:55:03.881 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /run 2024-05-10T03:55:03.881 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2024-05-10T03:55:03.881 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2024-05-10T03:55:03.881 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2024-05-10T03:55:03.881 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2024-05-10T03:55:03.881 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2024-05-10T03:55:03.881 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:55:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T03:55:04.029 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T03:55:04.068 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T03:55:04.150 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T03:55:04.233 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T03:55:04.242 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T03:55:04.244 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T03:55:04.264 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T03:55:04.604 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 118/162 2024-05-10T03:55:04.764 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T03:55:04.801 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T03:55:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T03:55:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T03:55:05.751 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:18.2.2-1330.g50174015.el8.x86_64 121/162 2024-05-10T03:55:05.962 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T03:55:05.993 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T03:55:05.996 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T03:55:06.206 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T03:55:06.224 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T03:55:06.268 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T03:55:06.426 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T03:55:06.441 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T03:55:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:18.2.2-1330.g50174015.el8.x86_64 121/162 2024-05-10T03:55:06.614 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T03:55:06.655 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T03:55:06.689 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T03:55:06.795 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T03:55:06.859 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T03:55:06.901 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T03:55:07.007 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T03:55:07.092 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T03:55:07.134 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T03:55:07.331 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T03:55:07.349 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T03:55:07.443 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T03:55:07.608 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T03:55:09.302 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T03:55:09.451 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T03:55:09.562 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 113/162 2024-05-10T03:55:09.790 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 114/162 2024-05-10T03:55:10.013 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T03:55:11.045 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T03:55:11.314 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T03:55:11.919 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T03:55:12.282 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T03:55:12.488 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T03:55:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T03:55:12.581 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T03:55:12.853 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 126/162 2024-05-10T03:55:12.943 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 126/162 2024-05-10T03:55:12.963 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 118/162 2024-05-10T03:55:13.156 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T03:55:13.187 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-10T03:55:13.195 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T03:55:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-10T03:55:13.346 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T03:55:13.358 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T03:55:13.424 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-10T03:55:13.491 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-10T03:55:13.892 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 129/162 2024-05-10T03:55:14.104 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-10T03:55:14.242 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-10T03:55:14.404 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-10T03:55:14.498 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-10T03:55:14.512 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-10T03:55:14.675 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-10T03:55:14.742 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-10T03:55:14.885 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:18.2.2-1330.g50174015.el8.x86_64 121/162 2024-05-10T03:55:14.896 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 131/162 2024-05-10T03:55:14.966 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-10T03:55:15.105 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T03:55:15.118 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-10T03:55:15.142 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T03:55:15.162 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 134/162 2024-05-10T03:55:15.311 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-10T03:55:15.321 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T03:55:15.364 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T03:55:15.525 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 134/162 2024-05-10T03:55:15.546 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T03:55:15.904 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T03:55:16.096 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T03:55:17.440 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T03:55:17.748 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T03:55:17.756 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T03:55:17.786 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T03:55:18.057 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T03:55:18.095 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T03:55:18.215 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-10T03:55:18.483 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-10T03:55:18.621 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-10T03:55:18.878 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-10T03:55:18.891 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-10T03:55:19.073 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 139/162 2024-05-10T03:55:19.123 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-10T03:55:19.306 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 140/162 2024-05-10T03:55:19.332 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 139/162 2024-05-10T03:55:19.532 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 140/162 2024-05-10T03:55:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 141/162 2024-05-10T03:55:19.828 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 141/162 2024-05-10T03:55:19.849 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-10T03:55:20.072 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-10T03:55:20.100 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-10T03:55:20.148 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-10T03:55:20.248 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-10T03:55:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-10T03:55:20.389 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-10T03:55:20.402 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-10T03:55:20.427 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-10T03:55:20.597 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-10T03:55:20.606 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-10T03:55:20.648 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-10T03:55:20.728 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-10T03:55:20.758 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-10T03:55:20.923 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-10T03:55:20.931 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-10T03:55:20.945 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-10T03:55:20.995 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T03:55:21.124 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-10T03:55:21.132 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-10T03:55:21.342 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-10T03:55:21.343 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-10T03:55:21.343 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T03:55:21.486 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-10T03:55:21.486 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T03:55:21.617 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T03:55:21.617 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T03:55:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T03:55:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T03:55:21.876 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T03:55:21.876 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T03:55:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T03:55:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T03:55:22.002 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 126/162 2024-05-10T03:55:22.146 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T03:55:22.229 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T03:55:22.317 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-10T03:55:22.357 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-10T03:55:22.442 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-10T03:55:22.567 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-10T03:55:22.587 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-10T03:55:22.625 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-10T03:55:22.807 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-10T03:55:22.807 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T03:55:22.842 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-10T03:55:22.842 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T03:55:22.971 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 129/162 2024-05-10T03:55:23.016 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T03:55:23.016 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T03:55:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T03:55:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T03:55:23.283 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T03:55:23.283 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T03:55:23.297 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-10T03:55:23.334 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T03:55:23.334 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T03:55:23.542 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T03:55:23.542 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T03:55:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T03:55:23.549 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T03:55:23.570 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-10T03:55:23.584 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-10T03:55:23.777 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T03:55:23.778 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T03:55:23.791 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T03:55:23.791 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T03:55:23.830 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-10T03:55:24.017 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T03:55:24.032 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T03:55:24.035 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-10T03:55:24.192 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-10T03:55:24.192 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T03:55:24.216 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-10T03:55:24.216 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T03:55:24.246 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 134/162 2024-05-10T03:55:25.145 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T03:55:25.164 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:18.2.2-1330.g50174015.el8.x86_64 1/162 2024-05-10T03:55:25.164 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2/162 2024-05-10T03:55:25.164 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 3/162 2024-05-10T03:55:25.164 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 4/162 2024-05-10T03:55:25.164 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1330.g50174 5/162 2024-05-10T03:55:25.164 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 6/162 2024-05-10T03:55:25.164 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 7/162 2024-05-10T03:55:25.164 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 8/162 2024-05-10T03:55:25.165 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T03:55:25.165 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T03:55:25.165 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T03:55:25.165 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T03:55:25.165 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 13/162 2024-05-10T03:55:25.165 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 14/162 2024-05-10T03:55:25.165 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 15/162 2024-05-10T03:55:25.165 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 16/162 2024-05-10T03:55:25.165 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 17/162 2024-05-10T03:55:25.165 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 18/162 2024-05-10T03:55:25.165 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T03:55:25.165 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 20/162 2024-05-10T03:55:25.165 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 21/162 2024-05-10T03:55:25.166 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 22/162 2024-05-10T03:55:25.166 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 23/162 2024-05-10T03:55:25.166 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 24/162 2024-05-10T03:55:25.166 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 25/162 2024-05-10T03:55:25.166 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 26/162 2024-05-10T03:55:25.166 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 27/162 2024-05-10T03:55:25.166 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 28/162 2024-05-10T03:55:25.166 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 29/162 2024-05-10T03:55:25.166 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 30/162 2024-05-10T03:55:25.167 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 31/162 2024-05-10T03:55:25.167 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 32/162 2024-05-10T03:55:25.167 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 33/162 2024-05-10T03:55:25.167 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 34/162 2024-05-10T03:55:25.167 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 35/162 2024-05-10T03:55:25.167 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 36/162 2024-05-10T03:55:25.167 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:18.2.2-1330.g50174015.el8.noarch 37/162 2024-05-10T03:55:25.167 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 38/162 2024-05-10T03:55:25.167 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-10T03:55:25.167 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-10T03:55:25.167 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-10T03:55:25.167 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-10T03:55:25.168 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-10T03:55:25.168 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-10T03:55:25.168 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-10T03:55:25.168 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-10T03:55:25.168 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-10T03:55:25.168 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-10T03:55:25.168 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-10T03:55:25.168 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-10T03:55:25.168 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-10T03:55:25.168 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-10T03:55:25.168 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-10T03:55:25.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-10T03:55:25.170 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-10T03:55:25.170 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-10T03:55:25.170 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-10T03:55:25.170 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-10T03:55:25.170 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-05-10T03:55:25.170 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-05-10T03:55:25.170 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-05-10T03:55:25.170 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-05-10T03:55:25.170 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-05-10T03:55:25.170 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-05-10T03:55:25.170 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-05-10T03:55:25.170 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 90/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 91/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 92/162 2024-05-10T03:55:25.171 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 93/162 2024-05-10T03:55:25.172 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 94/162 2024-05-10T03:55:25.172 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 95/162 2024-05-10T03:55:25.172 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 96/162 2024-05-10T03:55:25.172 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 97/162 2024-05-10T03:55:25.172 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 98/162 2024-05-10T03:55:25.172 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 99/162 2024-05-10T03:55:25.172 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 100/162 2024-05-10T03:55:25.172 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 101/162 2024-05-10T03:55:25.172 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 102/162 2024-05-10T03:55:25.172 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 103/162 2024-05-10T03:55:25.172 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 104/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.1-9.el8.x86_64 106/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 107/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 108/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 109/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 110/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 111/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 113/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 114/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 115/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 116/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 117/162 2024-05-10T03:55:25.173 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 118/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 119/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 120/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.1-8.el8.noarch 121/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpq-13.11-1.el8.x86_64 122/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : jq-1.6-7.el8.x86_64 123/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 124/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 125/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 126/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 127/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 128/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 129/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 130/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:18.2.2-1330.g50174015.el8.x86_64 131/162 2024-05-10T03:55:25.174 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 133/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-10T03:55:25.175 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-10T03:55:25.176 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-10T03:55:25.184 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T03:55:25.184 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:18.2.2-1330.g50174015.el8.x86_64 1/162 2024-05-10T03:55:25.184 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2/162 2024-05-10T03:55:25.184 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 3/162 2024-05-10T03:55:25.185 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 4/162 2024-05-10T03:55:25.185 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1330.g50174 5/162 2024-05-10T03:55:25.185 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 6/162 2024-05-10T03:55:25.185 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 7/162 2024-05-10T03:55:25.185 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 8/162 2024-05-10T03:55:25.185 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T03:55:25.185 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T03:55:25.185 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T03:55:25.185 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T03:55:25.185 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 13/162 2024-05-10T03:55:25.186 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 14/162 2024-05-10T03:55:25.186 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 15/162 2024-05-10T03:55:25.186 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 16/162 2024-05-10T03:55:25.186 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 17/162 2024-05-10T03:55:25.186 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 18/162 2024-05-10T03:55:25.186 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T03:55:25.186 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 20/162 2024-05-10T03:55:25.186 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 21/162 2024-05-10T03:55:25.186 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 22/162 2024-05-10T03:55:25.186 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 23/162 2024-05-10T03:55:25.187 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 24/162 2024-05-10T03:55:25.187 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 25/162 2024-05-10T03:55:25.187 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 26/162 2024-05-10T03:55:25.187 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 27/162 2024-05-10T03:55:25.187 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 28/162 2024-05-10T03:55:25.187 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 29/162 2024-05-10T03:55:25.187 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 30/162 2024-05-10T03:55:25.187 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 31/162 2024-05-10T03:55:25.187 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 32/162 2024-05-10T03:55:25.187 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 33/162 2024-05-10T03:55:25.188 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 34/162 2024-05-10T03:55:25.188 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 35/162 2024-05-10T03:55:25.188 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 36/162 2024-05-10T03:55:25.188 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:18.2.2-1330.g50174015.el8.noarch 37/162 2024-05-10T03:55:25.188 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 38/162 2024-05-10T03:55:25.188 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-10T03:55:25.188 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-10T03:55:25.188 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-10T03:55:25.188 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-10T03:55:25.188 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-10T03:55:25.189 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-10T03:55:25.189 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-10T03:55:25.189 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-10T03:55:25.189 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-10T03:55:25.189 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-10T03:55:25.189 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-10T03:55:25.189 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-10T03:55:25.189 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-10T03:55:25.189 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-10T03:55:25.189 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-10T03:55:25.190 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-10T03:55:25.190 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-10T03:55:25.190 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-10T03:55:25.190 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-10T03:55:25.190 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-10T03:55:25.190 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-10T03:55:25.190 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-10T03:55:25.190 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-10T03:55:25.190 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-10T03:55:25.190 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-10T03:55:25.191 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-10T03:55:25.191 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-10T03:55:25.191 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-10T03:55:25.191 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-10T03:55:25.191 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-10T03:55:25.191 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-10T03:55:25.191 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-10T03:55:25.191 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-10T03:55:25.191 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-05-10T03:55:25.192 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-05-10T03:55:25.192 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-05-10T03:55:25.192 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-05-10T03:55:25.192 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-05-10T03:55:25.192 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-05-10T03:55:25.192 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-05-10T03:55:25.192 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-05-10T03:55:25.192 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-05-10T03:55:25.192 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-05-10T03:55:25.192 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-05-10T03:55:25.193 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-05-10T03:55:25.193 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-05-10T03:55:25.193 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-05-10T03:55:25.193 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-05-10T03:55:25.193 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-05-10T03:55:25.193 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-05-10T03:55:25.193 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-05-10T03:55:25.193 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-05-10T03:55:25.193 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-05-10T03:55:25.193 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-05-10T03:55:25.193 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-05-10T03:55:25.194 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-05-10T03:55:25.194 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-05-10T03:55:25.194 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-05-10T03:55:25.194 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-05-10T03:55:25.194 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-05-10T03:55:25.194 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-05-10T03:55:25.194 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-05-10T03:55:25.194 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-05-10T03:55:25.195 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-05-10T03:55:25.195 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 103/162 2024-05-10T03:55:25.195 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-10T03:55:25.195 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-10T03:55:25.195 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-10T03:55:25.195 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-10T03:55:25.195 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-10T03:55:25.195 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-10T03:55:25.195 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-10T03:55:25.195 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-10T03:55:25.196 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-10T03:55:25.196 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-10T03:55:25.196 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-10T03:55:25.196 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-10T03:55:25.196 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-10T03:55:25.196 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-10T03:55:25.196 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-10T03:55:25.196 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-10T03:55:25.196 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-10T03:55:25.196 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-10T03:55:25.197 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-10T03:55:25.197 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-10T03:55:25.197 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-10T03:55:25.197 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-10T03:55:25.197 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-10T03:55:25.197 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-10T03:55:25.197 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-10T03:55:25.197 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-10T03:55:25.197 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-10T03:55:25.197 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:18.2.2-1330.g50174015.el8.x86_64 131/162 2024-05-10T03:55:25.198 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-10T03:55:25.198 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 133/162 2024-05-10T03:55:25.198 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-10T03:55:25.198 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-10T03:55:25.198 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-10T03:55:25.198 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-10T03:55:25.198 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-10T03:55:25.198 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-10T03:55:25.198 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-10T03:55:25.199 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-10T03:55:25.199 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-10T03:55:25.199 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-10T03:55:25.199 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-10T03:55:25.199 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-10T03:55:25.199 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-10T03:55:25.199 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-10T03:55:25.199 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-10T03:55:25.199 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-10T03:55:25.199 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-10T03:55:25.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-10T03:55:25.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-10T03:55:25.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-10T03:55:25.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-10T03:55:25.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-10T03:55:25.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-10T03:55:25.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-10T03:55:25.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-10T03:55:25.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-10T03:55:25.200 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-10T03:55:25.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-10T03:55:26.564 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T03:55:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T03:55:26.917 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T03:55:27.377 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-10T03:55:27.732 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-10T03:55:28.022 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-10T03:55:28.211 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 139/162 2024-05-10T03:55:28.444 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 140/162 2024-05-10T03:55:28.735 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 141/162 2024-05-10T03:55:29.045 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-10T03:55:29.269 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-10T03:55:29.344 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-10T03:55:29.586 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-10T03:55:29.599 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-10T03:55:29.769 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-10T03:55:29.778 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-10T03:55:29.987 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-10T03:55:30.157 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-10T03:55:30.354 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-10T03:55:30.523 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-10T03:55:30.523 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T03:55:30.814 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T03:55:30.814 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T03:55:31.081 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T03:55:31.081 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T03:55:31.325 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T03:55:31.554 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-10T03:55:31.721 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-10T03:55:31.970 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-10T03:55:31.970 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T03:55:32.246 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T03:55:32.246 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T03:55:32.480 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T03:55:32.480 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T03:55:32.729 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T03:55:32.730 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T03:55:32.954 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T03:55:32.955 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T03:55:33.188 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T03:55:33.329 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-10T03:55:33.329 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T03:55:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T03:55:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:18.2.2-1330.g50174015.el8.x86_64 1/162 2024-05-10T03:55:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2/162 2024-05-10T03:55:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 3/162 2024-05-10T03:55:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 4/162 2024-05-10T03:55:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1330.g50174 5/162 2024-05-10T03:55:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 6/162 2024-05-10T03:55:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 7/162 2024-05-10T03:55:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 8/162 2024-05-10T03:55:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T03:55:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T03:55:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T03:55:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T03:55:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 13/162 2024-05-10T03:55:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 14/162 2024-05-10T03:55:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 15/162 2024-05-10T03:55:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 16/162 2024-05-10T03:55:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 17/162 2024-05-10T03:55:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 18/162 2024-05-10T03:55:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T03:55:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 20/162 2024-05-10T03:55:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 21/162 2024-05-10T03:55:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 22/162 2024-05-10T03:55:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 23/162 2024-05-10T03:55:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 24/162 2024-05-10T03:55:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 25/162 2024-05-10T03:55:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 26/162 2024-05-10T03:55:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 27/162 2024-05-10T03:55:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 28/162 2024-05-10T03:55:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 29/162 2024-05-10T03:55:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 30/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 31/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 32/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 33/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 34/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 35/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 36/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:18.2.2-1330.g50174015.el8.noarch 37/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 38/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-10T03:55:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-10T03:55:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-10T03:55:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-05-10T03:55:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-05-10T03:55:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-05-10T03:55:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-05-10T03:55:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-05-10T03:55:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-05-10T03:55:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-05-10T03:55:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-05-10T03:55:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-05-10T03:55:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-05-10T03:55:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-05-10T03:55:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-05-10T03:55:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-05-10T03:55:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-05-10T03:55:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-05-10T03:55:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-05-10T03:55:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-05-10T03:55:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-05-10T03:55:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 90/162 2024-05-10T03:55:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 91/162 2024-05-10T03:55:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 92/162 2024-05-10T03:55:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 93/162 2024-05-10T03:55:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 94/162 2024-05-10T03:55:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 95/162 2024-05-10T03:55:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 96/162 2024-05-10T03:55:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 97/162 2024-05-10T03:55:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 98/162 2024-05-10T03:55:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 99/162 2024-05-10T03:55:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-10T03:55:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-10T03:55:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-10T03:55:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-10T03:55:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-10T03:55:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-10T03:55:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-10T03:55:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-10T03:55:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-10T03:55:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-10T03:55:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-10T03:55:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-10T03:55:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-10T03:55:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-10T03:55:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-10T03:55:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-10T03:55:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-10T03:55:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-10T03:55:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:18.2.2-1330.g50174015.el8.x86_64 131/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 133/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-10T03:55:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-10T03:55:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-10T03:55:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-10T03:55:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-10T03:55:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-10T03:55:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-10T03:55:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-10T03:55:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-10T03:55:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-10T03:55:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-10T03:55:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-10T03:55:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-10T03:55:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-10T03:55:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-10T03:55:39.796 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-10T03:55:39.796 INFO:teuthology.orchestra.run.smithi161.stdout:Installed products updated. 2024-05-10T03:55:40.421 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T03:55:40.421 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-05-10T03:55:40.421 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-10T03:55:40.421 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-10T03:55:40.421 INFO:teuthology.orchestra.run.smithi161.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-10T03:55:40.421 INFO:teuthology.orchestra.run.smithi161.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:40.421 INFO:teuthology.orchestra.run.smithi161.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:40.421 INFO:teuthology.orchestra.run.smithi161.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:40.421 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.421 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.422 INFO:teuthology.orchestra.run.smithi161.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:40.422 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:40.422 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-10T03:55:40.422 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-10T03:55:40.422 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:40.422 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:40.422 INFO:teuthology.orchestra.run.smithi161.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-10T03:55:40.422 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-10T03:55:40.422 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-05-10T03:55:40.422 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-10T03:55:40.422 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.1-8.el8.noarch 2024-05-10T03:55:40.422 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.0.4-10.el8.x86_64 2024-05-10T03:55:40.422 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.423 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:40.424 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.424 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.424 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.424 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:40.424 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:40.424 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.424 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:40.424 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-10T03:55:40.424 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.1-9.el8.x86_64 2024-05-10T03:55:40.424 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-10T03:55:40.424 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-10T03:55:40.424 INFO:teuthology.orchestra.run.smithi161.stdout: jq-1.6-7.el8.x86_64 2024-05-10T03:55:40.424 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-10T03:55:40.425 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-10T03:55:40.425 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-10T03:55:40.425 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.425 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.425 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.425 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-10T03:55:40.425 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-10T03:55:40.425 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-10T03:55:40.425 INFO:teuthology.orchestra.run.smithi161.stdout: libpq-13.11-1.el8.x86_64 2024-05-10T03:55:40.426 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-10T03:55:40.426 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-10T03:55:40.426 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.426 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.426 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-10T03:55:40.426 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.426 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-10T03:55:40.426 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-10T03:55:40.426 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-10T03:55:40.426 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-10T03:55:40.426 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:40.426 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-10T03:55:40.426 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-10T03:55:40.427 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-10T03:55:40.427 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-10T03:55:40.427 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-10T03:55:40.427 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-10T03:55:40.427 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-10T03:55:40.427 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-10T03:55:40.427 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-10T03:55:40.427 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-10T03:55:40.427 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T03:55:40.427 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T03:55:40.427 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T03:55:40.427 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-10T03:55:40.427 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-10T03:55:40.428 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-10T03:55:40.429 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-10T03:55:40.429 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-10T03:55:40.429 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-10T03:55:40.429 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-10T03:55:40.429 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-10T03:55:40.429 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-10T03:55:40.429 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-10T03:55:40.429 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-10T03:55:40.429 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-10T03:55:40.429 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-10T03:55:40.429 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-10T03:55:40.429 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-10T03:55:40.429 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-10T03:55:40.430 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-10T03:55:40.430 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-10T03:55:40.430 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-2.6-1.el8.noarch 2024-05-10T03:55:40.430 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-10T03:55:40.430 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-10T03:55:40.430 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-10T03:55:40.430 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-10T03:55:40.430 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-10T03:55:40.430 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-10T03:55:40.430 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.430 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.430 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-10T03:55:40.430 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-10T03:55:40.431 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-10T03:55:40.432 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.432 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.432 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:40.432 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-10T03:55:40.432 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-10T03:55:40.432 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-10T03:55:40.432 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-10T03:55:40.432 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-10T03:55:40.432 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-10T03:55:40.433 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T03:55:40.433 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-05-10T03:55:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-10T03:55:40.441 INFO:teuthology.orchestra.run.smithi150.stdout:Installed products updated. 2024-05-10T03:55:40.666 DEBUG:teuthology.parallel:result is None 2024-05-10T03:55:41.194 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:55:41.194 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-05-10T03:55:41.194 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-10T03:55:41.194 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-10T03:55:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-10T03:55:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-10T03:55:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-10T03:55:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.1-8.el8.noarch 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.0.4-10.el8.x86_64 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-10T03:55:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.1-9.el8.x86_64 2024-05-10T03:55:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-10T03:55:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-10T03:55:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: jq-1.6-7.el8.x86_64 2024-05-10T03:55:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-10T03:55:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-10T03:55:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-10T03:55:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: libpq-13.11-1.el8.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-10T03:55:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-10T03:55:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-10T03:55:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-10T03:55:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-10T03:55:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-10T03:55:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-10T03:55:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-10T03:55:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-10T03:55:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-10T03:55:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-10T03:55:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-10T03:55:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-10T03:55:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T03:55:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T03:55:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T03:55:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-10T03:55:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-10T03:55:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-10T03:55:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-10T03:55:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-10T03:55:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-10T03:55:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-2.6-1.el8.noarch 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-10T03:55:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-10T03:55:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-10T03:55:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-10T03:55:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-10T03:55:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-10T03:55:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-10T03:55:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-10T03:55:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-10T03:55:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-10T03:55:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-10T03:55:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-10T03:55:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-10T03:55:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-10T03:55:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-10T03:55:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-10T03:55:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-10T03:55:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:55:41.206 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-05-10T03:55:41.417 DEBUG:teuthology.parallel:result is None 2024-05-10T03:55:50.424 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-10T03:55:50.425 INFO:teuthology.orchestra.run.smithi181.stdout:Installed products updated. 2024-05-10T03:55:51.020 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:55:51.020 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2024-05-10T03:55:51.020 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-10T03:55:51.020 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-10T03:55:51.020 INFO:teuthology.orchestra.run.smithi181.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-10T03:55:51.020 INFO:teuthology.orchestra.run.smithi181.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:51.020 INFO:teuthology.orchestra.run.smithi181.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:51.020 INFO:teuthology.orchestra.run.smithi181.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:51.021 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.021 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.021 INFO:teuthology.orchestra.run.smithi181.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:51.021 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:51.021 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-10T03:55:51.021 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-10T03:55:51.021 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:51.021 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:51.021 INFO:teuthology.orchestra.run.smithi181.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-10T03:55:51.021 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-10T03:55:51.021 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-05-10T03:55:51.022 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-10T03:55:51.022 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.1-8.el8.noarch 2024-05-10T03:55:51.022 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.0.4-10.el8.x86_64 2024-05-10T03:55:51.022 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.022 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.022 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.022 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.022 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:51.022 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.022 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.022 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.022 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:51.022 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T03:55:51.023 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.1-9.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: jq-1.6-7.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: libpq-13.11-1.el8.x86_64 2024-05-10T03:55:51.024 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-10T03:55:51.025 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-10T03:55:51.026 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-10T03:55:51.026 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-10T03:55:51.026 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-10T03:55:51.026 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-10T03:55:51.026 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T03:55:51.026 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T03:55:51.026 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T03:55:51.026 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-10T03:55:51.026 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-10T03:55:51.026 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-10T03:55:51.026 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-10T03:55:51.026 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-10T03:55:51.026 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.027 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.027 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.029 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-10T03:55:51.029 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-10T03:55:51.029 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-10T03:55:51.029 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-10T03:55:51.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-10T03:55:51.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-10T03:55:51.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-10T03:55:51.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-10T03:55:51.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-10T03:55:51.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-10T03:55:51.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-10T03:55:51.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-10T03:55:51.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-10T03:55:51.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-10T03:55:51.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-10T03:55:51.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-10T03:55:51.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-2.6-1.el8.noarch 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-10T03:55:51.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-10T03:55:51.033 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.033 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.033 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T03:55:51.033 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-10T03:55:51.033 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-10T03:55:51.033 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-10T03:55:51.033 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-10T03:55:51.033 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-10T03:55:51.033 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-10T03:55:51.033 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:55:51.034 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-05-10T03:55:51.264 DEBUG:teuthology.parallel:result is None 2024-05-10T03:55:51.264 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:55:51.498 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T03:55:51.526 INFO:teuthology.orchestra.run.smithi150.stdout:18.2.2-1330.g50174015.el8 2024-05-10T03:55:51.527 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1330.g50174015.el8 2024-05-10T03:55:51.527 INFO:teuthology.task.install:The correct ceph version 18.2.2-1330.g50174015 is installed. 2024-05-10T03:55:51.529 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:55:51.668 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T03:55:51.698 INFO:teuthology.orchestra.run.smithi161.stdout:18.2.2-1330.g50174015.el8 2024-05-10T03:55:51.698 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1330.g50174015.el8 2024-05-10T03:55:51.699 INFO:teuthology.task.install:The correct ceph version 18.2.2-1330.g50174015 is installed. 2024-05-10T03:55:51.700 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:55:51.841 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T03:55:51.867 INFO:teuthology.orchestra.run.smithi181.stdout:18.2.2-1330.g50174015.el8 2024-05-10T03:55:51.867 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1330.g50174015.el8 2024-05-10T03:55:51.867 INFO:teuthology.task.install:The correct ceph version 18.2.2-1330.g50174015 is installed. 2024-05-10T03:55:51.869 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-10T03:55:51.869 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:55:51.869 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T03:55:51.919 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T03:55:51.919 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T03:55:51.969 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T03:55:51.970 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T03:55:52.015 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-10T03:55:52.015 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:55:52.015 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T03:55:52.046 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T03:55:52.114 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T03:55:52.115 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T03:55:52.147 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T03:55:52.219 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T03:55:52.219 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T03:55:52.250 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T03:55:52.319 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-10T03:55:52.319 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:55:52.319 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T03:55:52.349 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T03:55:52.420 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T03:55:52.420 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T03:55:52.452 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T03:55:52.524 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T03:55:52.524 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T03:55:52.553 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T03:55:52.621 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-10T03:55:52.621 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:55:52.621 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T03:55:52.651 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T03:55:52.721 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T03:55:52.721 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T03:55:52.752 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T03:55:52.825 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T03:55:52.826 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T03:55:52.854 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T03:55:52.923 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-10T03:55:53.018 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-10T03:55:53.018 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:55:53.018 INFO:tasks.cephadm:Cluster fsid is 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:55:53.019 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-10T03:55:53.019 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.150', 'mon.b': '172.21.15.161', 'mon.c': '172.21.15.181'} 2024-05-10T03:55:53.019 INFO:tasks.cephadm:First mon is mon.a on smithi150 2024-05-10T03:55:53.019 INFO:tasks.cephadm:First mgr is x 2024-05-10T03:55:53.019 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-10T03:55:53.019 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-05-10T03:55:53.053 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-05-10T03:55:53.086 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname $(hostname -s) 2024-05-10T03:55:53.119 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-10T03:55:53.120 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:55:53.334 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79270/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1330-g50174015', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1330.g50174015'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-10 02:06:05.327930', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240509.171335-reef-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-10T03:55:53.472 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240509.171335-reef-debug, sha1 50174015a7e79f951b988949abe236791f52d4da from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:55:53.473 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/x86_64/flavors/default/cephadm 2024-05-10T03:55:53.474 DEBUG:teuthology.orchestra.run.smithi150:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T03:55:53.821 INFO:teuthology.orchestra.run.smithi150.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 10 03:55 /home/ubuntu/cephtest/cephadm 2024-05-10T03:55:53.821 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T03:55:54.362 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 10 03:55 /home/ubuntu/cephtest/cephadm 2024-05-10T03:55:54.363 DEBUG:teuthology.orchestra.run.smithi181:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T03:55:54.711 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 10 03:55 /home/ubuntu/cephtest/cephadm 2024-05-10T03:55:54.712 DEBUG:teuthology.orchestra.run.smithi150:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T03:55:54.734 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T03:55:54.756 DEBUG:teuthology.orchestra.run.smithi181:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T03:55:54.794 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da on all hosts... 2024-05-10T03:55:54.794 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da pull 2024-05-10T03:55:54.797 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da pull 2024-05-10T03:55:54.799 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da pull 2024-05-10T03:55:55.714 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T03:55:55.761 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T03:55:56.233 INFO:teuthology.orchestra.run.smithi181.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T03:56:25.571 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-10T03:56:25.571 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable)", 2024-05-10T03:56:25.571 INFO:teuthology.orchestra.run.smithi181.stdout: "image_id": "0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b", 2024-05-10T03:56:25.571 INFO:teuthology.orchestra.run.smithi181.stdout: "repo_digests": [ 2024-05-10T03:56:25.571 INFO:teuthology.orchestra.run.smithi181.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:042b1f8e1a7496c1ea8ed86a601031b3e73743a9905239589edc685eef0b229c" 2024-05-10T03:56:25.571 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-05-10T03:56:25.571 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-10T03:56:26.462 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-10T03:56:26.462 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable)", 2024-05-10T03:56:26.462 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b", 2024-05-10T03:56:26.462 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2024-05-10T03:56:26.462 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:042b1f8e1a7496c1ea8ed86a601031b3e73743a9905239589edc685eef0b229c" 2024-05-10T03:56:26.462 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-10T03:56:26.462 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-10T03:56:31.323 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T03:56:31.323 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable)", 2024-05-10T03:56:31.323 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b", 2024-05-10T03:56:31.323 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-05-10T03:56:31.323 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:042b1f8e1a7496c1ea8ed86a601031b3e73743a9905239589edc685eef0b229c" 2024-05-10T03:56:31.323 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T03:56:31.323 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T03:56:31.343 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-05-10T03:56:31.373 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-05-10T03:56:31.411 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph 2024-05-10T03:56:31.448 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-05-10T03:56:31.475 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-05-10T03:56:31.507 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /etc/ceph 2024-05-10T03:56:31.538 INFO:tasks.cephadm:Writing seed config... 2024-05-10T03:56:31.539 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-10T03:56:31.539 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-10T03:56:31.539 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-10T03:56:31.539 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-10T03:56:31.539 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-10T03:56:31.539 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-10T03:56:31.539 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-10T03:56:31.539 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-10T03:56:31.539 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-10T03:56:31.539 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-10T03:56:31.539 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-10T03:56:31.540 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-10T03:56:31.541 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-10T03:56:31.542 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-10T03:56:31.542 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-10T03:56:31.542 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:56:31.542 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-10T03:56:31.560 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 325b4c82-0e81-11ef-bc98-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-10T03:56:31.560 DEBUG:teuthology.orchestra.run.smithi150:mon.a> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.a.service 2024-05-10T03:56:31.602 DEBUG:teuthology.orchestra.run.smithi150:mgr.x> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.x.service 2024-05-10T03:56:31.628 INFO:journalctl@ceph.mon.a.smithi150.stdout:-- Logs begin at Fri 2024-05-10 03:40:30 UTC. -- 2024-05-10T03:56:31.629 INFO:tasks.cephadm:Bootstrapping... 2024-05-10T03:56:31.629 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da -v bootstrap --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.150 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-10T03:56:31.654 INFO:journalctl@ceph.mgr.x.smithi150.stdout:-- Logs begin at Fri 2024-05-10 03:40:30 UTC. -- 2024-05-10T03:56:31.789 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-05-10T03:56:31.789 INFO:teuthology.orchestra.run.smithi150.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da', '-v', 'bootstrap', '--fsid', '325b4c82-0e81-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.150', '--skip-admin-label'] 2024-05-10T03:56:31.824 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stdout 4.6.1 2024-05-10T03:56:31.825 INFO:teuthology.orchestra.run.smithi150.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-10T03:56:31.825 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying podman|docker is present... 2024-05-10T03:56:31.861 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stdout 4.6.1 2024-05-10T03:56:31.862 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying lvm2 is present... 2024-05-10T03:56:31.862 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying time synchronization is in place... 2024-05-10T03:56:31.869 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-10T03:56:31.869 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-10T03:56:31.874 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-10T03:56:31.875 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout inactive 2024-05-10T03:56:31.880 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout enabled 2024-05-10T03:56:31.886 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout active 2024-05-10T03:56:31.886 INFO:teuthology.orchestra.run.smithi150.stdout:Unit chronyd.service is enabled and running 2024-05-10T03:56:31.887 INFO:teuthology.orchestra.run.smithi150.stdout:Repeating the final host check... 2024-05-10T03:56:31.925 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stdout 4.6.1 2024-05-10T03:56:31.925 INFO:teuthology.orchestra.run.smithi150.stdout:podman (/bin/podman) version 4.6.1 is present 2024-05-10T03:56:31.925 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl is present 2024-05-10T03:56:31.925 INFO:teuthology.orchestra.run.smithi150.stdout:lvcreate is present 2024-05-10T03:56:31.931 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-10T03:56:31.931 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-10T03:56:31.936 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-10T03:56:31.937 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout inactive 2024-05-10T03:56:31.942 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout enabled 2024-05-10T03:56:31.949 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout active 2024-05-10T03:56:31.949 INFO:teuthology.orchestra.run.smithi150.stdout:Unit chronyd.service is enabled and running 2024-05-10T03:56:31.949 INFO:teuthology.orchestra.run.smithi150.stdout:Host looks OK 2024-05-10T03:56:31.949 INFO:teuthology.orchestra.run.smithi150.stdout:Cluster fsid: 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:56:31.949 INFO:teuthology.orchestra.run.smithi150.stdout:Acquiring lock 139897179159016 on /run/cephadm/325b4c82-0e81-11ef-bc98-c7b262605968.lock 2024-05-10T03:56:31.949 INFO:teuthology.orchestra.run.smithi150.stdout:Lock 139897179159016 acquired on /run/cephadm/325b4c82-0e81-11ef-bc98-c7b262605968.lock 2024-05-10T03:56:31.950 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying IP 172.21.15.150 port 3300 ... 2024-05-10T03:56:31.950 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying IP 172.21.15.150 port 6789 ... 2024-05-10T03:56:31.950 INFO:teuthology.orchestra.run.smithi150.stdout:Base mon IP(s) is [172.21.15.150:3300, 172.21.15.150:6789], mon addrv is [v2:172.21.15.150:3300,v1:172.21.15.150:6789] 2024-05-10T03:56:31.954 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2024-05-10T03:56:31.954 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.150 metric 100 2024-05-10T03:56:31.956 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-10T03:56:31.956 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-05-10T03:56:31.957 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-05-10T03:56:31.959 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-10T03:56:31.959 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-10T03:56:31.959 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-10T03:56:31.959 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-10T03:56:31.959 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout inet6 fe80::d613:fbd0:302a:c5e5/64 scope link noprefixroute 2024-05-10T03:56:31.959 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-10T03:56:31.960 INFO:teuthology.orchestra.run.smithi150.stdout:Mon IP `172.21.15.150` is in CIDR network `172.21.0.0/20` 2024-05-10T03:56:31.960 INFO:teuthology.orchestra.run.smithi150.stdout:Mon IP `172.21.15.150` is in CIDR network `172.21.0.0/20` 2024-05-10T03:56:31.960 INFO:teuthology.orchestra.run.smithi150.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-10T03:56:31.961 INFO:teuthology.orchestra.run.smithi150.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-10T03:56:31.962 INFO:teuthology.orchestra.run.smithi150.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T03:56:32.323 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stdout 0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b 2024-05-10T03:56:32.323 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T03:56:32.323 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stderr Getting image source signatures 2024-05-10T03:56:32.323 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stderr Copying blob sha256:5889b1d3c79fb904da685578f1acb24d7c4091d50b19c3ba92279423a786328a 2024-05-10T03:56:32.323 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-10T03:56:32.323 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stderr Copying config sha256:0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b 2024-05-10T03:56:32.323 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-10T03:56:33.019 INFO:teuthology.orchestra.run.smithi150.stdout:ceph: stdout ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable) 2024-05-10T03:56:33.019 INFO:teuthology.orchestra.run.smithi150.stdout:Ceph version: ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable) 2024-05-10T03:56:33.019 INFO:teuthology.orchestra.run.smithi150.stdout:Extracting ceph user uid/gid from container image... 2024-05-10T03:56:33.607 INFO:teuthology.orchestra.run.smithi150.stdout:stat: stdout 167 167 2024-05-10T03:56:33.607 INFO:teuthology.orchestra.run.smithi150.stdout:Creating initial keys... 2024-05-10T03:56:34.212 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph-authtool: stdout AQDxmj1msCpfMxAAnkr8u/Md9wLJCrDgpYE+XQ== 2024-05-10T03:56:34.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph-authtool: stdout AQDymj1mt+mDGxAAiW5yJ49kmbWvHNVPdaY9SA== 2024-05-10T03:56:37.053 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph-authtool: stdout AQDzmj1m06TJMhAA+6A237Of4XYrZ5J/HXHubg== 2024-05-10T03:56:37.055 INFO:teuthology.orchestra.run.smithi150.stdout:Creating initial monmap... 2024-05-10T03:56:38.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-10T03:56:38.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-10T03:56:38.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:56:38.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-10T03:56:38.026 INFO:teuthology.orchestra.run.smithi150.stdout:monmaptool for a [v2:172.21.15.150:3300,v1:172.21.15.150:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-10T03:56:38.026 INFO:teuthology.orchestra.run.smithi150.stdout:setting min_mon_release = pacific 2024-05-10T03:56:38.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/monmaptool: set fsid to 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:56:38.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-10T03:56:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:56:38.026 INFO:teuthology.orchestra.run.smithi150.stdout:Creating mon... 2024-05-10T03:56:38.680 INFO:teuthology.orchestra.run.smithi150.stdout:create mon.a on 2024-05-10T03:56:38.812 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-05-10T03:56:38.903 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-10T03:56:39.026 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-325b4c82-0e81-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-325b4c82-0e81-11ef-bc98-c7b262605968.target. 2024-05-10T03:56:39.026 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-325b4c82-0e81-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-325b4c82-0e81-11ef-bc98-c7b262605968.target. 2024-05-10T03:56:39.213 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.a 2024-05-10T03:56:39.213 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Failed to reset failed state of unit ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-10T03:56:39.301 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-325b4c82-0e81-11ef-bc98-c7b262605968.target.wants/ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-325b4c82-0e81-11ef-bc98-c7b262605968@.service. 2024-05-10T03:56:39.434 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:39 smithi150 systemd[1]: Starting Ceph mon.a for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T03:56:39.666 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T03:56:39.666 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout disabled 2024-05-10T03:56:39.673 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T03:56:39.673 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout inactive 2024-05-10T03:56:39.673 INFO:teuthology.orchestra.run.smithi150.stdout:firewalld.service is not enabled 2024-05-10T03:56:39.673 INFO:teuthology.orchestra.run.smithi150.stdout:Not possible to enable service . firewalld.service is not available 2024-05-10T03:56:39.673 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for mon to start... 2024-05-10T03:56:39.674 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for mon... 2024-05-10T03:56:39.695 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:39 smithi150 bash[41956]: 6285c71b3b7489d003c82406d72b204317be221c1245accc186c8e1e6e8a36b2 2024-05-10T03:56:39.695 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:39 smithi150 systemd[1]: Started Ceph mon.a for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T03:56:39.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:39 smithi150 ceph-mon[41993]: mkfs 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:56:39.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:39 smithi150 ceph-mon[41993]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T03:56:40.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout cluster: 2024-05-10T03:56:40.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout id: 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:56:40.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-10T03:56:40.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T03:56:40.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout services: 2024-05-10T03:56:40.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.449263s) 2024-05-10T03:56:40.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-10T03:56:40.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-10T03:56:40.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T03:56:40.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout data: 2024-05-10T03:56:40.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-10T03:56:40.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-10T03:56:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-10T03:56:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout pgs: 2024-05-10T03:56:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T03:56:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.059+0000 7f8e1b562700 1 Processor -- start 2024-05-10T03:56:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.059+0000 7f8e1b562700 1 -- start start 2024-05-10T03:56:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.059+0000 7f8e1b562700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 0x7f8e14107c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.059+0000 7f8e1b562700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e141081f0 con 0x7f8e14107810 2024-05-10T03:56:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.060+0000 7f8e192fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 0x7f8e14107c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.060+0000 7f8e192fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 0x7f8e14107c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34240/0 (socket says 172.21.15.150:34240) 2024-05-10T03:56:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.060+0000 7f8e192fe700 1 -- 172.21.15.150:0/3897085143 learned_addr learned my addr 172.21.15.150:0/3897085143 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.060+0000 7f8e192fe700 1 -- 172.21.15.150:0/3897085143 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e14106490 con 0x7f8e14107810 2024-05-10T03:56:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.060+0000 7f8e192fe700 1 --2- 172.21.15.150:0/3897085143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 0x7f8e14107c20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8e100098a0 tx=0x7f8e10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681da74b452f6fb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.060+0000 7f8e0bfff700 1 -- 172.21.15.150:0/3897085143 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e10004160 con 0x7f8e14107810 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.060+0000 7f8e0bfff700 1 -- 172.21.15.150:0/3897085143 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e100042c0 con 0x7f8e14107810 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.061+0000 7f8e0bfff700 1 -- 172.21.15.150:0/3897085143 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e100045a0 con 0x7f8e14107810 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.061+0000 7f8e1b562700 1 -- 172.21.15.150:0/3897085143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 msgr2=0x7f8e14107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.061+0000 7f8e1b562700 1 --2- 172.21.15.150:0/3897085143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 0x7f8e14107c20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8e100098a0 tx=0x7f8e10009bb0 comp rx=0 tx=0).stop 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.061+0000 7f8e1b562700 1 -- 172.21.15.150:0/3897085143 shutdown_connections 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.061+0000 7f8e1b562700 1 --2- 172.21.15.150:0/3897085143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 0x7f8e14107c20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.061+0000 7f8e1b562700 1 -- 172.21.15.150:0/3897085143 >> 172.21.15.150:0/3897085143 conn(0x7f8e141010b0 msgr2=0x7f8e14103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.061+0000 7f8e1b562700 1 -- 172.21.15.150:0/3897085143 shutdown_connections 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.061+0000 7f8e1b562700 1 -- 172.21.15.150:0/3897085143 wait complete. 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.062+0000 7f8e1b562700 1 Processor -- start 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.062+0000 7f8e1b562700 1 -- start start 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.062+0000 7f8e1b562700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 0x7f8e14199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.062+0000 7f8e1b562700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1419a040 con 0x7f8e14107810 2024-05-10T03:56:40.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.062+0000 7f8e192fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 0x7f8e14199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.062+0000 7f8e192fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 0x7f8e14199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34244/0 (socket says 172.21.15.150:34244) 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.062+0000 7f8e192fe700 1 -- 172.21.15.150:0/3709999037 learned_addr learned my addr 172.21.15.150:0/3709999037 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.063+0000 7f8e192fe700 1 -- 172.21.15.150:0/3709999037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10009550 con 0x7f8e14107810 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.063+0000 7f8e192fe700 1 --2- 172.21.15.150:0/3709999037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 0x7f8e14199b00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8e10004000 tx=0x7f8e10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.063+0000 7f8e0a7fc700 1 -- 172.21.15.150:0/3709999037 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e10011630 con 0x7f8e14107810 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.063+0000 7f8e0a7fc700 1 -- 172.21.15.150:0/3709999037 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e10011790 con 0x7f8e14107810 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.063+0000 7f8e1b562700 1 -- 172.21.15.150:0/3709999037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1419a240 con 0x7f8e14107810 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.063+0000 7f8e0a7fc700 1 -- 172.21.15.150:0/3709999037 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e10011a30 con 0x7f8e14107810 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.063+0000 7f8e1b562700 1 -- 172.21.15.150:0/3709999037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1419a6e0 con 0x7f8e14107810 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.063+0000 7f8e0a7fc700 1 -- 172.21.15.150:0/3709999037 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f8e10021070 con 0x7f8e14107810 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.063+0000 7f8e0a7fc700 1 -- 172.21.15.150:0/3709999037 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8e1001b780 con 0x7f8e14107810 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.064+0000 7f8e1b562700 1 -- 172.21.15.150:0/3709999037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e14106800 con 0x7f8e14107810 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.066+0000 7f8e0a7fc700 1 -- 172.21.15.150:0/3709999037 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f8e1001ad00 con 0x7f8e14107810 2024-05-10T03:56:40.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.136+0000 7f8e1b562700 1 -- 172.21.15.150:0/3709999037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f8e14192da0 con 0x7f8e14107810 2024-05-10T03:56:40.505 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.136+0000 7f8e0a7fc700 1 -- 172.21.15.150:0/3709999037 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f8e1001a460 con 0x7f8e14107810 2024-05-10T03:56:40.505 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.137+0000 7f8e1b562700 1 -- 172.21.15.150:0/3709999037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 msgr2=0x7f8e14199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:40.505 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.137+0000 7f8e1b562700 1 --2- 172.21.15.150:0/3709999037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 0x7f8e14199b00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8e10004000 tx=0x7f8e10004750 comp rx=0 tx=0).stop 2024-05-10T03:56:40.505 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.137+0000 7f8e1b562700 1 -- 172.21.15.150:0/3709999037 shutdown_connections 2024-05-10T03:56:40.505 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.137+0000 7f8e1b562700 1 --2- 172.21.15.150:0/3709999037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e14107810 0x7f8e14199b00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:40.505 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.137+0000 7f8e1b562700 1 -- 172.21.15.150:0/3709999037 >> 172.21.15.150:0/3709999037 conn(0x7f8e141010b0 msgr2=0x7f8e1418dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:40.505 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.137+0000 7f8e1b562700 1 -- 172.21.15.150:0/3709999037 shutdown_connections 2024-05-10T03:56:40.505 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.137+0000 7f8e1b562700 1 -- 172.21.15.150:0/3709999037 wait complete. 2024-05-10T03:56:40.505 INFO:teuthology.orchestra.run.smithi150.stdout:mon is available 2024-05-10T03:56:40.505 INFO:teuthology.orchestra.run.smithi150.stdout:Assimilating anything we can from ceph.conf... 2024-05-10T03:56:40.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:40 smithi150 ceph-mon[41993]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T03:56:40.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:40 smithi150 ceph-mon[41993]: monmap e1: 1 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T03:56:40.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:40 smithi150 ceph-mon[41993]: fsmap 2024-05-10T03:56:40.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:40 smithi150 ceph-mon[41993]: osdmap e1: 0 total, 0 up, 0 in 2024-05-10T03:56:40.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:40 smithi150 ceph-mon[41993]: mgrmap e1: no daemons active 2024-05-10T03:56:40.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:40 smithi150 ceph-mon[41993]: from='client.? 172.21.15.150:0/3709999037' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-10T03:56:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T03:56:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout [global] 2024-05-10T03:56:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout fsid = 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:56:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.150:3300,v1:172.21.15.150:6789] 2024-05-10T03:56:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-10T03:56:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-10T03:56:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-10T03:56:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-10T03:56:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T03:56:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-10T03:56:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout [osd] 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.866+0000 7f0f4d6c4700 1 Processor -- start 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.866+0000 7f0f4d6c4700 1 -- start start 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.866+0000 7f0f4d6c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 0x7f0f48106620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.866+0000 7f0f4d6c4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f48106c10 con 0x7f0f48106210 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.866+0000 7f0f46ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 0x7f0f48106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.866+0000 7f0f46ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 0x7f0f48106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34256/0 (socket says 172.21.15.150:34256) 2024-05-10T03:56:41.335 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.866+0000 7f0f46ffd700 1 -- 172.21.15.150:0/919544382 learned_addr learned my addr 172.21.15.150:0/919544382 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.866+0000 7f0f46ffd700 1 -- 172.21.15.150:0/919544382 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f481073e0 con 0x7f0f48106210 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.867+0000 7f0f46ffd700 1 --2- 172.21.15.150:0/919544382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 0x7f0f48106620 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0f380098a0 tx=0x7f0f38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64af3b4cc637d9f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.867+0000 7f0f45ffb700 1 -- 172.21.15.150:0/919544382 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f38004160 con 0x7f0f48106210 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.867+0000 7f0f45ffb700 1 -- 172.21.15.150:0/919544382 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f380042c0 con 0x7f0f48106210 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.867+0000 7f0f45ffb700 1 -- 172.21.15.150:0/919544382 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f380045a0 con 0x7f0f48106210 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.867+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/919544382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 msgr2=0x7f0f48106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.867+0000 7f0f4d6c4700 1 --2- 172.21.15.150:0/919544382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 0x7f0f48106620 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0f380098a0 tx=0x7f0f38009bb0 comp rx=0 tx=0).stop 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.867+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/919544382 shutdown_connections 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.867+0000 7f0f4d6c4700 1 --2- 172.21.15.150:0/919544382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 0x7f0f48106620 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.867+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/919544382 >> 172.21.15.150:0/919544382 conn(0x7f0f48101870 msgr2=0x7f0f48103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.868+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/919544382 shutdown_connections 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.868+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/919544382 wait complete. 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.868+0000 7f0f4d6c4700 1 Processor -- start 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.869+0000 7f0f4d6c4700 1 -- start start 2024-05-10T03:56:41.336 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.869+0000 7f0f4d6c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 0x7f0f48199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.869+0000 7f0f4d6c4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4819a070 con 0x7f0f48106210 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.869+0000 7f0f46ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 0x7f0f48199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.869+0000 7f0f46ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 0x7f0f48199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34266/0 (socket says 172.21.15.150:34266) 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.869+0000 7f0f46ffd700 1 -- 172.21.15.150:0/3794446299 learned_addr learned my addr 172.21.15.150:0/3794446299 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.869+0000 7f0f46ffd700 1 -- 172.21.15.150:0/3794446299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f38009550 con 0x7f0f48106210 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.869+0000 7f0f46ffd700 1 --2- 172.21.15.150:0/3794446299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 0x7f0f48199b30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0f38004000 tx=0x7f0f38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.870+0000 7f0f2ffff700 1 -- 172.21.15.150:0/3794446299 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f38011630 con 0x7f0f48106210 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.870+0000 7f0f2ffff700 1 -- 172.21.15.150:0/3794446299 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f38011790 con 0x7f0f48106210 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.870+0000 7f0f2ffff700 1 -- 172.21.15.150:0/3794446299 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f38011a30 con 0x7f0f48106210 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.870+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/3794446299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4819a270 con 0x7f0f48106210 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.870+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/3794446299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4819a710 con 0x7f0f48106210 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.870+0000 7f0f2ffff700 1 -- 172.21.15.150:0/3794446299 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0f38021070 con 0x7f0f48106210 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.870+0000 7f0f2ffff700 1 -- 172.21.15.150:0/3794446299 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0f3801b780 con 0x7f0f48106210 2024-05-10T03:56:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.871+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/3794446299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f481932d0 con 0x7f0f48106210 2024-05-10T03:56:41.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.873+0000 7f0f2ffff700 1 -- 172.21.15.150:0/3794446299 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f0f3801acc0 con 0x7f0f48106210 2024-05-10T03:56:41.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.940+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/3794446299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0f48107750 con 0x7f0f48106210 2024-05-10T03:56:41.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.945+0000 7f0f2ffff700 1 -- 172.21.15.150:0/3794446299 <== mon.0 v2:172.21.15.150:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0f3801ae60 con 0x7f0f48106210 2024-05-10T03:56:41.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.945+0000 7f0f2ffff700 1 -- 172.21.15.150:0/3794446299 <== mon.0 v2:172.21.15.150:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f0f38035590 con 0x7f0f48106210 2024-05-10T03:56:41.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.946+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/3794446299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 msgr2=0x7f0f48199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:41.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.946+0000 7f0f4d6c4700 1 --2- 172.21.15.150:0/3794446299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 0x7f0f48199b30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0f38004000 tx=0x7f0f38004750 comp rx=0 tx=0).stop 2024-05-10T03:56:41.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.946+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/3794446299 shutdown_connections 2024-05-10T03:56:41.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.946+0000 7f0f4d6c4700 1 --2- 172.21.15.150:0/3794446299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f48106210 0x7f0f48199b30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:41.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.946+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/3794446299 >> 172.21.15.150:0/3794446299 conn(0x7f0f48101870 msgr2=0x7f0f48190410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:41.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.946+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/3794446299 shutdown_connections 2024-05-10T03:56:41.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:40.946+0000 7f0f4d6c4700 1 -- 172.21.15.150:0/3794446299 wait complete. 2024-05-10T03:56:41.338 INFO:teuthology.orchestra.run.smithi150.stdout:Generating new minimal ceph.conf... 2024-05-10T03:56:41.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:41 smithi150 ceph-mon[41993]: from='client.? 172.21.15.150:0/3794446299' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-10T03:56:41.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:41 smithi150 ceph-mon[41993]: from='client.? 172.21.15.150:0/3794446299' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-10T03:56:42.132 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.690+0000 7fa28a94c700 1 Processor -- start 2024-05-10T03:56:42.132 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.690+0000 7fa28a94c700 1 -- start start 2024-05-10T03:56:42.132 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.690+0000 7fa28a94c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 0x7fa284106600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:42.132 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.690+0000 7fa28a94c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa284106bf0 con 0x7fa2841061f0 2024-05-10T03:56:42.132 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.691+0000 7fa283fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 0x7fa284106600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:42.133 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.691+0000 7fa283fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 0x7fa284106600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34278/0 (socket says 172.21.15.150:34278) 2024-05-10T03:56:42.133 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.691+0000 7fa283fff700 1 -- 172.21.15.150:0/687249077 learned_addr learned my addr 172.21.15.150:0/687249077 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:42.133 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.691+0000 7fa283fff700 1 -- 172.21.15.150:0/687249077 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2841073c0 con 0x7fa2841061f0 2024-05-10T03:56:42.133 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.691+0000 7fa283fff700 1 --2- 172.21.15.150:0/687249077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 0x7fa284106600 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa27400bb40 tx=0x7fa27400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17758027e8c0f29 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:42.133 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.691+0000 7fa282ffd700 1 -- 172.21.15.150:0/687249077 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa27400dbe0 con 0x7fa2841061f0 2024-05-10T03:56:42.133 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.691+0000 7fa282ffd700 1 -- 172.21.15.150:0/687249077 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa2740044d0 con 0x7fa2841061f0 2024-05-10T03:56:42.133 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.691+0000 7fa282ffd700 1 -- 172.21.15.150:0/687249077 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa274010470 con 0x7fa2841061f0 2024-05-10T03:56:42.133 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.692+0000 7fa28a94c700 1 -- 172.21.15.150:0/687249077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 msgr2=0x7fa284106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:42.133 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.692+0000 7fa28a94c700 1 --2- 172.21.15.150:0/687249077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 0x7fa284106600 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa27400bb40 tx=0x7fa27400be50 comp rx=0 tx=0).stop 2024-05-10T03:56:42.133 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.692+0000 7fa28a94c700 1 -- 172.21.15.150:0/687249077 shutdown_connections 2024-05-10T03:56:42.133 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.692+0000 7fa28a94c700 1 --2- 172.21.15.150:0/687249077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 0x7fa284106600 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:42.133 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.692+0000 7fa28a94c700 1 -- 172.21.15.150:0/687249077 >> 172.21.15.150:0/687249077 conn(0x7fa284101850 msgr2=0x7fa284103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:42.133 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.692+0000 7fa28a94c700 1 -- 172.21.15.150:0/687249077 shutdown_connections 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.692+0000 7fa28a94c700 1 -- 172.21.15.150:0/687249077 wait complete. 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.693+0000 7fa28a94c700 1 Processor -- start 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.693+0000 7fa28a94c700 1 -- start start 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.693+0000 7fa28a94c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 0x7fa2841992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.693+0000 7fa28a94c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa284199810 con 0x7fa2841061f0 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.693+0000 7fa283fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 0x7fa2841992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.693+0000 7fa283fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 0x7fa2841992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34290/0 (socket says 172.21.15.150:34290) 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.693+0000 7fa283fff700 1 -- 172.21.15.150:0/1500599257 learned_addr learned my addr 172.21.15.150:0/1500599257 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.694+0000 7fa283fff700 1 -- 172.21.15.150:0/1500599257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27400b7f0 con 0x7fa2841061f0 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.694+0000 7fa283fff700 1 --2- 172.21.15.150:0/1500599257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 0x7fa2841992d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa274003c60 tx=0x7fa274003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.694+0000 7fa2817fa700 1 -- 172.21.15.150:0/1500599257 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa27401e070 con 0x7fa2841061f0 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.694+0000 7fa2817fa700 1 -- 172.21.15.150:0/1500599257 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa274004210 con 0x7fa2841061f0 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.694+0000 7fa28a94c700 1 -- 172.21.15.150:0/1500599257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa284199a10 con 0x7fa2841061f0 2024-05-10T03:56:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.694+0000 7fa2817fa700 1 -- 172.21.15.150:0/1500599257 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2740228d0 con 0x7fa2841061f0 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.694+0000 7fa28a94c700 1 -- 172.21.15.150:0/1500599257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa284199eb0 con 0x7fa2841061f0 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.694+0000 7fa2817fa700 1 -- 172.21.15.150:0/1500599257 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa27402c440 con 0x7fa2841061f0 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.694+0000 7fa2817fa700 1 -- 172.21.15.150:0/1500599257 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa2740195a0 con 0x7fa2841061f0 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.695+0000 7fa27affd700 1 -- 172.21.15.150:0/1500599257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2680052f0 con 0x7fa2841061f0 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.697+0000 7fa2817fa700 1 -- 172.21.15.150:0/1500599257 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fa274027070 con 0x7fa2841061f0 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.764+0000 7fa27affd700 1 -- 172.21.15.150:0/1500599257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa268005870 con 0x7fa2841061f0 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.764+0000 7fa2817fa700 1 -- 172.21.15.150:0/1500599257 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fa274010cf0 con 0x7fa2841061f0 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.765+0000 7fa27affd700 1 -- 172.21.15.150:0/1500599257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 msgr2=0x7fa2841992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.765+0000 7fa27affd700 1 --2- 172.21.15.150:0/1500599257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 0x7fa2841992d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa274003c60 tx=0x7fa274003980 comp rx=0 tx=0).stop 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.765+0000 7fa27affd700 1 -- 172.21.15.150:0/1500599257 shutdown_connections 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.765+0000 7fa27affd700 1 --2- 172.21.15.150:0/1500599257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2841061f0 0x7fa2841992d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.765+0000 7fa27affd700 1 -- 172.21.15.150:0/1500599257 >> 172.21.15.150:0/1500599257 conn(0x7fa284101850 msgr2=0x7fa28418f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.765+0000 7fa27affd700 1 -- 172.21.15.150:0/1500599257 shutdown_connections 2024-05-10T03:56:42.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:41.765+0000 7fa27affd700 1 -- 172.21.15.150:0/1500599257 wait complete. 2024-05-10T03:56:42.136 INFO:teuthology.orchestra.run.smithi150.stdout:Restarting the monitor... 2024-05-10T03:56:42.332 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:42 smithi150 systemd[1]: Stopping Ceph mon.a for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T03:56:42.660 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:42 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mon-a[41970]: 2024-05-10T03:56:42.327+0000 7f58d5d33700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T03:56:42.660 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:42 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mon-a[41970]: 2024-05-10T03:56:42.327+0000 7f58d5d33700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-10T03:56:42.660 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:42 smithi150 bash[42318]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mon-a 2024-05-10T03:56:42.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:42 smithi150 systemd[1]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.a.service: Succeeded. 2024-05-10T03:56:42.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:42 smithi150 systemd[1]: Stopped Ceph mon.a for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T03:56:42.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:42 smithi150 systemd[1]: Starting Ceph mon.a for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T03:56:43.146 INFO:teuthology.orchestra.run.smithi150.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-10T03:56:43.148 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 bash[42439]: 5de85c52a53e324645c73dea70eed8964a3782f7026701f08d032d573bcd4046 2024-05-10T03:56:43.148 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 systemd[1]: Started Ceph mon.a for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T03:56:43.148 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T03:56:43.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable), process ceph-mon, pid 7 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: pidfile_write: ignore empty --pid-file 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: load: jerasure load: lrc 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: RocksDB version: 7.9.2 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Git sha 0 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Compile date 2024-05-10 00:51:33 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: DB SUMMARY 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: DB Session ID: Y0J939CSRE6FACR66XMF 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: CURRENT file: CURRENT 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 97606 ; 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.error_if_exists: 0 2024-05-10T03:56:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.create_if_missing: 0 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.paranoid_checks: 1 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.env: 0x559b47092460 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.info_log: 0x559b48a15760 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.statistics: (nil) 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.use_fsync: 0 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_log_file_size: 0 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T03:56:43.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T03:56:43.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T03:56:43.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.allow_fallocate: 1 2024-05-10T03:56:43.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T03:56:43.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T03:56:43.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.use_direct_reads: 0 2024-05-10T03:56:43.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T03:56:43.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T03:56:43.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.db_log_dir: 2024-05-10T03:56:43.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.wal_dir: 2024-05-10T03:56:43.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T03:56:43.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T03:56:43.468 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.write_buffer_manager: 0x559b489590e0 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T03:56:43.469 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.unordered_write: 0 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.row_cache: None 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.wal_filter: None 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.two_write_queues: 0 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.wal_compression: 0 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.atomic_flush: 0 2024-05-10T03:56:43.470 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.log_readahead_size: 0 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_background_jobs: 2 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_background_compactions: -1 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_subcompactions: 1 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T03:56:43.471 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_open_files: -1 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_background_flushes: -1 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Compression algorithms supported: 2024-05-10T03:56:43.472 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: kZSTD supported: 0 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: kXpressCompression supported: 0 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: kZlibCompression supported: 1 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: kSnappyCompression supported: 1 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: kLZ4Compression supported: 1 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: kBZip2Compression supported: 0 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.merge_operator: 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compaction_filter: None 2024-05-10T03:56:43.473 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559b48a15880) 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout: cache_index_and_filter_blocks: 1 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout: pin_top_level_index_and_filter: 1 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout: index_type: 0 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout: data_block_index_type: 0 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout: index_shortening: 1 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout: checksum: 4 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout: no_block_cache: 0 2024-05-10T03:56:43.474 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_cache: 0x559b489db1f0 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_cache_name: BinnedLRUCache 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_cache_options: 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: capacity : 536870912 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: num_shard_bits : 4 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: strict_capacity_limit : 0 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: high_pri_pool_ratio: 0.000 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_cache_compressed: (nil) 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: persistent_cache: (nil) 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_size: 4096 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_size_deviation: 10 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_restart_interval: 16 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: index_block_restart_interval: 1 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: metadata_block_size: 4096 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: partition_filters: 0 2024-05-10T03:56:43.475 INFO:journalctl@ceph.mon.a.smithi150.stdout: use_delta_encoding: 1 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout: filter_policy: bloomfilter 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout: whole_key_filtering: 1 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout: verify_compression: 0 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout: read_amp_bytes_per_bit: 0 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout: format_version: 5 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout: enable_index_compression: 1 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_align: 0 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout: max_auto_readahead_size: 262144 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout: prepopulate_block_cache: 0 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout: initial_auto_readahead_size: 8192 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compression: NoCompression 2024-05-10T03:56:43.476 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.num_levels: 7 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T03:56:43.477 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T03:56:43.478 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T03:56:43.479 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.table_properties_collectors: 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.inplace_update_support: 0 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T03:56:43.480 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.bloom_locality: 0 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.max_successive_merges: 0 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.ttl: 2592000 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.enable_blob_files: false 2024-05-10T03:56:43.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.min_blob_size: 0 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f202080a-2b51-4772-9f7a-6b33cf696651 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715313403154796, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-10T03:56:43.482 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715313403155971, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 307, "table_properties": {"data_size": 91012, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13614, "raw_average_key_size": 50, "raw_value_size": 83590, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 270, "num_filter_entries": 270, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715313403, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f202080a-2b51-4772-9f7a-6b33cf696651", "db_session_id": "Y0J939CSRE6FACR66XMF", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715313403156053, "job": 1, "event": "recovery_finished"} 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559b4896dc00 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: DB pointer 0x559b4d75c000 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout: ** DB Stats ** 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T03:56:43.483 INFO:journalctl@ceph.mon.a.smithi150.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: ** Compaction Stats [default] ** 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: L0 2/0 92.67 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: Sum 2/0 92.67 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: ** Compaction Stats [default] ** 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T03:56:43.484 INFO:journalctl@ceph.mon.a.smithi150.stdout: 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout: Cumulative compaction: 0.00 GB write, 24.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout: Interval compaction: 0.00 GB write, 24.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout: Block cache BinnedLRUCache@0x559b489db1f0#7 capacity: 512.00 MB usage: 1.38 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.91 KB,0.000172853%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout: 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: starting mon.a rank 0 at public addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] at bind addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:56:43.485 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: mon.a@-1(???) e1 preinit fsid 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: mon.a@-1(???).mds e1 new map 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: mon.a@-1(???).mds e1 print_map 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout: e1 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout: legacy client fscid: -1 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout: 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout: No filesystems configured 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: monmap e1: 1 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T03:56:43.486 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: fsmap 2024-05-10T03:56:43.487 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: osdmap e1: 0 total, 0 up, 0 in 2024-05-10T03:56:43.487 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:43 smithi150 ceph-mon[42477]: mgrmap e1: no daemons active 2024-05-10T03:56:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.523+0000 7f60b2a24700 1 Processor -- start 2024-05-10T03:56:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.523+0000 7f60b2a24700 1 -- start start 2024-05-10T03:56:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.523+0000 7f60b2a24700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 0x7f60ac10a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.523+0000 7f60b2a24700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac07ae20 con 0x7f60ac107dc0 2024-05-10T03:56:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.523+0000 7f60abfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 0x7f60ac10a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.523+0000 7f60abfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 0x7f60ac10a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34296/0 (socket says 172.21.15.150:34296) 2024-05-10T03:56:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.523+0000 7f60abfff700 1 -- 172.21.15.150:0/3184499442 learned_addr learned my addr 172.21.15.150:0/3184499442 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.524+0000 7f60abfff700 1 -- 172.21.15.150:0/3184499442 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac10a720 con 0x7f60ac107dc0 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.524+0000 7f60abfff700 1 --2- 172.21.15.150:0/3184499442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 0x7f60ac10a1e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f609c0098a0 tx=0x7f609c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0473bf5485ca7b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.524+0000 7f60aaffd700 1 -- 172.21.15.150:0/3184499442 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f609c004160 con 0x7f60ac107dc0 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.524+0000 7f60aaffd700 1 -- 172.21.15.150:0/3184499442 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f609c0042c0 con 0x7f60ac107dc0 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.524+0000 7f60aaffd700 1 -- 172.21.15.150:0/3184499442 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f609c011a00 con 0x7f60ac107dc0 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.525+0000 7f60b2a24700 1 -- 172.21.15.150:0/3184499442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 msgr2=0x7f60ac10a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.525+0000 7f60b2a24700 1 --2- 172.21.15.150:0/3184499442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 0x7f60ac10a1e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f609c0098a0 tx=0x7f609c009bb0 comp rx=0 tx=0).stop 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.525+0000 7f60b2a24700 1 -- 172.21.15.150:0/3184499442 shutdown_connections 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.525+0000 7f60b2a24700 1 --2- 172.21.15.150:0/3184499442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 0x7f60ac10a1e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.525+0000 7f60b2a24700 1 -- 172.21.15.150:0/3184499442 >> 172.21.15.150:0/3184499442 conn(0x7f60ac101880 msgr2=0x7f60ac103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.525+0000 7f60b2a24700 1 -- 172.21.15.150:0/3184499442 shutdown_connections 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.525+0000 7f60b2a24700 1 -- 172.21.15.150:0/3184499442 wait complete. 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.525+0000 7f60b2a24700 1 Processor -- start 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.526+0000 7f60b2a24700 1 -- start start 2024-05-10T03:56:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.526+0000 7f60b2a24700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 0x7f60ac107540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.526+0000 7f60b2a24700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac107a80 con 0x7f60ac107dc0 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.526+0000 7f60abfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 0x7f60ac107540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.526+0000 7f60abfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 0x7f60ac107540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34312/0 (socket says 172.21.15.150:34312) 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.526+0000 7f60abfff700 1 -- 172.21.15.150:0/2987370699 learned_addr learned my addr 172.21.15.150:0/2987370699 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.526+0000 7f60abfff700 1 -- 172.21.15.150:0/2987370699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c009550 con 0x7f60ac107dc0 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.526+0000 7f60abfff700 1 --2- 172.21.15.150:0/2987370699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 0x7f60ac107540 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f609c011710 tx=0x7f609c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.527+0000 7f60a97fa700 1 -- 172.21.15.150:0/2987370699 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f609c003aa0 con 0x7f60ac107dc0 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.527+0000 7f60b2a24700 1 -- 172.21.15.150:0/2987370699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ac105cd0 con 0x7f60ac107dc0 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.527+0000 7f60a97fa700 1 -- 172.21.15.150:0/2987370699 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f609c003c00 con 0x7f60ac107dc0 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.527+0000 7f60a97fa700 1 -- 172.21.15.150:0/2987370699 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f609c024800 con 0x7f60ac107dc0 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.527+0000 7f60b2a24700 1 -- 172.21.15.150:0/2987370699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac106170 con 0x7f60ac107dc0 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.527+0000 7f60a97fa700 1 -- 172.21.15.150:0/2987370699 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f609c024c60 con 0x7f60ac107dc0 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.527+0000 7f60a97fa700 1 -- 172.21.15.150:0/2987370699 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f609c0193c0 con 0x7f60ac107dc0 2024-05-10T03:56:44.009 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.528+0000 7f60b2a24700 1 -- 172.21.15.150:0/2987370699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ac062840 con 0x7f60ac107dc0 2024-05-10T03:56:44.010 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.530+0000 7f60a97fa700 1 -- 172.21.15.150:0/2987370699 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f609c01abd0 con 0x7f60ac107dc0 2024-05-10T03:56:44.010 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.596+0000 7f60b2a24700 1 -- 172.21.15.150:0/2987370699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f60ac10b4c0 con 0x7f60ac107dc0 2024-05-10T03:56:44.010 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.602+0000 7f60a97fa700 1 -- 172.21.15.150:0/2987370699 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f609c01ad70 con 0x7f60ac107dc0 2024-05-10T03:56:44.010 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.603+0000 7f60b2a24700 1 -- 172.21.15.150:0/2987370699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 msgr2=0x7f60ac107540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:44.010 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.603+0000 7f60b2a24700 1 --2- 172.21.15.150:0/2987370699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 0x7f60ac107540 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f609c011710 tx=0x7f609c011ad0 comp rx=0 tx=0).stop 2024-05-10T03:56:44.010 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.603+0000 7f60b2a24700 1 -- 172.21.15.150:0/2987370699 shutdown_connections 2024-05-10T03:56:44.010 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.603+0000 7f60b2a24700 1 --2- 172.21.15.150:0/2987370699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac107dc0 0x7f60ac107540 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:44.010 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.603+0000 7f60b2a24700 1 -- 172.21.15.150:0/2987370699 >> 172.21.15.150:0/2987370699 conn(0x7f60ac101880 msgr2=0x7f60ac102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:44.010 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.603+0000 7f60b2a24700 1 -- 172.21.15.150:0/2987370699 shutdown_connections 2024-05-10T03:56:44.010 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:43.603+0000 7f60b2a24700 1 -- 172.21.15.150:0/2987370699 wait complete. 2024-05-10T03:56:44.010 INFO:teuthology.orchestra.run.smithi150.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-10T03:56:44.024 INFO:teuthology.orchestra.run.smithi150.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-10T03:56:44.024 INFO:teuthology.orchestra.run.smithi150.stdout:Creating mgr... 2024-05-10T03:56:44.024 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-10T03:56:44.025 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-10T03:56:44.209 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.x 2024-05-10T03:56:44.209 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Failed to reset failed state of unit ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-10T03:56:44.298 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-325b4c82-0e81-11ef-bc98-c7b262605968.target.wants/ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-325b4c82-0e81-11ef-bc98-c7b262605968@.service. 2024-05-10T03:56:44.510 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:44 smithi150 systemd[1]: Starting Ceph mgr.x for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T03:56:44.682 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T03:56:44.682 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout disabled 2024-05-10T03:56:44.689 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T03:56:44.689 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout inactive 2024-05-10T03:56:44.689 INFO:teuthology.orchestra.run.smithi150.stdout:firewalld.service is not enabled 2024-05-10T03:56:44.689 INFO:teuthology.orchestra.run.smithi150.stdout:Not possible to enable service . firewalld.service is not available 2024-05-10T03:56:44.697 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T03:56:44.697 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout disabled 2024-05-10T03:56:44.706 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T03:56:44.706 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout inactive 2024-05-10T03:56:44.706 INFO:teuthology.orchestra.run.smithi150.stdout:firewalld.service is not enabled 2024-05-10T03:56:44.707 INFO:teuthology.orchestra.run.smithi150.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-10T03:56:44.707 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for mgr to start... 2024-05-10T03:56:44.707 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for mgr... 2024-05-10T03:56:44.821 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:44 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2987370699' entity='client.admin' 2024-05-10T03:56:44.821 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:44 smithi150 bash[42712]: 0eaac6f82a17de5f9937f76053776e40bbf0ccf8adce2b965b1a842216a04efa 2024-05-10T03:56:44.821 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:44 smithi150 systemd[1]: Started Ceph mgr.x for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T03:56:45.376 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:45 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:45.098+0000 7f6028722200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T03:56:45.376 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:45 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:45.196+0000 7f6028722200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T03:56:45.376 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:45 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:45.287+0000 7f6028722200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T03:56:45.376 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:45 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:45.372+0000 7f6028722200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T03:56:45.488 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T03:56:45.488 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T03:56:45.488 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsid": "325b4c82-0e81-11ef-bc98-c7b262605968", 2024-05-10T03:56:45.488 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 0 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_age": 1, 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T03:56:45.489 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:45.490 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-10T03:56:45.490 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T03:56:45.490 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:45.490 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T03:56:45.490 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:45.490 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T03:56:45.490 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T03:56:45.490 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T03:56:45.498 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T03:56:45.498 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T03:56:45.498 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T03:56:45.498 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:45.498 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T03:56:45.498 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T03:56:45.498 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T03:56:45.498 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T03:56:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T03:56:39.690289+0000", 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T03:56:45.500 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.070+0000 7fc33d691700 1 Processor -- start 2024-05-10T03:56:45.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.070+0000 7fc33d691700 1 -- start start 2024-05-10T03:56:45.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.070+0000 7fc33d691700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 0x7fc338106630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:45.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.070+0000 7fc33d691700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338106c20 con 0x7fc338106220 2024-05-10T03:56:45.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.071+0000 7fc336ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 0x7fc338106630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:45.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.071+0000 7fc336ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 0x7fc338106630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34342/0 (socket says 172.21.15.150:34342) 2024-05-10T03:56:45.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.071+0000 7fc336ffd700 1 -- 172.21.15.150:0/2763953685 learned_addr learned my addr 172.21.15.150:0/2763953685 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:45.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.071+0000 7fc336ffd700 1 -- 172.21.15.150:0/2763953685 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3381073f0 con 0x7fc338106220 2024-05-10T03:56:45.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.071+0000 7fc336ffd700 1 --2- 172.21.15.150:0/2763953685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 0x7fc338106630 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc3280098a0 tx=0x7fc328009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a40af70d07afd40 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:45.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.071+0000 7fc335ffb700 1 -- 172.21.15.150:0/2763953685 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc328004160 con 0x7fc338106220 2024-05-10T03:56:45.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.071+0000 7fc335ffb700 1 -- 172.21.15.150:0/2763953685 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc3280042c0 con 0x7fc338106220 2024-05-10T03:56:45.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.071+0000 7fc335ffb700 1 -- 172.21.15.150:0/2763953685 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc328011a00 con 0x7fc338106220 2024-05-10T03:56:45.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.072+0000 7fc33d691700 1 -- 172.21.15.150:0/2763953685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 msgr2=0x7fc338106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:45.501 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.072+0000 7fc33d691700 1 --2- 172.21.15.150:0/2763953685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 0x7fc338106630 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc3280098a0 tx=0x7fc328009bb0 comp rx=0 tx=0).stop 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.072+0000 7fc33d691700 1 -- 172.21.15.150:0/2763953685 shutdown_connections 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.072+0000 7fc33d691700 1 --2- 172.21.15.150:0/2763953685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 0x7fc338106630 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.072+0000 7fc33d691700 1 -- 172.21.15.150:0/2763953685 >> 172.21.15.150:0/2763953685 conn(0x7fc338101860 msgr2=0x7fc338103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.072+0000 7fc33d691700 1 -- 172.21.15.150:0/2763953685 shutdown_connections 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.072+0000 7fc33d691700 1 -- 172.21.15.150:0/2763953685 wait complete. 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.073+0000 7fc33d691700 1 Processor -- start 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.073+0000 7fc33d691700 1 -- start start 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.073+0000 7fc33d691700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 0x7fc33807a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.073+0000 7fc33d691700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33807ae30 con 0x7fc338106220 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.073+0000 7fc336ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 0x7fc33807a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.073+0000 7fc336ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 0x7fc33807a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34346/0 (socket says 172.21.15.150:34346) 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.073+0000 7fc336ffd700 1 -- 172.21.15.150:0/2475925055 learned_addr learned my addr 172.21.15.150:0/2475925055 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.074+0000 7fc336ffd700 1 -- 172.21.15.150:0/2475925055 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc328009550 con 0x7fc338106220 2024-05-10T03:56:45.502 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.074+0000 7fc336ffd700 1 --2- 172.21.15.150:0/2475925055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 0x7fc33807a8f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc328009130 tx=0x7fc328011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:45.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.074+0000 7fc31ffff700 1 -- 172.21.15.150:0/2475925055 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3280038b0 con 0x7fc338106220 2024-05-10T03:56:45.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.074+0000 7fc33d691700 1 -- 172.21.15.150:0/2475925055 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33819e290 con 0x7fc338106220 2024-05-10T03:56:45.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.074+0000 7fc31ffff700 1 -- 172.21.15.150:0/2475925055 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc328003a10 con 0x7fc338106220 2024-05-10T03:56:45.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.074+0000 7fc31ffff700 1 -- 172.21.15.150:0/2475925055 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc328019550 con 0x7fc338106220 2024-05-10T03:56:45.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.074+0000 7fc33d691700 1 -- 172.21.15.150:0/2475925055 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33819e670 con 0x7fc338106220 2024-05-10T03:56:45.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.074+0000 7fc31ffff700 1 -- 172.21.15.150:0/2475925055 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fc328019b10 con 0x7fc338106220 2024-05-10T03:56:45.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.074+0000 7fc31ffff700 1 -- 172.21.15.150:0/2475925055 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc32802de10 con 0x7fc338106220 2024-05-10T03:56:45.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.075+0000 7fc33d691700 1 -- 172.21.15.150:0/2475925055 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc338062840 con 0x7fc338106220 2024-05-10T03:56:45.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.077+0000 7fc31ffff700 1 -- 172.21.15.150:0/2475925055 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fc328003b80 con 0x7fc338106220 2024-05-10T03:56:45.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.150+0000 7fc33d691700 1 -- 172.21.15.150:0/2475925055 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc33810a7f0 con 0x7fc338106220 2024-05-10T03:56:45.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.150+0000 7fc31ffff700 1 -- 172.21.15.150:0/2475925055 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc32802d440 con 0x7fc338106220 2024-05-10T03:56:45.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.151+0000 7fc33d691700 1 -- 172.21.15.150:0/2475925055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 msgr2=0x7fc33807a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:45.503 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.151+0000 7fc33d691700 1 --2- 172.21.15.150:0/2475925055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 0x7fc33807a8f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc328009130 tx=0x7fc328011ad0 comp rx=0 tx=0).stop 2024-05-10T03:56:45.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.151+0000 7fc33d691700 1 -- 172.21.15.150:0/2475925055 shutdown_connections 2024-05-10T03:56:45.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.151+0000 7fc33d691700 1 --2- 172.21.15.150:0/2475925055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338106220 0x7fc33807a8f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:45.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.151+0000 7fc33d691700 1 -- 172.21.15.150:0/2475925055 >> 172.21.15.150:0/2475925055 conn(0x7fc338101860 msgr2=0x7fc338102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:45.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.152+0000 7fc33d691700 1 -- 172.21.15.150:0/2475925055 shutdown_connections 2024-05-10T03:56:45.504 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:45.152+0000 7fc33d691700 1 -- 172.21.15.150:0/2475925055 wait complete. 2024-05-10T03:56:45.504 INFO:teuthology.orchestra.run.smithi150.stdout:mgr not available, waiting (1/15)... 2024-05-10T03:56:45.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:45 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2475925055' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T03:56:46.270 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:46 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:46.016+0000 7f6028722200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:56:46.621 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:46 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:46.267+0000 7f6028722200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T03:56:46.621 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:46 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:46.352+0000 7f6028722200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T03:56:46.621 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:46 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:46.460+0000 7f6028722200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T03:56:46.622 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:46 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:46.618+0000 7f6028722200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T03:56:46.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:46 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:46.744+0000 7f6028722200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T03:56:46.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:46 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:46.829+0000 7f6028722200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T03:56:47.464 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:47 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:47.198+0000 7f6028722200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T03:56:47.903 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:47 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:47.593+0000 7f6028722200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T03:56:47.903 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:47 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:47.798+0000 7f6028722200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T03:56:48.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:48 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2292745311' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T03:56:48.168 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:47 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:47.900+0000 7f6028722200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T03:56:48.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T03:56:48.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T03:56:48.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsid": "325b4c82-0e81-11ef-bc98-c7b262605968", 2024-05-10T03:56:48.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T03:56:48.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T03:56:48.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T03:56:48.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T03:56:48.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:48.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 0 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:48.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T03:56:48.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:48.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T03:56:39.690289+0000", 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.885+0000 7f6b82416700 1 Processor -- start 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.885+0000 7f6b82416700 1 -- start start 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.886+0000 7f6b82416700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 0x7f6b7c10a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.886+0000 7f6b82416700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c07ae20 con 0x7f6b7c107dd0 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.886+0000 7f6b7bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 0x7f6b7c10a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.886+0000 7f6b7bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 0x7f6b7c10a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34362/0 (socket says 172.21.15.150:34362) 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.886+0000 7f6b7bfff700 1 -- 172.21.15.150:0/3651664306 learned_addr learned my addr 172.21.15.150:0/3651664306 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.886+0000 7f6b7bfff700 1 -- 172.21.15.150:0/3651664306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c10a730 con 0x7f6b7c107dd0 2024-05-10T03:56:48.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.886+0000 7f6b7bfff700 1 --2- 172.21.15.150:0/3651664306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 0x7f6b7c10a1f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0098a0 tx=0x7f6b6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a03f3bdeaeb8ba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.886+0000 7f6b7b7fe700 1 -- 172.21.15.150:0/3651664306 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b6c004160 con 0x7f6b7c107dd0 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.887+0000 7f6b7b7fe700 1 -- 172.21.15.150:0/3651664306 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6b6c0042c0 con 0x7f6b7c107dd0 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.887+0000 7f6b7b7fe700 1 -- 172.21.15.150:0/3651664306 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b6c011a00 con 0x7f6b7c107dd0 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.888+0000 7f6b82416700 1 -- 172.21.15.150:0/3651664306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 msgr2=0x7f6b7c10a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.888+0000 7f6b82416700 1 --2- 172.21.15.150:0/3651664306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 0x7f6b7c10a1f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0098a0 tx=0x7f6b6c009bb0 comp rx=0 tx=0).stop 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.888+0000 7f6b82416700 1 -- 172.21.15.150:0/3651664306 shutdown_connections 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.888+0000 7f6b82416700 1 --2- 172.21.15.150:0/3651664306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 0x7f6b7c10a1f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.888+0000 7f6b82416700 1 -- 172.21.15.150:0/3651664306 >> 172.21.15.150:0/3651664306 conn(0x7f6b7c101850 msgr2=0x7f6b7c103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.888+0000 7f6b82416700 1 -- 172.21.15.150:0/3651664306 shutdown_connections 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.888+0000 7f6b82416700 1 -- 172.21.15.150:0/3651664306 wait complete. 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.889+0000 7f6b82416700 1 Processor -- start 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.889+0000 7f6b82416700 1 -- start start 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.889+0000 7f6b82416700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 0x7f6b7c19b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.889+0000 7f6b82416700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c19ba40 con 0x7f6b7c107dd0 2024-05-10T03:56:48.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.889+0000 7f6b7bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 0x7f6b7c19b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.889+0000 7f6b7bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 0x7f6b7c19b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34374/0 (socket says 172.21.15.150:34374) 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.889+0000 7f6b7bfff700 1 -- 172.21.15.150:0/2292745311 learned_addr learned my addr 172.21.15.150:0/2292745311 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.889+0000 7f6b7bfff700 1 -- 172.21.15.150:0/2292745311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c009550 con 0x7f6b7c107dd0 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.889+0000 7f6b7bfff700 1 --2- 172.21.15.150:0/2292745311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 0x7f6b7c19b500 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009130 tx=0x7f6b6c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.889+0000 7f6b79ffb700 1 -- 172.21.15.150:0/2292745311 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b6c0038b0 con 0x7f6b7c107dd0 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.890+0000 7f6b79ffb700 1 -- 172.21.15.150:0/2292745311 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6b6c003a10 con 0x7f6b7c107dd0 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.890+0000 7f6b82416700 1 -- 172.21.15.150:0/2292745311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7c19bc40 con 0x7f6b7c107dd0 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.890+0000 7f6b79ffb700 1 -- 172.21.15.150:0/2292745311 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b6c019550 con 0x7f6b7c107dd0 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.890+0000 7f6b82416700 1 -- 172.21.15.150:0/2292745311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7c19c0e0 con 0x7f6b7c107dd0 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.890+0000 7f6b79ffb700 1 -- 172.21.15.150:0/2292745311 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f6b6c019af0 con 0x7f6b7c107dd0 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.890+0000 7f6b79ffb700 1 -- 172.21.15.150:0/2292745311 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6b6c0243f0 con 0x7f6b7c107dd0 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.891+0000 7f6b82416700 1 -- 172.21.15.150:0/2292745311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7c10b4d0 con 0x7f6b7c107dd0 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.893+0000 7f6b79ffb700 1 -- 172.21.15.150:0/2292745311 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f6b6c02de30 con 0x7f6b7c107dd0 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.965+0000 7f6b82416700 1 -- 172.21.15.150:0/2292745311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6b7c10b6f0 con 0x7f6b7c107dd0 2024-05-10T03:56:48.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.965+0000 7f6b79ffb700 1 -- 172.21.15.150:0/2292745311 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f6b6c02d440 con 0x7f6b7c107dd0 2024-05-10T03:56:48.352 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.966+0000 7f6b82416700 1 -- 172.21.15.150:0/2292745311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 msgr2=0x7f6b7c19b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:48.352 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.966+0000 7f6b82416700 1 --2- 172.21.15.150:0/2292745311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 0x7f6b7c19b500 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009130 tx=0x7f6b6c011ad0 comp rx=0 tx=0).stop 2024-05-10T03:56:48.352 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.966+0000 7f6b82416700 1 -- 172.21.15.150:0/2292745311 shutdown_connections 2024-05-10T03:56:48.352 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.966+0000 7f6b82416700 1 --2- 172.21.15.150:0/2292745311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b7c107dd0 0x7f6b7c19b500 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:48.352 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.966+0000 7f6b82416700 1 -- 172.21.15.150:0/2292745311 >> 172.21.15.150:0/2292745311 conn(0x7f6b7c101850 msgr2=0x7f6b7c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:48.352 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.966+0000 7f6b82416700 1 -- 172.21.15.150:0/2292745311 shutdown_connections 2024-05-10T03:56:48.352 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:47.966+0000 7f6b82416700 1 -- 172.21.15.150:0/2292745311 wait complete. 2024-05-10T03:56:48.352 INFO:teuthology.orchestra.run.smithi150.stdout:mgr not available, waiting (2/15)... 2024-05-10T03:56:48.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:48 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:48.684+0000 7f6028722200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T03:56:48.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:48 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:48.780+0000 7f6028722200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T03:56:48.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:48 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:48.927+0000 7f6028722200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T03:56:49.931 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:49 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:49.555+0000 7f6028722200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T03:56:49.931 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:49 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:49.735+0000 7f6028722200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T03:56:50.714 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:49 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:49.927+0000 7f6028722200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T03:56:51.088 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T03:56:51.088 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T03:56:51.088 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsid": "325b4c82-0e81-11ef-bc98-c7b262605968", 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 0 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-10T03:56:51.089 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T03:56:51.090 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:51.090 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-10T03:56:51.090 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T03:56:51.090 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:51.090 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T03:56:51.090 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:51.090 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T03:56:51.090 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T03:56:51.090 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T03:56:51.099 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T03:56:51.099 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T03:56:51.099 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T03:56:51.099 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:51.099 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T03:56:51.099 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T03:56:51.099 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T03:56:51.099 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T03:56:51.099 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T03:56:51.099 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T03:56:51.099 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T03:56:51.099 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T03:56:51.099 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T03:56:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T03:56:39.690289+0000", 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.668+0000 7f9672f79700 1 Processor -- start 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.668+0000 7f9672f79700 1 -- start start 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.668+0000 7f9672f79700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 0x7f966c105a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.668+0000 7f9672f79700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c105fe0 con 0x7f966c105600 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.668+0000 7f9670d15700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 0x7f966c105a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.668+0000 7f9670d15700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 0x7f966c105a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47212/0 (socket says 172.21.15.150:47212) 2024-05-10T03:56:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.668+0000 7f9670d15700 1 -- 172.21.15.150:0/4247747996 learned_addr learned my addr 172.21.15.150:0/4247747996 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:51.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.669+0000 7f9670d15700 1 -- 172.21.15.150:0/4247747996 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966c106120 con 0x7f966c105600 2024-05-10T03:56:51.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.669+0000 7f9670d15700 1 --2- 172.21.15.150:0/4247747996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 0x7f966c105a10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96600098a0 tx=0x7f9660009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1a46c79dd13aa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:51.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.669+0000 7f966b7fe700 1 -- 172.21.15.150:0/4247747996 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9660004160 con 0x7f966c105600 2024-05-10T03:56:51.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.669+0000 7f966b7fe700 1 -- 172.21.15.150:0/4247747996 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f96600042c0 con 0x7f966c105600 2024-05-10T03:56:51.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.669+0000 7f966b7fe700 1 -- 172.21.15.150:0/4247747996 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9660011a00 con 0x7f966c105600 2024-05-10T03:56:51.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.670+0000 7f9672f79700 1 -- 172.21.15.150:0/4247747996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 msgr2=0x7f966c105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:51.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.670+0000 7f9672f79700 1 --2- 172.21.15.150:0/4247747996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 0x7f966c105a10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96600098a0 tx=0x7f9660009bb0 comp rx=0 tx=0).stop 2024-05-10T03:56:51.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.670+0000 7f9672f79700 1 -- 172.21.15.150:0/4247747996 shutdown_connections 2024-05-10T03:56:51.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.670+0000 7f9672f79700 1 --2- 172.21.15.150:0/4247747996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 0x7f966c105a10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.670+0000 7f9672f79700 1 -- 172.21.15.150:0/4247747996 >> 172.21.15.150:0/4247747996 conn(0x7f966c1010b0 msgr2=0x7f966c103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.670+0000 7f9672f79700 1 -- 172.21.15.150:0/4247747996 shutdown_connections 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.670+0000 7f9672f79700 1 -- 172.21.15.150:0/4247747996 wait complete. 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.670+0000 7f9672f79700 1 Processor -- start 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.671+0000 7f9672f79700 1 -- start start 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.671+0000 7f9672f79700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 0x7f966c19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.671+0000 7f9672f79700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c19ba50 con 0x7f966c105600 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.671+0000 7f9670d15700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 0x7f966c19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.671+0000 7f9670d15700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 0x7f966c19b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47224/0 (socket says 172.21.15.150:47224) 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.671+0000 7f9670d15700 1 -- 172.21.15.150:0/67004842 learned_addr learned my addr 172.21.15.150:0/67004842 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.671+0000 7f9670d15700 1 -- 172.21.15.150:0/67004842 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9660009550 con 0x7f966c105600 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.671+0000 7f9670d15700 1 --2- 172.21.15.150:0/67004842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 0x7f966c19b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9660009130 tx=0x7f9660011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.671+0000 7f9669ffb700 1 -- 172.21.15.150:0/67004842 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96600038b0 con 0x7f966c105600 2024-05-10T03:56:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.671+0000 7f9669ffb700 1 -- 172.21.15.150:0/67004842 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9660003a10 con 0x7f966c105600 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.671+0000 7f9672f79700 1 -- 172.21.15.150:0/67004842 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966c19bc50 con 0x7f966c105600 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.671+0000 7f9669ffb700 1 -- 172.21.15.150:0/67004842 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9660019550 con 0x7f966c105600 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.672+0000 7f9672f79700 1 -- 172.21.15.150:0/67004842 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966c19c0f0 con 0x7f966c105600 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.672+0000 7f9669ffb700 1 -- 172.21.15.150:0/67004842 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9660019ac0 con 0x7f966c105600 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.672+0000 7f9669ffb700 1 -- 172.21.15.150:0/67004842 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f966002dd20 con 0x7f966c105600 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.673+0000 7f9672f79700 1 -- 172.21.15.150:0/67004842 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f966c062840 con 0x7f966c105600 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.675+0000 7f9669ffb700 1 -- 172.21.15.150:0/67004842 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f9660003b80 con 0x7f966c105600 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.747+0000 7f9672f79700 1 -- 172.21.15.150:0/67004842 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f966c19eb10 con 0x7f966c105600 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.748+0000 7f9669ffb700 1 -- 172.21.15.150:0/67004842 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f966002d440 con 0x7f966c105600 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.749+0000 7f9672f79700 1 -- 172.21.15.150:0/67004842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 msgr2=0x7f966c19b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.749+0000 7f9672f79700 1 --2- 172.21.15.150:0/67004842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 0x7f966c19b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9660009130 tx=0x7f9660011ad0 comp rx=0 tx=0).stop 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.749+0000 7f9672f79700 1 -- 172.21.15.150:0/67004842 shutdown_connections 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.749+0000 7f9672f79700 1 --2- 172.21.15.150:0/67004842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966c105600 0x7f966c19b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:51.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.749+0000 7f9672f79700 1 -- 172.21.15.150:0/67004842 >> 172.21.15.150:0/67004842 conn(0x7f966c1010b0 msgr2=0x7f966c102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:51.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.749+0000 7f9672f79700 1 -- 172.21.15.150:0/67004842 shutdown_connections 2024-05-10T03:56:51.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:50.749+0000 7f9672f79700 1 -- 172.21.15.150:0/67004842 wait complete. 2024-05-10T03:56:51.105 INFO:teuthology.orchestra.run.smithi150.stdout:mgr not available, waiting (3/15)... 2024-05-10T03:56:51.187 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:50 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/67004842' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T03:56:51.188 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:50 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:50.892+0000 7f6028722200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T03:56:51.464 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:51 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:51.184+0000 7f6028722200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:56:51.464 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:51 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:51.273+0000 7f6028722200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T03:56:51.803 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:56:51 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:56:51.463+0000 7f6028722200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T03:56:52.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:51 smithi150 ceph-mon[42477]: Activating manager daemon x 2024-05-10T03:56:52.217 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:51 smithi150 ceph-mon[42477]: mgrmap e2: x(active, starting, since 0.00348555s) 2024-05-10T03:56:52.217 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:51 smithi150 ceph-mon[42477]: from='mgr.14100 172.21.15.150:0/4278133661' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T03:56:52.217 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:51 smithi150 ceph-mon[42477]: from='mgr.14100 172.21.15.150:0/4278133661' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T03:56:52.217 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:51 smithi150 ceph-mon[42477]: from='mgr.14100 172.21.15.150:0/4278133661' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T03:56:52.218 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:51 smithi150 ceph-mon[42477]: from='mgr.14100 172.21.15.150:0/4278133661' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:56:52.218 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:51 smithi150 ceph-mon[42477]: from='mgr.14100 172.21.15.150:0/4278133661' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T03:56:52.218 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:51 smithi150 ceph-mon[42477]: Manager daemon x is now available 2024-05-10T03:56:52.218 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:51 smithi150 ceph-mon[42477]: from='mgr.14100 172.21.15.150:0/4278133661' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T03:56:52.218 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:51 smithi150 ceph-mon[42477]: from='mgr.14100 172.21.15.150:0/4278133661' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T03:56:52.218 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:51 smithi150 ceph-mon[42477]: from='mgr.14100 172.21.15.150:0/4278133661' entity='mgr.x' 2024-05-10T03:56:52.218 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:51 smithi150 ceph-mon[42477]: from='mgr.14100 172.21.15.150:0/4278133661' entity='mgr.x' 2024-05-10T03:56:52.218 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:51 smithi150 ceph-mon[42477]: from='mgr.14100 172.21.15.150:0/4278133661' entity='mgr.x' 2024-05-10T03:56:53.763 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:53 smithi150 ceph-mon[42477]: mgrmap e3: x(active, since 1.00765s) 2024-05-10T03:56:54.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:54 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1729230524' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T03:56:55.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:55 smithi150 ceph-mon[42477]: mgrmap e4: x(active, since 3s) 2024-05-10T03:56:56.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T03:56:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T03:56:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsid": "325b4c82-0e81-11ef-bc98-c7b262605968", 2024-05-10T03:56:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T03:56:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T03:56:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T03:56:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T03:56:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T03:56:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T03:56:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 0 2024-05-10T03:56:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T03:56:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T03:56:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T03:56:56.161 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T03:56:56.161 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-10T03:56:56.161 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T03:56:56.161 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:56.161 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-10T03:56:56.161 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T03:56:56.161 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:56.161 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T03:56:56.161 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:56.161 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T03:56:56.161 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T03:56:56.161 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T03:56:56.171 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T03:56:56.173 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:56.173 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T03:56:56.173 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T03:56:56.173 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:56.173 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T03:56:56.173 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T03:56:56.173 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T03:56:56.173 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T03:56:56.173 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T03:56:56.173 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T03:56:56.174 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T03:56:56.174 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T03:56:56.174 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T03:56:56.174 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:56.174 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T03:56:56.174 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T03:56:56.174 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T03:56:39.690289+0000", 2024-05-10T03:56:56.174 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T03:56:56.174 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T03:56:56.174 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T03:56:56.174 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T03:56:56.175 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.185+0000 7f37b5c65700 1 Processor -- start 2024-05-10T03:56:56.175 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.185+0000 7f37b5c65700 1 -- start start 2024-05-10T03:56:56.175 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.186+0000 7f37b5c65700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 0x7f37b010a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:56.175 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.186+0000 7f37b5c65700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b007ade0 con 0x7f37b0107e30 2024-05-10T03:56:56.175 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.186+0000 7f37af7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 0x7f37b010a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:56.175 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.186+0000 7f37af7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 0x7f37b010a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47280/0 (socket says 172.21.15.150:47280) 2024-05-10T03:56:56.175 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.186+0000 7f37af7fe700 1 -- 172.21.15.150:0/976801265 learned_addr learned my addr 172.21.15.150:0/976801265 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:56.175 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.186+0000 7f37af7fe700 1 -- 172.21.15.150:0/976801265 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b010a790 con 0x7f37b0107e30 2024-05-10T03:56:56.175 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.186+0000 7f37af7fe700 1 --2- 172.21.15.150:0/976801265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 0x7f37b010a250 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f37a00098a0 tx=0x7f37a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6403376e911083 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:56.175 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.186+0000 7f37ae7fc700 1 -- 172.21.15.150:0/976801265 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37a0004160 con 0x7f37b0107e30 2024-05-10T03:56:56.175 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.187+0000 7f37ae7fc700 1 -- 172.21.15.150:0/976801265 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f37a00042c0 con 0x7f37b0107e30 2024-05-10T03:56:56.175 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.187+0000 7f37ae7fc700 1 -- 172.21.15.150:0/976801265 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37a0011a00 con 0x7f37b0107e30 2024-05-10T03:56:56.176 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.187+0000 7f37b5c65700 1 -- 172.21.15.150:0/976801265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 msgr2=0x7f37b010a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:56.176 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.187+0000 7f37b5c65700 1 --2- 172.21.15.150:0/976801265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 0x7f37b010a250 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f37a00098a0 tx=0x7f37a0009bb0 comp rx=0 tx=0).stop 2024-05-10T03:56:56.176 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.187+0000 7f37b5c65700 1 -- 172.21.15.150:0/976801265 shutdown_connections 2024-05-10T03:56:56.176 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.187+0000 7f37b5c65700 1 --2- 172.21.15.150:0/976801265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 0x7f37b010a250 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:56.176 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.187+0000 7f37b5c65700 1 -- 172.21.15.150:0/976801265 >> 172.21.15.150:0/976801265 conn(0x7f37b0101910 msgr2=0x7f37b0103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:56.176 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.187+0000 7f37b5c65700 1 -- 172.21.15.150:0/976801265 shutdown_connections 2024-05-10T03:56:56.176 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.187+0000 7f37b5c65700 1 -- 172.21.15.150:0/976801265 wait complete. 2024-05-10T03:56:56.176 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.188+0000 7f37b5c65700 1 Processor -- start 2024-05-10T03:56:56.176 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.188+0000 7f37b5c65700 1 -- start start 2024-05-10T03:56:56.176 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.188+0000 7f37b5c65700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 0x7f37b019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:56.176 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.188+0000 7f37b5c65700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b019ba60 con 0x7f37b0107e30 2024-05-10T03:56:56.177 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.189+0000 7f37af7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 0x7f37b019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:56.177 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.189+0000 7f37af7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 0x7f37b019b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47284/0 (socket says 172.21.15.150:47284) 2024-05-10T03:56:56.177 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.189+0000 7f37af7fe700 1 -- 172.21.15.150:0/1729230524 learned_addr learned my addr 172.21.15.150:0/1729230524 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:56.177 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.189+0000 7f37af7fe700 1 -- 172.21.15.150:0/1729230524 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a0009550 con 0x7f37b0107e30 2024-05-10T03:56:56.177 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.189+0000 7f37af7fe700 1 --2- 172.21.15.150:0/1729230524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 0x7f37b019b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f37a0011710 tx=0x7f37a0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:56.177 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.189+0000 7f37acff9700 1 -- 172.21.15.150:0/1729230524 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37a0003af0 con 0x7f37b0107e30 2024-05-10T03:56:56.177 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.189+0000 7f37acff9700 1 -- 172.21.15.150:0/1729230524 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f37a0003c50 con 0x7f37b0107e30 2024-05-10T03:56:56.177 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.189+0000 7f37b5c65700 1 -- 172.21.15.150:0/1729230524 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b019bc60 con 0x7f37b0107e30 2024-05-10T03:56:56.178 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.189+0000 7f37acff9700 1 -- 172.21.15.150:0/1729230524 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37a0024780 con 0x7f37b0107e30 2024-05-10T03:56:56.178 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.189+0000 7f37b5c65700 1 -- 172.21.15.150:0/1729230524 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b019c100 con 0x7f37b0107e30 2024-05-10T03:56:56.178 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.190+0000 7f37b5c65700 1 -- 172.21.15.150:0/1729230524 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b010b530 con 0x7f37b0107e30 2024-05-10T03:56:56.178 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.191+0000 7f37acff9700 1 -- 172.21.15.150:0/1729230524 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 3) v1 ==== 45424+0+0 (secure 0 0 0) 0x7f37a001ab40 con 0x7f37b0107e30 2024-05-10T03:56:56.178 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.191+0000 7f37acff9700 1 --2- 172.21.15.150:0/1729230524 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f379c0385d0 0x7f379c03aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:56.178 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.191+0000 7f37acff9700 1 -- 172.21.15.150:0/1729230524 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f37a004c620 con 0x7f37b0107e30 2024-05-10T03:56:56.178 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.191+0000 7f37aeffd700 1 --2- 172.21.15.150:0/1729230524 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f379c0385d0 0x7f379c03aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:56.178 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.191+0000 7f37aeffd700 1 --2- 172.21.15.150:0/1729230524 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f379c0385d0 0x7f379c03aae0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3798006fd0 tx=0x7f3798006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:56.179 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.196+0000 7f37acff9700 1 -- 172.21.15.150:0/1729230524 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f37a002cb50 con 0x7f37b0107e30 2024-05-10T03:56:56.179 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.469+0000 7f37b5c65700 1 -- 172.21.15.150:0/1729230524 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f37b010b740 con 0x7f37b0107e30 2024-05-10T03:56:56.179 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.470+0000 7f37acff9700 1 -- 172.21.15.150:0/1729230524 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f37a002cb50 con 0x7f37b0107e30 2024-05-10T03:56:56.179 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.472+0000 7f37b5c65700 1 -- 172.21.15.150:0/1729230524 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f379c0385d0 msgr2=0x7f379c03aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:56.179 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.472+0000 7f37b5c65700 1 --2- 172.21.15.150:0/1729230524 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f379c0385d0 0x7f379c03aae0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3798006fd0 tx=0x7f3798006e40 comp rx=0 tx=0).stop 2024-05-10T03:56:56.179 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.472+0000 7f37b5c65700 1 -- 172.21.15.150:0/1729230524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 msgr2=0x7f37b019b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:56.179 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.472+0000 7f37b5c65700 1 --2- 172.21.15.150:0/1729230524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 0x7f37b019b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f37a0011710 tx=0x7f37a0011ad0 comp rx=0 tx=0).stop 2024-05-10T03:56:56.179 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.472+0000 7f37b5c65700 1 -- 172.21.15.150:0/1729230524 shutdown_connections 2024-05-10T03:56:56.180 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.472+0000 7f37b5c65700 1 --2- 172.21.15.150:0/1729230524 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f379c0385d0 0x7f379c03aae0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:56.180 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.472+0000 7f37b5c65700 1 --2- 172.21.15.150:0/1729230524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37b0107e30 0x7f37b019b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:56.180 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.472+0000 7f37b5c65700 1 -- 172.21.15.150:0/1729230524 >> 172.21.15.150:0/1729230524 conn(0x7f37b0101910 msgr2=0x7f37b01033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:56.180 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.472+0000 7f37b5c65700 1 -- 172.21.15.150:0/1729230524 shutdown_connections 2024-05-10T03:56:56.180 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:54.472+0000 7f37b5c65700 1 -- 172.21.15.150:0/1729230524 wait complete. 2024-05-10T03:56:56.180 INFO:teuthology.orchestra.run.smithi150.stdout:mgr is available 2024-05-10T03:56:58.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:56:57 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3127283515' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-10T03:56:59.933 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T03:56:59.933 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout [global] 2024-05-10T03:56:59.933 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout fsid = 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:56:59.934 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-10T03:56:59.934 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-10T03:56:59.934 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-10T03:56:59.934 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-10T03:56:59.934 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T03:56:59.934 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-10T03:56:59.934 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-10T03:56:59.934 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T03:56:59.934 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout [osd] 2024-05-10T03:56:59.934 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-10T03:56:59.934 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.573+0000 7f8d9d4d8700 1 Processor -- start 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.573+0000 7f8d9d4d8700 1 -- start start 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.573+0000 7f8d9d4d8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 0x7f8d98107be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.573+0000 7f8d9d4d8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d981081d0 con 0x7f8d981077d0 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.573+0000 7f8d96ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 0x7f8d98107be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.573+0000 7f8d96ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 0x7f8d98107be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47288/0 (socket says 172.21.15.150:47288) 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.573+0000 7f8d96ffd700 1 -- 172.21.15.150:0/891543929 learned_addr learned my addr 172.21.15.150:0/891543929 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.574+0000 7f8d96ffd700 1 -- 172.21.15.150:0/891543929 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d98106450 con 0x7f8d981077d0 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.574+0000 7f8d96ffd700 1 --2- 172.21.15.150:0/891543929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 0x7f8d98107be0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8d880098a0 tx=0x7f8d88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9364c7cbad61460e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.574+0000 7f8d95ffb700 1 -- 172.21.15.150:0/891543929 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d88004160 con 0x7f8d981077d0 2024-05-10T03:56:59.935 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.574+0000 7f8d95ffb700 1 -- 172.21.15.150:0/891543929 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8d880042c0 con 0x7f8d981077d0 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.574+0000 7f8d95ffb700 1 -- 172.21.15.150:0/891543929 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d88011a00 con 0x7f8d981077d0 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.575+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/891543929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 msgr2=0x7f8d98107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.575+0000 7f8d9d4d8700 1 --2- 172.21.15.150:0/891543929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 0x7f8d98107be0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8d880098a0 tx=0x7f8d88009bb0 comp rx=0 tx=0).stop 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.575+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/891543929 shutdown_connections 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.575+0000 7f8d9d4d8700 1 --2- 172.21.15.150:0/891543929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 0x7f8d98107be0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.575+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/891543929 >> 172.21.15.150:0/891543929 conn(0x7f8d98100ff0 msgr2=0x7f8d98103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.575+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/891543929 shutdown_connections 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.575+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/891543929 wait complete. 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.575+0000 7f8d9d4d8700 1 Processor -- start 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.576+0000 7f8d9d4d8700 1 -- start start 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.576+0000 7f8d9d4d8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 0x7f8d98103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.576+0000 7f8d9d4d8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d98103780 con 0x7f8d981077d0 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.576+0000 7f8d96ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 0x7f8d98103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:59.936 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.576+0000 7f8d96ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 0x7f8d98103240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47296/0 (socket says 172.21.15.150:47296) 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.576+0000 7f8d96ffd700 1 -- 172.21.15.150:0/3127283515 learned_addr learned my addr 172.21.15.150:0/3127283515 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.576+0000 7f8d96ffd700 1 -- 172.21.15.150:0/3127283515 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d88009550 con 0x7f8d981077d0 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.576+0000 7f8d96ffd700 1 --2- 172.21.15.150:0/3127283515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 0x7f8d98103240 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009130 tx=0x7f8d88011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.577+0000 7f8d7ffff700 1 -- 172.21.15.150:0/3127283515 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d880038b0 con 0x7f8d981077d0 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.577+0000 7f8d7ffff700 1 -- 172.21.15.150:0/3127283515 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8d88003a10 con 0x7f8d981077d0 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.577+0000 7f8d7ffff700 1 -- 172.21.15.150:0/3127283515 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d88019550 con 0x7f8d981077d0 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.577+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/3127283515 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d981019d0 con 0x7f8d981077d0 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.577+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/3127283515 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d98101e70 con 0x7f8d981077d0 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.578+0000 7f8d7ffff700 1 -- 172.21.15.150:0/3127283515 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f8d88019810 con 0x7f8d981077d0 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.578+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/3127283515 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d98102880 con 0x7f8d981077d0 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.578+0000 7f8d7ffff700 1 --2- 172.21.15.150:0/3127283515 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f8d80038660 0x7f8d8003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.578+0000 7f8d7ffff700 1 -- 172.21.15.150:0/3127283515 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8d8804c420 con 0x7f8d981077d0 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.578+0000 7f8d967fc700 1 --2- 172.21.15.150:0/3127283515 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f8d80038660 0x7f8d8003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.578+0000 7f8d967fc700 1 --2- 172.21.15.150:0/3127283515 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f8d80038660 0x7f8d8003ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c006fd0 tx=0x7f8d8c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:56:59.937 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.583+0000 7f8d7ffff700 1 -- 172.21.15.150:0/3127283515 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8d880189a0 con 0x7f8d981077d0 2024-05-10T03:56:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.778+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/3127283515 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8d98062840 con 0x7f8d981077d0 2024-05-10T03:56:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.779+0000 7f8d7ffff700 1 -- 172.21.15.150:0/3127283515 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f8d88018350 con 0x7f8d981077d0 2024-05-10T03:56:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.781+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/3127283515 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f8d80038660 msgr2=0x7f8d8003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.781+0000 7f8d9d4d8700 1 --2- 172.21.15.150:0/3127283515 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f8d80038660 0x7f8d8003ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c006fd0 tx=0x7f8d8c006e40 comp rx=0 tx=0).stop 2024-05-10T03:56:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.781+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/3127283515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 msgr2=0x7f8d98103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:56:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.781+0000 7f8d9d4d8700 1 --2- 172.21.15.150:0/3127283515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 0x7f8d98103240 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009130 tx=0x7f8d88011ad0 comp rx=0 tx=0).stop 2024-05-10T03:56:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.781+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/3127283515 shutdown_connections 2024-05-10T03:56:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.781+0000 7f8d9d4d8700 1 --2- 172.21.15.150:0/3127283515 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f8d80038660 0x7f8d8003ab70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.781+0000 7f8d9d4d8700 1 --2- 172.21.15.150:0/3127283515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d981077d0 0x7f8d98103240 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:56:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.781+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/3127283515 >> 172.21.15.150:0/3127283515 conn(0x7f8d98100ff0 msgr2=0x7f8d98191cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:56:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.782+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/3127283515 shutdown_connections 2024-05-10T03:56:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:56:57.782+0000 7f8d9d4d8700 1 -- 172.21.15.150:0/3127283515 wait complete. 2024-05-10T03:56:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:Enabling cephadm module... 2024-05-10T03:57:02.153 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:01 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3618077242' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-10T03:57:02.153 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:01 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: ignoring --setuser ceph since I am not root 2024-05-10T03:57:02.153 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:01 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: ignoring --setgroup ceph since I am not root 2024-05-10T03:57:02.230 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.544+0000 7f6b5e62d700 1 Processor -- start 2024-05-10T03:57:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.544+0000 7f6b5e62d700 1 -- start start 2024-05-10T03:57:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.544+0000 7f6b5e62d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 0x7f6b58106060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.544+0000 7f6b5e62d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5807ada0 con 0x7f6b58105c50 2024-05-10T03:57:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.544+0000 7f6b57fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 0x7f6b58106060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.545+0000 7f6b57fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 0x7f6b58106060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45760/0 (socket says 172.21.15.150:45760) 2024-05-10T03:57:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.545+0000 7f6b57fff700 1 -- 172.21.15.150:0/370029725 learned_addr learned my addr 172.21.15.150:0/370029725 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.545+0000 7f6b57fff700 1 -- 172.21.15.150:0/370029725 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5807aee0 con 0x7f6b58105c50 2024-05-10T03:57:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.545+0000 7f6b57fff700 1 --2- 172.21.15.150:0/370029725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 0x7f6b58106060 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0098a0 tx=0x7f6b4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c128266ce147f77b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.545+0000 7f6b56ffd700 1 -- 172.21.15.150:0/370029725 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b4c004160 con 0x7f6b58105c50 2024-05-10T03:57:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.545+0000 7f6b56ffd700 1 -- 172.21.15.150:0/370029725 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6b4c0042c0 con 0x7f6b58105c50 2024-05-10T03:57:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.545+0000 7f6b56ffd700 1 -- 172.21.15.150:0/370029725 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b4c011a00 con 0x7f6b58105c50 2024-05-10T03:57:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.546+0000 7f6b5e62d700 1 -- 172.21.15.150:0/370029725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 msgr2=0x7f6b58106060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.546+0000 7f6b5e62d700 1 --2- 172.21.15.150:0/370029725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 0x7f6b58106060 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0098a0 tx=0x7f6b4c009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.546+0000 7f6b5e62d700 1 -- 172.21.15.150:0/370029725 shutdown_connections 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.546+0000 7f6b5e62d700 1 --2- 172.21.15.150:0/370029725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 0x7f6b58106060 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.546+0000 7f6b5e62d700 1 -- 172.21.15.150:0/370029725 >> 172.21.15.150:0/370029725 conn(0x7f6b58101820 msgr2=0x7f6b58103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.546+0000 7f6b5e62d700 1 -- 172.21.15.150:0/370029725 shutdown_connections 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.546+0000 7f6b5e62d700 1 -- 172.21.15.150:0/370029725 wait complete. 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.547+0000 7f6b5e62d700 1 Processor -- start 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.547+0000 7f6b5e62d700 1 -- start start 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.547+0000 7f6b5e62d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 0x7f6b5819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.547+0000 7f6b5e62d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5819dbf0 con 0x7f6b58105c50 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.547+0000 7f6b57fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 0x7f6b5819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.547+0000 7f6b57fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 0x7f6b5819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45766/0 (socket says 172.21.15.150:45766) 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.547+0000 7f6b57fff700 1 -- 172.21.15.150:0/3618077242 learned_addr learned my addr 172.21.15.150:0/3618077242 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.548+0000 7f6b57fff700 1 -- 172.21.15.150:0/3618077242 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c009550 con 0x7f6b58105c50 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.548+0000 7f6b57fff700 1 --2- 172.21.15.150:0/3618077242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 0x7f6b5819d6b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c011710 tx=0x7f6b4c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:02.232 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.548+0000 7f6b557fa700 1 -- 172.21.15.150:0/3618077242 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b4c003aa0 con 0x7f6b58105c50 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.548+0000 7f6b557fa700 1 -- 172.21.15.150:0/3618077242 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6b4c003c00 con 0x7f6b58105c50 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.548+0000 7f6b5e62d700 1 -- 172.21.15.150:0/3618077242 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5819ddf0 con 0x7f6b58105c50 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.548+0000 7f6b557fa700 1 -- 172.21.15.150:0/3618077242 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b4c024800 con 0x7f6b58105c50 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.548+0000 7f6b5e62d700 1 -- 172.21.15.150:0/3618077242 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5819e290 con 0x7f6b58105c50 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.549+0000 7f6b557fa700 1 -- 172.21.15.150:0/3618077242 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f6b4c024960 con 0x7f6b58105c50 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.549+0000 7f6b5e62d700 1 -- 172.21.15.150:0/3618077242 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b58196d80 con 0x7f6b58105c50 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.549+0000 7f6b557fa700 1 --2- 172.21.15.150:0/3618077242 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f6b40038700 0x7f6b4003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.549+0000 7f6b557fa700 1 -- 172.21.15.150:0/3618077242 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6b4c04c0f0 con 0x7f6b58105c50 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.549+0000 7f6b577fe700 1 --2- 172.21.15.150:0/3618077242 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f6b40038700 0x7f6b4003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.550+0000 7f6b577fe700 1 --2- 172.21.15.150:0/3618077242 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f6b40038700 0x7f6b4003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6b48006fd0 tx=0x7f6b48006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.554+0000 7f6b557fa700 1 -- 172.21.15.150:0/3618077242 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6b4c07fd60 con 0x7f6b58105c50 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.796+0000 7f6b5e62d700 1 -- 172.21.15.150:0/3618077242 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f6b5807aee0 con 0x7f6b58105c50 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.849+0000 7f6b557fa700 1 -- 172.21.15.150:0/3618077242 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f6b4c024cc0 con 0x7f6b58105c50 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.851+0000 7f6b557fa700 1 -- 172.21.15.150:0/3618077242 <== mon.0 v2:172.21.15.150:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f6b4c00bc70 con 0x7f6b58105c50 2024-05-10T03:57:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.853+0000 7f6b5e62d700 1 -- 172.21.15.150:0/3618077242 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f6b40038700 msgr2=0x7f6b4003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:02.234 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.853+0000 7f6b5e62d700 1 --2- 172.21.15.150:0/3618077242 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f6b40038700 0x7f6b4003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6b48006fd0 tx=0x7f6b48006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:02.234 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.853+0000 7f6b5e62d700 1 -- 172.21.15.150:0/3618077242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 msgr2=0x7f6b5819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:02.234 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.853+0000 7f6b5e62d700 1 --2- 172.21.15.150:0/3618077242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 0x7f6b5819d6b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c011710 tx=0x7f6b4c011ad0 comp rx=0 tx=0).stop 2024-05-10T03:57:02.234 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.854+0000 7f6b5e62d700 1 -- 172.21.15.150:0/3618077242 shutdown_connections 2024-05-10T03:57:02.234 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.854+0000 7f6b5e62d700 1 --2- 172.21.15.150:0/3618077242 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7f6b40038700 0x7f6b4003ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:02.234 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.854+0000 7f6b5e62d700 1 --2- 172.21.15.150:0/3618077242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b58105c50 0x7f6b5819d6b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:02.234 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.854+0000 7f6b5e62d700 1 -- 172.21.15.150:0/3618077242 >> 172.21.15.150:0/3618077242 conn(0x7f6b58101820 msgr2=0x7f6b581918e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:02.234 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.854+0000 7f6b5e62d700 1 -- 172.21.15.150:0/3618077242 shutdown_connections 2024-05-10T03:57:02.234 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:01.854+0000 7f6b5e62d700 1 -- 172.21.15.150:0/3618077242 wait complete. 2024-05-10T03:57:02.432 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:02 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:02.330+0000 7f1e67089200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T03:57:02.714 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:02 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:02.429+0000 7f1e67089200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T03:57:02.714 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:02 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:02.524+0000 7f1e67089200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T03:57:02.714 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:02 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:02.615+0000 7f1e67089200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T03:57:03.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T03:57:03.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-10T03:57:03.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T03:57:03.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-10T03:57:03.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-10T03:57:03.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T03:57:03.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.551+0000 7fca58f7a700 1 Processor -- start 2024-05-10T03:57:03.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.551+0000 7fca58f7a700 1 -- start start 2024-05-10T03:57:03.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.551+0000 7fca58f7a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 0x7fca54106600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:03.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.551+0000 7fca58f7a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca54106bf0 con 0x7fca541061f0 2024-05-10T03:57:03.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.551+0000 7fca5259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 0x7fca54106600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:03.135 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.551+0000 7fca5259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 0x7fca54106600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45788/0 (socket says 172.21.15.150:45788) 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.551+0000 7fca5259c700 1 -- 172.21.15.150:0/3350578949 learned_addr learned my addr 172.21.15.150:0/3350578949 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.552+0000 7fca5259c700 1 -- 172.21.15.150:0/3350578949 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca541073c0 con 0x7fca541061f0 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.552+0000 7fca5259c700 1 --2- 172.21.15.150:0/3350578949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 0x7fca54106600 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fca440098a0 tx=0x7fca44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b75af0fe89a8aea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.552+0000 7fca5159a700 1 -- 172.21.15.150:0/3350578949 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca44004160 con 0x7fca541061f0 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.552+0000 7fca5159a700 1 -- 172.21.15.150:0/3350578949 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca440042c0 con 0x7fca541061f0 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.552+0000 7fca5159a700 1 -- 172.21.15.150:0/3350578949 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca44011a00 con 0x7fca541061f0 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.553+0000 7fca58f7a700 1 -- 172.21.15.150:0/3350578949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 msgr2=0x7fca54106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.553+0000 7fca58f7a700 1 --2- 172.21.15.150:0/3350578949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 0x7fca54106600 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fca440098a0 tx=0x7fca44009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.553+0000 7fca58f7a700 1 -- 172.21.15.150:0/3350578949 shutdown_connections 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.553+0000 7fca58f7a700 1 --2- 172.21.15.150:0/3350578949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 0x7fca54106600 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.553+0000 7fca58f7a700 1 -- 172.21.15.150:0/3350578949 >> 172.21.15.150:0/3350578949 conn(0x7fca54101850 msgr2=0x7fca54103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.553+0000 7fca58f7a700 1 -- 172.21.15.150:0/3350578949 shutdown_connections 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.553+0000 7fca58f7a700 1 -- 172.21.15.150:0/3350578949 wait complete. 2024-05-10T03:57:03.136 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.554+0000 7fca58f7a700 1 Processor -- start 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.554+0000 7fca58f7a700 1 -- start start 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.554+0000 7fca58f7a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 0x7fca5419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.554+0000 7fca58f7a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5419a940 con 0x7fca541061f0 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.554+0000 7fca5259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 0x7fca5419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.554+0000 7fca5259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 0x7fca5419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45802/0 (socket says 172.21.15.150:45802) 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.554+0000 7fca5259c700 1 -- 172.21.15.150:0/2120387266 learned_addr learned my addr 172.21.15.150:0/2120387266 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.555+0000 7fca5259c700 1 -- 172.21.15.150:0/2120387266 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca44009550 con 0x7fca541061f0 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.555+0000 7fca5259c700 1 --2- 172.21.15.150:0/2120387266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 0x7fca5419a400 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca44003f50 tx=0x7fca44011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.555+0000 7fca437fe700 1 -- 172.21.15.150:0/2120387266 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca44003890 con 0x7fca541061f0 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.555+0000 7fca437fe700 1 -- 172.21.15.150:0/2120387266 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca44020070 con 0x7fca541061f0 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.555+0000 7fca437fe700 1 -- 172.21.15.150:0/2120387266 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca4401c4b0 con 0x7fca541061f0 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.555+0000 7fca58f7a700 1 -- 172.21.15.150:0/2120387266 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5419ab40 con 0x7fca541061f0 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.555+0000 7fca58f7a700 1 -- 172.21.15.150:0/2120387266 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5419afe0 con 0x7fca541061f0 2024-05-10T03:57:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.556+0000 7fca437fe700 1 -- 172.21.15.150:0/2120387266 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fca44022030 con 0x7fca541061f0 2024-05-10T03:57:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.556+0000 7fca437fe700 1 --2- 172.21.15.150:0/2120387266 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fca3c038700 0x7fca3c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.556+0000 7fca437fe700 1 -- 172.21.15.150:0/2120387266 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fca4404f950 con 0x7fca541061f0 2024-05-10T03:57:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.556+0000 7fca51d9b700 1 -- 172.21.15.150:0/2120387266 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fca3c038700 msgr2=0x7fca3c03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/3392737437 2024-05-10T03:57:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.556+0000 7fca51d9b700 1 --2- 172.21.15.150:0/2120387266 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fca3c038700 0x7fca3c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T03:57:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.556+0000 7fca58f7a700 1 -- 172.21.15.150:0/2120387266 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5410a5f0 con 0x7fca541061f0 2024-05-10T03:57:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.561+0000 7fca437fe700 1 -- 172.21.15.150:0/2120387266 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fca4402e490 con 0x7fca541061f0 2024-05-10T03:57:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.757+0000 7fca51d9b700 1 -- 172.21.15.150:0/2120387266 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fca3c038700 msgr2=0x7fca3c03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/3392737437 2024-05-10T03:57:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.757+0000 7fca51d9b700 1 --2- 172.21.15.150:0/2120387266 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fca3c038700 0x7fca3c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T03:57:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.816+0000 7fca58f7a700 1 -- 172.21.15.150:0/2120387266 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fca5407a490 con 0x7fca541061f0 2024-05-10T03:57:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.817+0000 7fca437fe700 1 -- 172.21.15.150:0/2120387266 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fca4402d3a0 con 0x7fca541061f0 2024-05-10T03:57:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.819+0000 7fca58f7a700 1 -- 172.21.15.150:0/2120387266 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fca3c038700 msgr2=0x7fca3c03ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T03:57:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.819+0000 7fca58f7a700 1 --2- 172.21.15.150:0/2120387266 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fca3c038700 0x7fca3c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.819+0000 7fca58f7a700 1 -- 172.21.15.150:0/2120387266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 msgr2=0x7fca5419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:03.139 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.819+0000 7fca58f7a700 1 --2- 172.21.15.150:0/2120387266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 0x7fca5419a400 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca44003f50 tx=0x7fca44011b40 comp rx=0 tx=0).stop 2024-05-10T03:57:03.139 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.819+0000 7fca58f7a700 1 -- 172.21.15.150:0/2120387266 shutdown_connections 2024-05-10T03:57:03.139 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.819+0000 7fca58f7a700 1 --2- 172.21.15.150:0/2120387266 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fca3c038700 0x7fca3c03ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:03.139 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.819+0000 7fca58f7a700 1 --2- 172.21.15.150:0/2120387266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca541061f0 0x7fca5419a400 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:03.139 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.819+0000 7fca58f7a700 1 -- 172.21.15.150:0/2120387266 >> 172.21.15.150:0/2120387266 conn(0x7fca54101850 msgr2=0x7fca54102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:03.139 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.819+0000 7fca58f7a700 1 -- 172.21.15.150:0/2120387266 shutdown_connections 2024-05-10T03:57:03.139 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:02.819+0000 7fca58f7a700 1 -- 172.21.15.150:0/2120387266 wait complete. 2024-05-10T03:57:03.139 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for the mgr to restart... 2024-05-10T03:57:03.139 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for mgr epoch 5... 2024-05-10T03:57:03.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:02 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3618077242' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-10T03:57:03.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:02 smithi150 ceph-mon[42477]: mgrmap e5: x(active, since 10s) 2024-05-10T03:57:03.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:02 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2120387266' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-10T03:57:03.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:03 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:03.284+0000 7f1e67089200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:57:03.556 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:03 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:03.553+0000 7f1e67089200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T03:57:03.915 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:03 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:03.640+0000 7f1e67089200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T03:57:03.915 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:03 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:03.751+0000 7f1e67089200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T03:57:03.915 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:03 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:03.911+0000 7f1e67089200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T03:57:04.214 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:04.039+0000 7f1e67089200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T03:57:04.214 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:04.126+0000 7f1e67089200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T03:57:04.896 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:04.500+0000 7f1e67089200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T03:57:04.896 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:04.893+0000 7f1e67089200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T03:57:05.189 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:05 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:05.092+0000 7f1e67089200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T03:57:05.464 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:05 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:05.186+0000 7f1e67089200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T03:57:06.214 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:05 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:05.956+0000 7f1e67089200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T03:57:06.214 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:06 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:06.054+0000 7f1e67089200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T03:57:06.214 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:06 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:06.204+0000 7f1e67089200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T03:57:07.214 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:06 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:06.837+0000 7f1e67089200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T03:57:07.214 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:07 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:07.023+0000 7f1e67089200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T03:57:07.714 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:07 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:07.220+0000 7f1e67089200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T03:57:08.452 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:08 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:08.158+0000 7f1e67089200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T03:57:08.714 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:08 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:08.449+0000 7f1e67089200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:57:08.714 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:08 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:08.539+0000 7f1e67089200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T03:57:09.214 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:08 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:08.733+0000 7f1e67089200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T03:57:09.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:08 smithi150 ceph-mon[42477]: Active manager daemon x restarted 2024-05-10T03:57:09.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:08 smithi150 ceph-mon[42477]: Activating manager daemon x 2024-05-10T03:57:09.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:08 smithi150 ceph-mon[42477]: osdmap e2: 0 total, 0 up, 0 in 2024-05-10T03:57:09.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:08 smithi150 ceph-mon[42477]: mgrmap e6: x(active, starting, since 0.00398806s) 2024-05-10T03:57:09.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:08 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:57:09.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:08 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T03:57:09.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:08 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T03:57:09.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:08 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T03:57:09.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:08 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T03:57:09.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:08 smithi150 ceph-mon[42477]: Manager daemon x is now available 2024-05-10T03:57:09.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:08 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:09.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:08 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:09.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:08 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:57:10.124 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T03:57:10.124 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-10T03:57:10.124 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-10T03:57:10.124 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.493+0000 7fa0d0bc9700 1 Processor -- start 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.493+0000 7fa0d0bc9700 1 -- start start 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.493+0000 7fa0d0bc9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 0x7fa0cc106620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.493+0000 7fa0d0bc9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc106c10 con 0x7fa0cc106210 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.493+0000 7fa0ca59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 0x7fa0cc106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.493+0000 7fa0ca59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 0x7fa0cc106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45816/0 (socket says 172.21.15.150:45816) 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.493+0000 7fa0ca59c700 1 -- 172.21.15.150:0/3700167108 learned_addr learned my addr 172.21.15.150:0/3700167108 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.494+0000 7fa0ca59c700 1 -- 172.21.15.150:0/3700167108 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc1073e0 con 0x7fa0cc106210 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.494+0000 7fa0ca59c700 1 --2- 172.21.15.150:0/3700167108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 0x7fa0cc106620 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0098a0 tx=0x7fa0bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fb748b9b557a43 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.494+0000 7fa0c959a700 1 -- 172.21.15.150:0/3700167108 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0bc004160 con 0x7fa0cc106210 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.494+0000 7fa0c959a700 1 -- 172.21.15.150:0/3700167108 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa0bc0042c0 con 0x7fa0cc106210 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.494+0000 7fa0c959a700 1 -- 172.21.15.150:0/3700167108 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0bc011a00 con 0x7fa0cc106210 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.495+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/3700167108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 msgr2=0x7fa0cc106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.495+0000 7fa0d0bc9700 1 --2- 172.21.15.150:0/3700167108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 0x7fa0cc106620 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0098a0 tx=0x7fa0bc009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.495+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/3700167108 shutdown_connections 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.495+0000 7fa0d0bc9700 1 --2- 172.21.15.150:0/3700167108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 0x7fa0cc106620 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.495+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/3700167108 >> 172.21.15.150:0/3700167108 conn(0x7fa0cc101870 msgr2=0x7fa0cc103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.495+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/3700167108 shutdown_connections 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.495+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/3700167108 wait complete. 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.496+0000 7fa0d0bc9700 1 Processor -- start 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.497+0000 7fa0d0bc9700 1 -- start start 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.497+0000 7fa0d0bc9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 0x7fa0cc19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.497+0000 7fa0d0bc9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc19a940 con 0x7fa0cc106210 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.497+0000 7fa0ca59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 0x7fa0cc19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.497+0000 7fa0ca59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 0x7fa0cc19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45832/0 (socket says 172.21.15.150:45832) 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.497+0000 7fa0ca59c700 1 -- 172.21.15.150:0/1178943492 learned_addr learned my addr 172.21.15.150:0/1178943492 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.497+0000 7fa0ca59c700 1 -- 172.21.15.150:0/1178943492 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc009550 con 0x7fa0cc106210 2024-05-10T03:57:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.497+0000 7fa0ca59c700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 0x7fa0cc19a400 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009130 tx=0x7fa0bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:10.127 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.498+0000 7fa0bb7fe700 1 -- 172.21.15.150:0/1178943492 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0bc0038b0 con 0x7fa0cc106210 2024-05-10T03:57:10.127 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.498+0000 7fa0bb7fe700 1 -- 172.21.15.150:0/1178943492 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa0bc003a10 con 0x7fa0cc106210 2024-05-10T03:57:10.127 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.498+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/1178943492 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc19ab40 con 0x7fa0cc106210 2024-05-10T03:57:10.127 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.498+0000 7fa0bb7fe700 1 -- 172.21.15.150:0/1178943492 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0bc019550 con 0x7fa0cc106210 2024-05-10T03:57:10.127 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.498+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/1178943492 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc19af60 con 0x7fa0cc106210 2024-05-10T03:57:10.127 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.499+0000 7fa0bb7fe700 1 -- 172.21.15.150:0/1178943492 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fa0bc019810 con 0x7fa0cc106210 2024-05-10T03:57:10.127 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.499+0000 7fa0bb7fe700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 0x7fa0b403cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:10.127 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.499+0000 7fa0bb7fe700 1 -- 172.21.15.150:0/1178943492 --> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0b403d4e0 con 0x7fa0b403a8c0 2024-05-10T03:57:10.127 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.499+0000 7fa0c9d9b700 1 -- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 msgr2=0x7fa0b403cdd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/3392737437 2024-05-10T03:57:10.127 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.499+0000 7fa0bb7fe700 1 -- 172.21.15.150:0/1178943492 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa0bc04c430 con 0x7fa0cc106210 2024-05-10T03:57:10.127 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.499+0000 7fa0c9d9b700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 0x7fa0b403cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T03:57:10.127 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.700+0000 7fa0c9d9b700 1 -- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 msgr2=0x7fa0b403cdd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/3392737437 2024-05-10T03:57:10.127 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:03.700+0000 7fa0c9d9b700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 0x7fa0b403cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:04.101+0000 7fa0c9d9b700 1 -- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 msgr2=0x7fa0b403cdd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/3392737437 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:04.101+0000 7fa0c9d9b700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 0x7fa0b403cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:04.902+0000 7fa0c9d9b700 1 -- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 msgr2=0x7fa0b403cdd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/3392737437 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:04.902+0000 7fa0c9d9b700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 0x7fa0b403cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:06.503+0000 7fa0c9d9b700 1 -- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 msgr2=0x7fa0b403cdd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/3392737437 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:06.503+0000 7fa0c9d9b700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 0x7fa0b403cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:08.748+0000 7fa0bb7fe700 1 -- 172.21.15.150:0/1178943492 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mgrmap(e 6) v1 ==== 45308+0+0 (secure 0 0 0) 0x7fa0bc019cf0 con 0x7fa0cc106210 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:08.748+0000 7fa0bb7fe700 1 -- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 msgr2=0x7fa0b403cdd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:08.748+0000 7fa0bb7fe700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 0x7fa0b403cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.749+0000 7fa0bb7fe700 1 -- 172.21.15.150:0/1178943492 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7fa0bc04e0b0 con 0x7fa0cc106210 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.750+0000 7fa0bb7fe700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fa0b403dfd0 0x7fa0b40403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.750+0000 7fa0bb7fe700 1 -- 172.21.15.150:0/1178943492 --> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0b403d4e0 con 0x7fa0b403dfd0 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.750+0000 7fa0c9d9b700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fa0b403dfd0 0x7fa0b40403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.751+0000 7fa0c9d9b700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fa0b403dfd0 0x7fa0b40403f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0003a10 tx=0x7fa0c0009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.751+0000 7fa0bb7fe700 1 -- 172.21.15.150:0/1178943492 <== mgr.14120 v2:172.21.15.150:6800/999775795 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fa0b403d4e0 con 0x7fa0b403dfd0 2024-05-10T03:57:10.128 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.758+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/1178943492 --> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa0cc19b0f0 con 0x7fa0b403dfd0 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.758+0000 7fa0bb7fe700 1 -- 172.21.15.150:0/1178943492 <== mgr.14120 v2:172.21.15.150:6800/999775795 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fa0cc19b0f0 con 0x7fa0b403dfd0 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.758+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fa0b403dfd0 msgr2=0x7fa0b40403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.758+0000 7fa0d0bc9700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fa0b403dfd0 0x7fa0b40403f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0003a10 tx=0x7fa0c0009290 comp rx=0 tx=0).stop 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.758+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 msgr2=0x7fa0cc19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.758+0000 7fa0d0bc9700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 0x7fa0cc19a400 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009130 tx=0x7fa0bc011ad0 comp rx=0 tx=0).stop 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.758+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/1178943492 shutdown_connections 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.758+0000 7fa0d0bc9700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fa0b403dfd0 0x7fa0b40403f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.759+0000 7fa0d0bc9700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0cc106210 0x7fa0cc19a400 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.759+0000 7fa0d0bc9700 1 --2- 172.21.15.150:0/1178943492 >> [v2:172.21.15.150:6800/3392737437,v1:172.21.15.150:6801/3392737437] conn(0x7fa0b403a8c0 0x7fa0b403cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.759+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/1178943492 >> 172.21.15.150:0/1178943492 conn(0x7fa0cc101870 msgr2=0x7fa0cc1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.759+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/1178943492 shutdown_connections 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:09.759+0000 7fa0d0bc9700 1 -- 172.21.15.150:0/1178943492 wait complete. 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:mgr epoch 5 is available 2024-05-10T03:57:10.129 INFO:teuthology.orchestra.run.smithi150.stdout:Setting orchestrator backend to cephadm... 2024-05-10T03:57:10.204 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:09 smithi150 ceph-mon[42477]: Found migration_current of "None". Setting to last migration. 2024-05-10T03:57:10.204 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:09 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:57:10.204 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:09 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T03:57:10.204 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:09 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T03:57:10.204 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:09 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:10.204 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:09 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:10.204 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:09 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:57:10.205 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:09 smithi150 ceph-mon[42477]: mgrmap e7: x(active, since 1.00612s) 2024-05-10T03:57:10.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:10 smithi150 ceph-mon[42477]: [10/May/2024:03:57:09] ENGINE Bus STARTING 2024-05-10T03:57:10.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:10 smithi150 ceph-mon[42477]: [10/May/2024:03:57:09] ENGINE Serving on http://172.21.15.150:8765 2024-05-10T03:57:10.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:10 smithi150 ceph-mon[42477]: [10/May/2024:03:57:09] ENGINE Serving on https://172.21.15.150:7150 2024-05-10T03:57:10.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:10 smithi150 ceph-mon[42477]: [10/May/2024:03:57:09] ENGINE Bus STARTED 2024-05-10T03:57:10.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:10 smithi150 ceph-mon[42477]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-10T03:57:10.965 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:10 smithi150 ceph-mon[42477]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-10T03:57:10.965 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:10 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:10.965 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:10 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:57:11.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.439+0000 7f29a44de700 1 Processor -- start 2024-05-10T03:57:11.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.439+0000 7f29a44de700 1 -- start start 2024-05-10T03:57:11.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.439+0000 7f29a44de700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 0x7f299c10a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:11.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.439+0000 7f29a44de700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c07adc0 con 0x7f299c107e40 2024-05-10T03:57:11.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.440+0000 7f29a227a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 0x7f299c10a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:11.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.440+0000 7f29a227a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 0x7f299c10a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32834/0 (socket says 172.21.15.150:32834) 2024-05-10T03:57:11.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.440+0000 7f29a227a700 1 -- 172.21.15.150:0/1735962660 learned_addr learned my addr 172.21.15.150:0/1735962660 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:11.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.440+0000 7f29a227a700 1 -- 172.21.15.150:0/1735962660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c10a7a0 con 0x7f299c107e40 2024-05-10T03:57:11.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.440+0000 7f29a227a700 1 --2- 172.21.15.150:0/1735962660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 0x7f299c10a260 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f298c0098a0 tx=0x7f298c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfb50e3db0b6378 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:11.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.440+0000 7f29a1278700 1 -- 172.21.15.150:0/1735962660 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f298c004160 con 0x7f299c107e40 2024-05-10T03:57:11.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.440+0000 7f29a1278700 1 -- 172.21.15.150:0/1735962660 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f298c0042c0 con 0x7f299c107e40 2024-05-10T03:57:11.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.440+0000 7f29a1278700 1 -- 172.21.15.150:0/1735962660 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f298c011a00 con 0x7f299c107e40 2024-05-10T03:57:11.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.441+0000 7f29a44de700 1 -- 172.21.15.150:0/1735962660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 msgr2=0x7f299c10a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:11.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.441+0000 7f29a44de700 1 --2- 172.21.15.150:0/1735962660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 0x7f299c10a260 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f298c0098a0 tx=0x7f298c009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:11.014 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.441+0000 7f29a44de700 1 -- 172.21.15.150:0/1735962660 shutdown_connections 2024-05-10T03:57:11.014 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.441+0000 7f29a44de700 1 --2- 172.21.15.150:0/1735962660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 0x7f299c10a260 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:11.014 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.441+0000 7f29a44de700 1 -- 172.21.15.150:0/1735962660 >> 172.21.15.150:0/1735962660 conn(0x7f299c101820 msgr2=0x7f299c103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:11.014 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.441+0000 7f29a44de700 1 -- 172.21.15.150:0/1735962660 shutdown_connections 2024-05-10T03:57:11.014 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.441+0000 7f29a44de700 1 -- 172.21.15.150:0/1735962660 wait complete. 2024-05-10T03:57:11.014 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.442+0000 7f29a44de700 1 Processor -- start 2024-05-10T03:57:11.014 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.442+0000 7f29a44de700 1 -- start start 2024-05-10T03:57:11.014 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.442+0000 7f29a44de700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 0x7f299c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:11.014 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.442+0000 7f29a44de700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c19b9a0 con 0x7f299c107e40 2024-05-10T03:57:11.014 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.442+0000 7f29a227a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 0x7f299c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:11.014 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.442+0000 7f29a227a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 0x7f299c19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32844/0 (socket says 172.21.15.150:32844) 2024-05-10T03:57:11.014 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.442+0000 7f29a227a700 1 -- 172.21.15.150:0/3733635830 learned_addr learned my addr 172.21.15.150:0/3733635830 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:11.014 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.442+0000 7f29a227a700 1 -- 172.21.15.150:0/3733635830 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c009550 con 0x7f299c107e40 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.442+0000 7f29a227a700 1 --2- 172.21.15.150:0/3733635830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 0x7f299c19b460 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f298c009130 tx=0x7f298c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.443+0000 7f29937fe700 1 -- 172.21.15.150:0/3733635830 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f298c0038b0 con 0x7f299c107e40 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.443+0000 7f29a44de700 1 -- 172.21.15.150:0/3733635830 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299c19bba0 con 0x7f299c107e40 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.443+0000 7f29937fe700 1 -- 172.21.15.150:0/3733635830 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f298c003a10 con 0x7f299c107e40 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.443+0000 7f29937fe700 1 -- 172.21.15.150:0/3733635830 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f298c019550 con 0x7f299c107e40 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.443+0000 7f29a44de700 1 -- 172.21.15.150:0/3733635830 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299c19c040 con 0x7f299c107e40 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.443+0000 7f29937fe700 1 -- 172.21.15.150:0/3733635830 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f298c019810 con 0x7f299c107e40 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.443+0000 7f29a44de700 1 -- 172.21.15.150:0/3733635830 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299c10b4e0 con 0x7f299c107e40 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.444+0000 7f29937fe700 1 --2- 172.21.15.150:0/3733635830 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f2988038570 0x7f298803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.444+0000 7f29937fe700 1 -- 172.21.15.150:0/3733635830 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f298c04c5f0 con 0x7f299c107e40 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.444+0000 7f29a1a79700 1 --2- 172.21.15.150:0/3733635830 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f2988038570 0x7f298803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.444+0000 7f29a1a79700 1 --2- 172.21.15.150:0/3733635830 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f2988038570 0x7f298803aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2998006fd0 tx=0x7f2998006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.449+0000 7f29937fe700 1 -- 172.21.15.150:0/3733635830 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f298c018db0 con 0x7f299c107e40 2024-05-10T03:57:11.015 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.653+0000 7f29a44de700 1 -- 172.21.15.150:0/3733635830 --> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f299c102880 con 0x7f2988038570 2024-05-10T03:57:11.016 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.663+0000 7f29937fe700 1 -- 172.21.15.150:0/3733635830 <== mgr.14120 v2:172.21.15.150:6800/999775795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f299c102880 con 0x7f2988038570 2024-05-10T03:57:11.016 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.665+0000 7f29a44de700 1 -- 172.21.15.150:0/3733635830 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f2988038570 msgr2=0x7f298803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:11.016 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.665+0000 7f29a44de700 1 --2- 172.21.15.150:0/3733635830 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f2988038570 0x7f298803aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2998006fd0 tx=0x7f2998006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:11.016 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.665+0000 7f29a44de700 1 -- 172.21.15.150:0/3733635830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 msgr2=0x7f299c19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:11.016 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.665+0000 7f29a44de700 1 --2- 172.21.15.150:0/3733635830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 0x7f299c19b460 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f298c009130 tx=0x7f298c011ad0 comp rx=0 tx=0).stop 2024-05-10T03:57:11.016 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.665+0000 7f29a44de700 1 -- 172.21.15.150:0/3733635830 shutdown_connections 2024-05-10T03:57:11.016 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.665+0000 7f29a44de700 1 --2- 172.21.15.150:0/3733635830 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f2988038570 0x7f298803aa80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:11.016 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.665+0000 7f29a44de700 1 --2- 172.21.15.150:0/3733635830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f299c107e40 0x7f299c19b460 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:11.016 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.665+0000 7f29a44de700 1 -- 172.21.15.150:0/3733635830 >> 172.21.15.150:0/3733635830 conn(0x7f299c101820 msgr2=0x7f299c102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:11.016 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.665+0000 7f29a44de700 1 -- 172.21.15.150:0/3733635830 shutdown_connections 2024-05-10T03:57:11.016 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:10.665+0000 7f29a44de700 1 -- 172.21.15.150:0/3733635830 wait complete. 2024-05-10T03:57:11.921 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-10T03:57:11.921 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.345+0000 7fd5512b4700 1 Processor -- start 2024-05-10T03:57:11.921 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.345+0000 7fd5512b4700 1 -- start start 2024-05-10T03:57:11.921 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.346+0000 7fd5512b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 0x7fd54c10a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:11.921 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.346+0000 7fd5512b4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c07adc0 con 0x7fd54c107e60 2024-05-10T03:57:11.921 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.346+0000 7fd54affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 0x7fd54c10a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:11.921 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.346+0000 7fd54affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 0x7fd54c10a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32850/0 (socket says 172.21.15.150:32850) 2024-05-10T03:57:11.921 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.346+0000 7fd54affd700 1 -- 172.21.15.150:0/768302036 learned_addr learned my addr 172.21.15.150:0/768302036 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:11.921 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.346+0000 7fd54affd700 1 -- 172.21.15.150:0/768302036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c10a7c0 con 0x7fd54c107e60 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.346+0000 7fd54affd700 1 --2- 172.21.15.150:0/768302036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 0x7fd54c10a280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5400098a0 tx=0x7fd540009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1110b131f980219c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.346+0000 7fd549ffb700 1 -- 172.21.15.150:0/768302036 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd540004160 con 0x7fd54c107e60 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.346+0000 7fd549ffb700 1 -- 172.21.15.150:0/768302036 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd5400042c0 con 0x7fd54c107e60 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.346+0000 7fd549ffb700 1 -- 172.21.15.150:0/768302036 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd540011a00 con 0x7fd54c107e60 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.347+0000 7fd5512b4700 1 -- 172.21.15.150:0/768302036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 msgr2=0x7fd54c10a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.347+0000 7fd5512b4700 1 --2- 172.21.15.150:0/768302036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 0x7fd54c10a280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5400098a0 tx=0x7fd540009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.347+0000 7fd5512b4700 1 -- 172.21.15.150:0/768302036 shutdown_connections 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.347+0000 7fd5512b4700 1 --2- 172.21.15.150:0/768302036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 0x7fd54c10a280 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.347+0000 7fd5512b4700 1 -- 172.21.15.150:0/768302036 >> 172.21.15.150:0/768302036 conn(0x7fd54c101880 msgr2=0x7fd54c103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.347+0000 7fd5512b4700 1 -- 172.21.15.150:0/768302036 shutdown_connections 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.347+0000 7fd5512b4700 1 -- 172.21.15.150:0/768302036 wait complete. 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.348+0000 7fd5512b4700 1 Processor -- start 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.348+0000 7fd5512b4700 1 -- start start 2024-05-10T03:57:11.922 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.348+0000 7fd5512b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 0x7fd54c1075e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.348+0000 7fd5512b4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c107b20 con 0x7fd54c107e60 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.348+0000 7fd54affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 0x7fd54c1075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.348+0000 7fd54affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 0x7fd54c1075e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32856/0 (socket says 172.21.15.150:32856) 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.348+0000 7fd54affd700 1 -- 172.21.15.150:0/4187554771 learned_addr learned my addr 172.21.15.150:0/4187554771 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.349+0000 7fd54affd700 1 -- 172.21.15.150:0/4187554771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd540009550 con 0x7fd54c107e60 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.349+0000 7fd54affd700 1 --2- 172.21.15.150:0/4187554771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 0x7fd54c1075e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd540009130 tx=0x7fd540011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.349+0000 7fd533fff700 1 -- 172.21.15.150:0/4187554771 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5400038b0 con 0x7fd54c107e60 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.349+0000 7fd533fff700 1 -- 172.21.15.150:0/4187554771 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd540003a10 con 0x7fd54c107e60 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.349+0000 7fd533fff700 1 -- 172.21.15.150:0/4187554771 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd540019550 con 0x7fd54c107e60 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.349+0000 7fd5512b4700 1 -- 172.21.15.150:0/4187554771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54c105d70 con 0x7fd54c107e60 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.349+0000 7fd5512b4700 1 -- 172.21.15.150:0/4187554771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54c106210 con 0x7fd54c107e60 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.350+0000 7fd533fff700 1 -- 172.21.15.150:0/4187554771 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7fd5400197d0 con 0x7fd54c107e60 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.350+0000 7fd5512b4700 1 -- 172.21.15.150:0/4187554771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54c062840 con 0x7fd54c107e60 2024-05-10T03:57:11.923 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.350+0000 7fd533fff700 1 --2- 172.21.15.150:0/4187554771 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fd534038590 0x7fd53403aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:11.924 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.350+0000 7fd533fff700 1 -- 172.21.15.150:0/4187554771 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd54004c800 con 0x7fd54c107e60 2024-05-10T03:57:11.924 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.350+0000 7fd54a7fc700 1 --2- 172.21.15.150:0/4187554771 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fd534038590 0x7fd53403aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:11.924 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.351+0000 7fd54a7fc700 1 --2- 172.21.15.150:0/4187554771 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fd534038590 0x7fd53403aaa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd53c006fd0 tx=0x7fd53c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:11.924 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.355+0000 7fd533fff700 1 -- 172.21.15.150:0/4187554771 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd540018b80 con 0x7fd54c107e60 2024-05-10T03:57:11.924 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.549+0000 7fd5512b4700 1 -- 172.21.15.150:0/4187554771 --> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fd54c106700 con 0x7fd534038590 2024-05-10T03:57:11.924 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.550+0000 7fd533fff700 1 -- 172.21.15.150:0/4187554771 <== mgr.14120 v2:172.21.15.150:6800/999775795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fd54c106700 con 0x7fd534038590 2024-05-10T03:57:11.924 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.552+0000 7fd5512b4700 1 -- 172.21.15.150:0/4187554771 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fd534038590 msgr2=0x7fd53403aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:11.924 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.552+0000 7fd5512b4700 1 --2- 172.21.15.150:0/4187554771 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fd534038590 0x7fd53403aaa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd53c006fd0 tx=0x7fd53c006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:11.924 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.552+0000 7fd5512b4700 1 -- 172.21.15.150:0/4187554771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 msgr2=0x7fd54c1075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:11.924 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.552+0000 7fd5512b4700 1 --2- 172.21.15.150:0/4187554771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 0x7fd54c1075e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd540009130 tx=0x7fd540011ad0 comp rx=0 tx=0).stop 2024-05-10T03:57:11.924 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.552+0000 7fd5512b4700 1 -- 172.21.15.150:0/4187554771 shutdown_connections 2024-05-10T03:57:11.924 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.552+0000 7fd5512b4700 1 --2- 172.21.15.150:0/4187554771 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fd534038590 0x7fd53403aaa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:11.924 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.552+0000 7fd5512b4700 1 --2- 172.21.15.150:0/4187554771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54c107e60 0x7fd54c1075e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:11.925 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.552+0000 7fd5512b4700 1 -- 172.21.15.150:0/4187554771 >> 172.21.15.150:0/4187554771 conn(0x7fd54c101880 msgr2=0x7fd54c102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:11.925 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.552+0000 7fd5512b4700 1 -- 172.21.15.150:0/4187554771 shutdown_connections 2024-05-10T03:57:11.925 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:11.552+0000 7fd5512b4700 1 -- 172.21.15.150:0/4187554771 wait complete. 2024-05-10T03:57:11.925 INFO:teuthology.orchestra.run.smithi150.stdout:Generating ssh key... 2024-05-10T03:57:11.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:11 smithi150 ceph-mon[42477]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:11.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:11 smithi150 ceph-mon[42477]: mgrmap e8: x(active, since 2s) 2024-05-10T03:57:13.056 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:12 smithi150 ceph-mon[42477]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:13.056 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: Generating public/private rsa key pair. 2024-05-10T03:57:13.056 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: Your identification has been saved in /tmp/tmpuj1e3xfe/key. 2024-05-10T03:57:13.056 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: Your public key has been saved in /tmp/tmpuj1e3xfe/key.pub. 2024-05-10T03:57:13.056 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: The key fingerprint is: 2024-05-10T03:57:13.056 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: SHA256:r9JBS4ussxpeniP0R36VSTMES5UIKw/PjJ5KpEaWKeA ceph-325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:57:13.056 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: The key's randomart image is: 2024-05-10T03:57:13.056 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: +---[RSA 3072]----+ 2024-05-10T03:57:13.056 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: | ..o+.. | 2024-05-10T03:57:13.056 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: | o..o | 2024-05-10T03:57:13.056 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: |. o . .. | 2024-05-10T03:57:13.057 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: |o o O o + | 2024-05-10T03:57:13.057 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: |.E. ..=+S+ = | 2024-05-10T03:57:13.057 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: |+o.. .+ +.+ | 2024-05-10T03:57:13.057 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: |.oo.++ . o. | 2024-05-10T03:57:13.057 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: |.o.==.+ o. | 2024-05-10T03:57:13.057 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: | +o+= o. | 2024-05-10T03:57:13.057 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: +----[SHA256]-----+ 2024-05-10T03:57:13.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.513+0000 7f3d7a410700 1 Processor -- start 2024-05-10T03:57:13.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.513+0000 7f3d7a410700 1 -- start start 2024-05-10T03:57:13.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.514+0000 7f3d7a410700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 0x7f3d7410a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:13.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.514+0000 7f3d7a410700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7407ade0 con 0x7f3d74107e10 2024-05-10T03:57:13.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.514+0000 7f3d73fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 0x7f3d7410a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:13.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.514+0000 7f3d73fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 0x7f3d7410a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32860/0 (socket says 172.21.15.150:32860) 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.514+0000 7f3d73fff700 1 -- 172.21.15.150:0/2994262204 learned_addr learned my addr 172.21.15.150:0/2994262204 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.514+0000 7f3d73fff700 1 -- 172.21.15.150:0/2994262204 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7410a770 con 0x7f3d74107e10 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.514+0000 7f3d73fff700 1 --2- 172.21.15.150:0/2994262204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 0x7f3d7410a230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3d680098a0 tx=0x7f3d68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7efc5adfd7a8bdf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.514+0000 7f3d72ffd700 1 -- 172.21.15.150:0/2994262204 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d68004160 con 0x7f3d74107e10 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.515+0000 7f3d72ffd700 1 -- 172.21.15.150:0/2994262204 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3d680042c0 con 0x7f3d74107e10 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.515+0000 7f3d72ffd700 1 -- 172.21.15.150:0/2994262204 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d68011a00 con 0x7f3d74107e10 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.515+0000 7f3d7a410700 1 -- 172.21.15.150:0/2994262204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 msgr2=0x7f3d7410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.515+0000 7f3d7a410700 1 --2- 172.21.15.150:0/2994262204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 0x7f3d7410a230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3d680098a0 tx=0x7f3d68009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.515+0000 7f3d7a410700 1 -- 172.21.15.150:0/2994262204 shutdown_connections 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.515+0000 7f3d7a410700 1 --2- 172.21.15.150:0/2994262204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 0x7f3d7410a230 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.515+0000 7f3d7a410700 1 -- 172.21.15.150:0/2994262204 >> 172.21.15.150:0/2994262204 conn(0x7f3d74101850 msgr2=0x7f3d74103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.515+0000 7f3d7a410700 1 -- 172.21.15.150:0/2994262204 shutdown_connections 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.515+0000 7f3d7a410700 1 -- 172.21.15.150:0/2994262204 wait complete. 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.516+0000 7f3d7a410700 1 Processor -- start 2024-05-10T03:57:13.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.516+0000 7f3d7a410700 1 -- start start 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.516+0000 7f3d7a410700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 0x7f3d7419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.516+0000 7f3d7a410700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7419ba00 con 0x7f3d74107e10 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.516+0000 7f3d73fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 0x7f3d7419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.516+0000 7f3d73fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 0x7f3d7419b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32870/0 (socket says 172.21.15.150:32870) 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.516+0000 7f3d73fff700 1 -- 172.21.15.150:0/3986155664 learned_addr learned my addr 172.21.15.150:0/3986155664 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.517+0000 7f3d73fff700 1 -- 172.21.15.150:0/3986155664 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d68009550 con 0x7f3d74107e10 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.517+0000 7f3d73fff700 1 --2- 172.21.15.150:0/3986155664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 0x7f3d7419b4c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009130 tx=0x7f3d68011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.517+0000 7f3d717fa700 1 -- 172.21.15.150:0/3986155664 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d680038b0 con 0x7f3d74107e10 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.517+0000 7f3d717fa700 1 -- 172.21.15.150:0/3986155664 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3d68003a10 con 0x7f3d74107e10 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.517+0000 7f3d7a410700 1 -- 172.21.15.150:0/3986155664 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7419bc00 con 0x7f3d74107e10 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.517+0000 7f3d717fa700 1 -- 172.21.15.150:0/3986155664 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d68019550 con 0x7f3d74107e10 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.517+0000 7f3d7a410700 1 -- 172.21.15.150:0/3986155664 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7419c0a0 con 0x7f3d74107e10 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.518+0000 7f3d717fa700 1 -- 172.21.15.150:0/3986155664 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f3d680197d0 con 0x7f3d74107e10 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.518+0000 7f3d7a410700 1 -- 172.21.15.150:0/3986155664 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7410b510 con 0x7f3d74107e10 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.518+0000 7f3d717fa700 1 --2- 172.21.15.150:0/3986155664 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f3d5c0386b0 0x7f3d5c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:13.256 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.518+0000 7f3d717fa700 1 -- 172.21.15.150:0/3986155664 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3d6804c7a0 con 0x7f3d74107e10 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.518+0000 7f3d737fe700 1 --2- 172.21.15.150:0/3986155664 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f3d5c0386b0 0x7f3d5c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.519+0000 7f3d737fe700 1 --2- 172.21.15.150:0/3986155664 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f3d5c0386b0 0x7f3d5c03abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3d64006fd0 tx=0x7f3d64006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.524+0000 7f3d717fa700 1 -- 172.21.15.150:0/3986155664 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3d68018b80 con 0x7f3d74107e10 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.715+0000 7f3d7a410700 1 -- 172.21.15.150:0/3986155664 --> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d741029a0 con 0x7f3d5c0386b0 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.870+0000 7f3d717fa700 1 -- 172.21.15.150:0/3986155664 <== mgr.14120 v2:172.21.15.150:6800/999775795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3d741029a0 con 0x7f3d5c0386b0 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.873+0000 7f3d7a410700 1 -- 172.21.15.150:0/3986155664 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f3d5c0386b0 msgr2=0x7f3d5c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.873+0000 7f3d7a410700 1 --2- 172.21.15.150:0/3986155664 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f3d5c0386b0 0x7f3d5c03abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3d64006fd0 tx=0x7f3d64006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.873+0000 7f3d7a410700 1 -- 172.21.15.150:0/3986155664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 msgr2=0x7f3d7419b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.873+0000 7f3d7a410700 1 --2- 172.21.15.150:0/3986155664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 0x7f3d7419b4c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009130 tx=0x7f3d68011ad0 comp rx=0 tx=0).stop 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.873+0000 7f3d7a410700 1 -- 172.21.15.150:0/3986155664 shutdown_connections 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.873+0000 7f3d7a410700 1 --2- 172.21.15.150:0/3986155664 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f3d5c0386b0 0x7f3d5c03abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.873+0000 7f3d7a410700 1 --2- 172.21.15.150:0/3986155664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d74107e10 0x7f3d7419b4c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.873+0000 7f3d7a410700 1 -- 172.21.15.150:0/3986155664 >> 172.21.15.150:0/3986155664 conn(0x7f3d74101850 msgr2=0x7f3d74102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.873+0000 7f3d7a410700 1 -- 172.21.15.150:0/3986155664 shutdown_connections 2024-05-10T03:57:13.257 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:12.873+0000 7f3d7a410700 1 -- 172.21.15.150:0/3986155664 wait complete. 2024-05-10T03:57:14.162 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCF8OsiFjLszF1SuH1SpsUyhZ8TstfczHwY/MdC5+ASFQMwKYIBR50HANADpJYs5vMIyfZD4wJpjnFc5foOY3pmIsSwfPglh3nU69tQG7plwF9g8Tn4DTPIaVHDHFHpELce0dayv1tk8WUOYTc8YiJoowGbyb2EZlNGPSU4ubI3pgf+ludW7luzVAp99LyTZAGqlxKOMoxsc+JTCAcWbmAsial8GcJrpmNOjbSA7H5KC3CneWuT07qxo0jHfAJn82SC4SxlP2bj9CzB9c1iC9BYajQfKxOkWxFv+iWlk2FruYmHkE3UJ8FhGnwMIgsUtz8UZkwZ5Z+fuXUD6NvPfVhKt9VoJWPU/NXnydNns2eF2yA1TTBA8aLEDqsbK5pLtSqRuSHahxDC5uOtCx19m/rd0+YNM7bQR0ZebexMOvTx0FFnIGek6eGuKASexTBuQfWhEnckQIpY6ar6MqZTTBaFyw6eZ3ivfDxwGHMhq8mYmGoxOFOsqKyH04kFZjSKt8= ceph-325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:57:14.162 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.596+0000 7f20985cd700 1 Processor -- start 2024-05-10T03:57:14.163 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.596+0000 7f20985cd700 1 -- start start 2024-05-10T03:57:14.163 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.596+0000 7f20985cd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 0x7f2090107be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:14.163 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.596+0000 7f20985cd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20901081d0 con 0x7f20901077d0 2024-05-10T03:57:14.163 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.596+0000 7f2096369700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 0x7f2090107be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:14.163 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.596+0000 7f2096369700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 0x7f2090107be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32876/0 (socket says 172.21.15.150:32876) 2024-05-10T03:57:14.163 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.596+0000 7f2096369700 1 -- 172.21.15.150:0/107810994 learned_addr learned my addr 172.21.15.150:0/107810994 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:14.163 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.597+0000 7f2096369700 1 -- 172.21.15.150:0/107810994 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2090106450 con 0x7f20901077d0 2024-05-10T03:57:14.163 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.597+0000 7f2096369700 1 --2- 172.21.15.150:0/107810994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 0x7f2090107be0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f208c0098a0 tx=0x7f208c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96815937b20dd81d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:14.163 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.597+0000 7f2095367700 1 -- 172.21.15.150:0/107810994 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f208c004160 con 0x7f20901077d0 2024-05-10T03:57:14.163 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.597+0000 7f2095367700 1 -- 172.21.15.150:0/107810994 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f208c0042c0 con 0x7f20901077d0 2024-05-10T03:57:14.163 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.597+0000 7f2095367700 1 -- 172.21.15.150:0/107810994 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f208c011a00 con 0x7f20901077d0 2024-05-10T03:57:14.163 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.598+0000 7f20985cd700 1 -- 172.21.15.150:0/107810994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 msgr2=0x7f2090107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:14.164 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.598+0000 7f20985cd700 1 --2- 172.21.15.150:0/107810994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 0x7f2090107be0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f208c0098a0 tx=0x7f208c009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:14.164 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.598+0000 7f20985cd700 1 -- 172.21.15.150:0/107810994 shutdown_connections 2024-05-10T03:57:14.164 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.598+0000 7f20985cd700 1 --2- 172.21.15.150:0/107810994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 0x7f2090107be0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:14.164 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.598+0000 7f20985cd700 1 -- 172.21.15.150:0/107810994 >> 172.21.15.150:0/107810994 conn(0x7f2090100ff0 msgr2=0x7f2090103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:14.164 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.598+0000 7f20985cd700 1 -- 172.21.15.150:0/107810994 shutdown_connections 2024-05-10T03:57:14.164 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.598+0000 7f20985cd700 1 -- 172.21.15.150:0/107810994 wait complete. 2024-05-10T03:57:14.164 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.599+0000 7f20985cd700 1 Processor -- start 2024-05-10T03:57:14.164 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.599+0000 7f20985cd700 1 -- start start 2024-05-10T03:57:14.164 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.599+0000 7f20985cd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 0x7f2090105d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:14.164 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.599+0000 7f20985cd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2090106260 con 0x7f20901077d0 2024-05-10T03:57:14.164 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.599+0000 7f2096369700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 0x7f2090105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:14.164 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.599+0000 7f2096369700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 0x7f2090105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32886/0 (socket says 172.21.15.150:32886) 2024-05-10T03:57:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.599+0000 7f2096369700 1 -- 172.21.15.150:0/2794914703 learned_addr learned my addr 172.21.15.150:0/2794914703 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.599+0000 7f2096369700 1 -- 172.21.15.150:0/2794914703 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c009550 con 0x7f20901077d0 2024-05-10T03:57:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.599+0000 7f2096369700 1 --2- 172.21.15.150:0/2794914703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 0x7f2090105d20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f208c009130 tx=0x7f208c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.599+0000 7f20877fe700 1 -- 172.21.15.150:0/2794914703 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f208c0038b0 con 0x7f20901077d0 2024-05-10T03:57:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.599+0000 7f20877fe700 1 -- 172.21.15.150:0/2794914703 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f208c003a10 con 0x7f20901077d0 2024-05-10T03:57:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.599+0000 7f20877fe700 1 -- 172.21.15.150:0/2794914703 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f208c019550 con 0x7f20901077d0 2024-05-10T03:57:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.600+0000 7f20985cd700 1 -- 172.21.15.150:0/2794914703 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209019e220 con 0x7f20901077d0 2024-05-10T03:57:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.600+0000 7f20985cd700 1 -- 172.21.15.150:0/2794914703 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209019e660 con 0x7f20901077d0 2024-05-10T03:57:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.601+0000 7f20877fe700 1 -- 172.21.15.150:0/2794914703 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f208c0197c0 con 0x7f20901077d0 2024-05-10T03:57:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.601+0000 7f20985cd700 1 -- 172.21.15.150:0/2794914703 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209010a690 con 0x7f20901077d0 2024-05-10T03:57:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.601+0000 7f20877fe700 1 --2- 172.21.15.150:0/2794914703 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f207c038700 0x7f207c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.601+0000 7f20877fe700 1 -- 172.21.15.150:0/2794914703 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f208c04ca20 con 0x7f20901077d0 2024-05-10T03:57:14.166 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.601+0000 7f2095b68700 1 --2- 172.21.15.150:0/2794914703 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f207c038700 0x7f207c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:14.166 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.601+0000 7f2095b68700 1 --2- 172.21.15.150:0/2794914703 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f207c038700 0x7f207c03ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2080006fd0 tx=0x7f2080006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:14.166 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.606+0000 7f20877fe700 1 -- 172.21.15.150:0/2794914703 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f208c0189a0 con 0x7f20901077d0 2024-05-10T03:57:14.166 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.798+0000 7f20985cd700 1 -- 172.21.15.150:0/2794914703 --> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f2090102110 con 0x7f207c038700 2024-05-10T03:57:14.166 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.799+0000 7f20877fe700 1 -- 172.21.15.150:0/2794914703 <== mgr.14120 v2:172.21.15.150:6800/999775795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f2090102110 con 0x7f207c038700 2024-05-10T03:57:14.166 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.801+0000 7f20985cd700 1 -- 172.21.15.150:0/2794914703 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f207c038700 msgr2=0x7f207c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:14.166 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.801+0000 7f20985cd700 1 --2- 172.21.15.150:0/2794914703 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f207c038700 0x7f207c03ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2080006fd0 tx=0x7f2080006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:14.166 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.801+0000 7f20985cd700 1 -- 172.21.15.150:0/2794914703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 msgr2=0x7f2090105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:14.166 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.801+0000 7f20985cd700 1 --2- 172.21.15.150:0/2794914703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 0x7f2090105d20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f208c009130 tx=0x7f208c011ad0 comp rx=0 tx=0).stop 2024-05-10T03:57:14.166 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.801+0000 7f20985cd700 1 -- 172.21.15.150:0/2794914703 shutdown_connections 2024-05-10T03:57:14.166 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.801+0000 7f20985cd700 1 --2- 172.21.15.150:0/2794914703 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f207c038700 0x7f207c03ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:14.166 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.801+0000 7f20985cd700 1 --2- 172.21.15.150:0/2794914703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20901077d0 0x7f2090105d20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:14.167 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.801+0000 7f20985cd700 1 -- 172.21.15.150:0/2794914703 >> 172.21.15.150:0/2794914703 conn(0x7f2090100ff0 msgr2=0x7f20901019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:14.167 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.801+0000 7f20985cd700 1 -- 172.21.15.150:0/2794914703 shutdown_connections 2024-05-10T03:57:14.167 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:13.801+0000 7f20985cd700 1 -- 172.21.15.150:0/2794914703 wait complete. 2024-05-10T03:57:14.167 INFO:teuthology.orchestra.run.smithi150.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-10T03:57:14.167 INFO:teuthology.orchestra.run.smithi150.stdout:Adding key to root@localhost authorized_keys... 2024-05-10T03:57:14.167 INFO:teuthology.orchestra.run.smithi150.stdout:Adding host smithi150... 2024-05-10T03:57:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:13 smithi150 ceph-mon[42477]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:13 smithi150 ceph-mon[42477]: Generating ssh key... 2024-05-10T03:57:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:13 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:13 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:15.098 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:14 smithi150 ceph-mon[42477]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:16.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:15 smithi150 ceph-mon[42477]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "addr": "172.21.15.150", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:17.155 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout Added host 'smithi150' with addr '172.21.15.150' 2024-05-10T03:57:17.155 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.532+0000 7f0a4459a700 1 Processor -- start 2024-05-10T03:57:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.532+0000 7f0a4459a700 1 -- start start 2024-05-10T03:57:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.532+0000 7f0a4459a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 0x7f0a3c10a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.532+0000 7f0a4459a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c07ade0 con 0x7f0a3c107e20 2024-05-10T03:57:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.532+0000 7f0a42336700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 0x7f0a3c10a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.532+0000 7f0a42336700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 0x7f0a3c10a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32900/0 (socket says 172.21.15.150:32900) 2024-05-10T03:57:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.532+0000 7f0a42336700 1 -- 172.21.15.150:0/3551282474 learned_addr learned my addr 172.21.15.150:0/3551282474 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.533+0000 7f0a42336700 1 -- 172.21.15.150:0/3551282474 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c10a780 con 0x7f0a3c107e20 2024-05-10T03:57:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.533+0000 7f0a42336700 1 --2- 172.21.15.150:0/3551282474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 0x7f0a3c10a240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0a380098a0 tx=0x7f0a38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39799711d577300 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.533+0000 7f0a41334700 1 -- 172.21.15.150:0/3551282474 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a38004160 con 0x7f0a3c107e20 2024-05-10T03:57:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.533+0000 7f0a41334700 1 -- 172.21.15.150:0/3551282474 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0a380042c0 con 0x7f0a3c107e20 2024-05-10T03:57:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.533+0000 7f0a41334700 1 -- 172.21.15.150:0/3551282474 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a38011a00 con 0x7f0a3c107e20 2024-05-10T03:57:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.534+0000 7f0a4459a700 1 -- 172.21.15.150:0/3551282474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 msgr2=0x7f0a3c10a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.534+0000 7f0a4459a700 1 --2- 172.21.15.150:0/3551282474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 0x7f0a3c10a240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0a380098a0 tx=0x7f0a38009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.534+0000 7f0a4459a700 1 -- 172.21.15.150:0/3551282474 shutdown_connections 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.534+0000 7f0a4459a700 1 --2- 172.21.15.150:0/3551282474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 0x7f0a3c10a240 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.534+0000 7f0a4459a700 1 -- 172.21.15.150:0/3551282474 >> 172.21.15.150:0/3551282474 conn(0x7f0a3c101820 msgr2=0x7f0a3c103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.534+0000 7f0a4459a700 1 -- 172.21.15.150:0/3551282474 shutdown_connections 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.534+0000 7f0a4459a700 1 -- 172.21.15.150:0/3551282474 wait complete. 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.534+0000 7f0a4459a700 1 Processor -- start 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.534+0000 7f0a4459a700 1 -- start start 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.535+0000 7f0a4459a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 0x7f0a3c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.535+0000 7f0a4459a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c19b9a0 con 0x7f0a3c107e20 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.535+0000 7f0a42336700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 0x7f0a3c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.535+0000 7f0a42336700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 0x7f0a3c19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32910/0 (socket says 172.21.15.150:32910) 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.535+0000 7f0a42336700 1 -- 172.21.15.150:0/1466962093 learned_addr learned my addr 172.21.15.150:0/1466962093 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.535+0000 7f0a42336700 1 -- 172.21.15.150:0/1466962093 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a38009550 con 0x7f0a3c107e20 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.535+0000 7f0a42336700 1 --2- 172.21.15.150:0/1466962093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 0x7f0a3c19b460 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009130 tx=0x7f0a38011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:17.157 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.535+0000 7f0a337fe700 1 -- 172.21.15.150:0/1466962093 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a380038b0 con 0x7f0a3c107e20 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.535+0000 7f0a337fe700 1 -- 172.21.15.150:0/1466962093 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0a38003a10 con 0x7f0a3c107e20 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.535+0000 7f0a337fe700 1 -- 172.21.15.150:0/1466962093 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a38019550 con 0x7f0a3c107e20 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.535+0000 7f0a4459a700 1 -- 172.21.15.150:0/1466962093 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c19bba0 con 0x7f0a3c107e20 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.535+0000 7f0a4459a700 1 -- 172.21.15.150:0/1466962093 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c19c040 con 0x7f0a3c107e20 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.536+0000 7f0a4459a700 1 -- 172.21.15.150:0/1466962093 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c10b520 con 0x7f0a3c107e20 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.537+0000 7f0a337fe700 1 -- 172.21.15.150:0/1466962093 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f0a3801a960 con 0x7f0a3c107e20 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.537+0000 7f0a337fe700 1 --2- 172.21.15.150:0/1466962093 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f0a280386e0 0x7f0a2803abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.537+0000 7f0a337fe700 1 -- 172.21.15.150:0/1466962093 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0a3804cf70 con 0x7f0a3c107e20 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.537+0000 7f0a41b35700 1 --2- 172.21.15.150:0/1466962093 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f0a280386e0 0x7f0a2803abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.538+0000 7f0a41b35700 1 --2- 172.21.15.150:0/1466962093 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f0a280386e0 0x7f0a2803abf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c006fd0 tx=0x7f0a2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.542+0000 7f0a337fe700 1 -- 172.21.15.150:0/1466962093 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0a38003b80 con 0x7f0a3c107e20 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:14.737+0000 7f0a4459a700 1 -- 172.21.15.150:0/1466962093 --> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi150", "addr": "172.21.15.150", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a3c102880 con 0x7f0a280386e0 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:16.844+0000 7f0a337fe700 1 -- 172.21.15.150:0/1466962093 <== mgr.14120 v2:172.21.15.150:6800/999775795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0a3c102880 con 0x7f0a280386e0 2024-05-10T03:57:17.158 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:16.847+0000 7f0a4459a700 1 -- 172.21.15.150:0/1466962093 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f0a280386e0 msgr2=0x7f0a2803abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:17.159 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:16.847+0000 7f0a4459a700 1 --2- 172.21.15.150:0/1466962093 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f0a280386e0 0x7f0a2803abf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c006fd0 tx=0x7f0a2c006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:17.159 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:16.847+0000 7f0a4459a700 1 -- 172.21.15.150:0/1466962093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 msgr2=0x7f0a3c19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:17.159 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:16.847+0000 7f0a4459a700 1 --2- 172.21.15.150:0/1466962093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 0x7f0a3c19b460 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009130 tx=0x7f0a38011ad0 comp rx=0 tx=0).stop 2024-05-10T03:57:17.159 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:16.847+0000 7f0a4459a700 1 -- 172.21.15.150:0/1466962093 shutdown_connections 2024-05-10T03:57:17.159 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:16.847+0000 7f0a4459a700 1 --2- 172.21.15.150:0/1466962093 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f0a280386e0 0x7f0a2803abf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:17.159 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:16.847+0000 7f0a4459a700 1 --2- 172.21.15.150:0/1466962093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a3c107e20 0x7f0a3c19b460 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:17.159 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:16.847+0000 7f0a4459a700 1 -- 172.21.15.150:0/1466962093 >> 172.21.15.150:0/1466962093 conn(0x7f0a3c101820 msgr2=0x7f0a3c102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:17.159 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:16.847+0000 7f0a4459a700 1 -- 172.21.15.150:0/1466962093 shutdown_connections 2024-05-10T03:57:17.159 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:16.847+0000 7f0a4459a700 1 -- 172.21.15.150:0/1466962093 wait complete. 2024-05-10T03:57:17.159 INFO:teuthology.orchestra.run.smithi150.stdout:Deploying unmanaged mon service... 2024-05-10T03:57:17.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:16 smithi150 ceph-mon[42477]: Deploying cephadm binary to smithi150 2024-05-10T03:57:17.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:16 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:17.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:16 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:57:18.142 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:17 smithi150 ceph-mon[42477]: Added host smithi150 2024-05-10T03:57:18.142 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:17 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:18.199 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-10T03:57:18.199 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.503+0000 7fb909f12700 1 Processor -- start 2024-05-10T03:57:18.199 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.504+0000 7fb909f12700 1 -- start start 2024-05-10T03:57:18.199 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.504+0000 7fb909f12700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 0x7fb90410a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:18.199 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.504+0000 7fb909f12700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90407ade0 con 0x7fb904107e10 2024-05-10T03:57:18.199 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.504+0000 7fb9037fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 0x7fb90410a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:18.199 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.504+0000 7fb9037fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 0x7fb90410a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32920/0 (socket says 172.21.15.150:32920) 2024-05-10T03:57:18.199 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.504+0000 7fb9037fe700 1 -- 172.21.15.150:0/929725766 learned_addr learned my addr 172.21.15.150:0/929725766 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:18.199 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.505+0000 7fb9037fe700 1 -- 172.21.15.150:0/929725766 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90410a770 con 0x7fb904107e10 2024-05-10T03:57:18.199 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.505+0000 7fb9037fe700 1 --2- 172.21.15.150:0/929725766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 0x7fb90410a230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40098a0 tx=0x7fb8f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c1dcb145f90c4b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:18.199 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.505+0000 7fb9027fc700 1 -- 172.21.15.150:0/929725766 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8f4004160 con 0x7fb904107e10 2024-05-10T03:57:18.200 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.505+0000 7fb9027fc700 1 -- 172.21.15.150:0/929725766 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb8f40042c0 con 0x7fb904107e10 2024-05-10T03:57:18.200 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.505+0000 7fb9027fc700 1 -- 172.21.15.150:0/929725766 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8f4011a00 con 0x7fb904107e10 2024-05-10T03:57:18.200 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.506+0000 7fb909f12700 1 -- 172.21.15.150:0/929725766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 msgr2=0x7fb90410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:18.200 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.506+0000 7fb909f12700 1 --2- 172.21.15.150:0/929725766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 0x7fb90410a230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40098a0 tx=0x7fb8f4009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:18.200 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.506+0000 7fb909f12700 1 -- 172.21.15.150:0/929725766 shutdown_connections 2024-05-10T03:57:18.200 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.506+0000 7fb909f12700 1 --2- 172.21.15.150:0/929725766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 0x7fb90410a230 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:18.200 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.506+0000 7fb909f12700 1 -- 172.21.15.150:0/929725766 >> 172.21.15.150:0/929725766 conn(0x7fb904101910 msgr2=0x7fb904103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:18.200 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.506+0000 7fb909f12700 1 -- 172.21.15.150:0/929725766 shutdown_connections 2024-05-10T03:57:18.200 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.506+0000 7fb909f12700 1 -- 172.21.15.150:0/929725766 wait complete. 2024-05-10T03:57:18.200 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.507+0000 7fb909f12700 1 Processor -- start 2024-05-10T03:57:18.200 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.507+0000 7fb909f12700 1 -- start start 2024-05-10T03:57:18.200 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.508+0000 7fb909f12700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 0x7fb90419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:18.200 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.508+0000 7fb909f12700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90419b9b0 con 0x7fb904107e10 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.508+0000 7fb9037fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 0x7fb90419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.508+0000 7fb9037fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 0x7fb90419b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32928/0 (socket says 172.21.15.150:32928) 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.508+0000 7fb9037fe700 1 -- 172.21.15.150:0/1513198811 learned_addr learned my addr 172.21.15.150:0/1513198811 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.508+0000 7fb9037fe700 1 -- 172.21.15.150:0/1513198811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f4009550 con 0x7fb904107e10 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.508+0000 7fb9037fe700 1 --2- 172.21.15.150:0/1513198811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 0x7fb90419b470 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4011710 tx=0x7fb8f4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.508+0000 7fb900ff9700 1 -- 172.21.15.150:0/1513198811 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8f4003aa0 con 0x7fb904107e10 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.508+0000 7fb909f12700 1 -- 172.21.15.150:0/1513198811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90419bbb0 con 0x7fb904107e10 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.508+0000 7fb900ff9700 1 -- 172.21.15.150:0/1513198811 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb8f4003c00 con 0x7fb904107e10 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.508+0000 7fb900ff9700 1 -- 172.21.15.150:0/1513198811 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8f4024800 con 0x7fb904107e10 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.508+0000 7fb909f12700 1 -- 172.21.15.150:0/1513198811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90419c050 con 0x7fb904107e10 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.509+0000 7fb900ff9700 1 -- 172.21.15.150:0/1513198811 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fb8f4024a30 con 0x7fb904107e10 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.509+0000 7fb900ff9700 1 --2- 172.21.15.150:0/1513198811 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fb8ec038700 0x7fb8ec03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.509+0000 7fb900ff9700 1 -- 172.21.15.150:0/1513198811 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb8f404d670 con 0x7fb904107e10 2024-05-10T03:57:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.510+0000 7fb902ffd700 1 --2- 172.21.15.150:0/1513198811 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fb8ec038700 0x7fb8ec03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:18.202 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.510+0000 7fb909f12700 1 -- 172.21.15.150:0/1513198811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb90410b510 con 0x7fb904107e10 2024-05-10T03:57:18.202 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.510+0000 7fb902ffd700 1 --2- 172.21.15.150:0/1513198811 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fb8ec038700 0x7fb8ec03ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8006fd0 tx=0x7fb8f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:18.202 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.516+0000 7fb900ff9700 1 -- 172.21.15.150:0/1513198811 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb8f4010790 con 0x7fb904107e10 2024-05-10T03:57:18.202 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.730+0000 7fb909f12700 1 -- 172.21.15.150:0/1513198811 --> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb904103a10 con 0x7fb8ec038700 2024-05-10T03:57:18.202 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.736+0000 7fb900ff9700 1 -- 172.21.15.150:0/1513198811 <== mgr.14120 v2:172.21.15.150:6800/999775795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb904103a10 con 0x7fb8ec038700 2024-05-10T03:57:18.202 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.738+0000 7fb909f12700 1 -- 172.21.15.150:0/1513198811 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fb8ec038700 msgr2=0x7fb8ec03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:18.202 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.738+0000 7fb909f12700 1 --2- 172.21.15.150:0/1513198811 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fb8ec038700 0x7fb8ec03ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8006fd0 tx=0x7fb8f8006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:18.202 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.738+0000 7fb909f12700 1 -- 172.21.15.150:0/1513198811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 msgr2=0x7fb90419b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:18.202 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.738+0000 7fb909f12700 1 --2- 172.21.15.150:0/1513198811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 0x7fb90419b470 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4011710 tx=0x7fb8f4011ad0 comp rx=0 tx=0).stop 2024-05-10T03:57:18.202 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.738+0000 7fb909f12700 1 -- 172.21.15.150:0/1513198811 shutdown_connections 2024-05-10T03:57:18.202 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.738+0000 7fb909f12700 1 --2- 172.21.15.150:0/1513198811 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fb8ec038700 0x7fb8ec03ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:18.202 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.738+0000 7fb909f12700 1 --2- 172.21.15.150:0/1513198811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb904107e10 0x7fb90419b470 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:18.202 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.738+0000 7fb909f12700 1 -- 172.21.15.150:0/1513198811 >> 172.21.15.150:0/1513198811 conn(0x7fb904101910 msgr2=0x7fb904103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:18.203 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.738+0000 7fb909f12700 1 -- 172.21.15.150:0/1513198811 shutdown_connections 2024-05-10T03:57:18.203 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:17.738+0000 7fb909f12700 1 -- 172.21.15.150:0/1513198811 wait complete. 2024-05-10T03:57:18.203 INFO:teuthology.orchestra.run.smithi150.stdout:Deploying unmanaged mgr service... 2024-05-10T03:57:19.205 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:18 smithi150 ceph-mon[42477]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:19.205 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:18 smithi150 ceph-mon[42477]: Saving service mon spec with placement count:5 2024-05-10T03:57:19.205 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:18 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:19.205 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:18 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:19.206 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:18 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:19.373 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-10T03:57:19.379 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.578+0000 7fdc4543e700 1 Processor -- start 2024-05-10T03:57:19.379 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.578+0000 7fdc4543e700 1 -- start start 2024-05-10T03:57:19.379 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.579+0000 7fdc4543e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 0x7fdc40107c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:19.379 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.579+0000 7fdc4543e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc40108260 con 0x7fdc40107840 2024-05-10T03:57:19.379 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.579+0000 7fdc3effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 0x7fdc40107c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:19.379 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.579+0000 7fdc3effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 0x7fdc40107c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32940/0 (socket says 172.21.15.150:32940) 2024-05-10T03:57:19.379 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.579+0000 7fdc3effd700 1 -- 172.21.15.150:0/2024823794 learned_addr learned my addr 172.21.15.150:0/2024823794 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:19.379 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.579+0000 7fdc3effd700 1 -- 172.21.15.150:0/2024823794 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc401083a0 con 0x7fdc40107840 2024-05-10T03:57:19.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.579+0000 7fdc3effd700 1 --2- 172.21.15.150:0/2024823794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 0x7fdc40107c50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdc300098a0 tx=0x7fdc30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90db55d29c53e67a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:19.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.580+0000 7fdc3dffb700 1 -- 172.21.15.150:0/2024823794 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc30004160 con 0x7fdc40107840 2024-05-10T03:57:19.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.580+0000 7fdc3dffb700 1 -- 172.21.15.150:0/2024823794 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc300042c0 con 0x7fdc40107840 2024-05-10T03:57:19.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.580+0000 7fdc3dffb700 1 -- 172.21.15.150:0/2024823794 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc300119b0 con 0x7fdc40107840 2024-05-10T03:57:19.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.581+0000 7fdc4543e700 1 -- 172.21.15.150:0/2024823794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 msgr2=0x7fdc40107c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:19.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.581+0000 7fdc4543e700 1 --2- 172.21.15.150:0/2024823794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 0x7fdc40107c50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdc300098a0 tx=0x7fdc30009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:19.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.581+0000 7fdc4543e700 1 -- 172.21.15.150:0/2024823794 shutdown_connections 2024-05-10T03:57:19.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.581+0000 7fdc4543e700 1 --2- 172.21.15.150:0/2024823794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 0x7fdc40107c50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:19.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.581+0000 7fdc4543e700 1 -- 172.21.15.150:0/2024823794 >> 172.21.15.150:0/2024823794 conn(0x7fdc40101030 msgr2=0x7fdc40103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:19.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.581+0000 7fdc4543e700 1 -- 172.21.15.150:0/2024823794 shutdown_connections 2024-05-10T03:57:19.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.581+0000 7fdc4543e700 1 -- 172.21.15.150:0/2024823794 wait complete. 2024-05-10T03:57:19.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.581+0000 7fdc4543e700 1 Processor -- start 2024-05-10T03:57:19.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.582+0000 7fdc4543e700 1 -- start start 2024-05-10T03:57:19.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.582+0000 7fdc4543e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 0x7fdc4019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:19.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.582+0000 7fdc4543e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4019dbe0 con 0x7fdc40107840 2024-05-10T03:57:19.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.582+0000 7fdc3effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 0x7fdc4019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:19.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.582+0000 7fdc3effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 0x7fdc4019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:32942/0 (socket says 172.21.15.150:32942) 2024-05-10T03:57:19.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.582+0000 7fdc3effd700 1 -- 172.21.15.150:0/1974524801 learned_addr learned my addr 172.21.15.150:0/1974524801 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:19.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.582+0000 7fdc3effd700 1 -- 172.21.15.150:0/1974524801 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc30009550 con 0x7fdc40107840 2024-05-10T03:57:19.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.582+0000 7fdc3effd700 1 --2- 172.21.15.150:0/1974524801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 0x7fdc4019d6a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc30011670 tx=0x7fdc30011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:19.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.583+0000 7fdc27fff700 1 -- 172.21.15.150:0/1974524801 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc30003af0 con 0x7fdc40107840 2024-05-10T03:57:19.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.583+0000 7fdc27fff700 1 -- 172.21.15.150:0/1974524801 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc30003c50 con 0x7fdc40107840 2024-05-10T03:57:19.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.583+0000 7fdc27fff700 1 -- 172.21.15.150:0/1974524801 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc30024820 con 0x7fdc40107840 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.583+0000 7fdc4543e700 1 -- 172.21.15.150:0/1974524801 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4019dde0 con 0x7fdc40107840 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.583+0000 7fdc4543e700 1 -- 172.21.15.150:0/1974524801 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4019e280 con 0x7fdc40107840 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.584+0000 7fdc27fff700 1 -- 172.21.15.150:0/1974524801 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fdc30024aa0 con 0x7fdc40107840 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.584+0000 7fdc4543e700 1 -- 172.21.15.150:0/1974524801 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4010d740 con 0x7fdc40107840 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.584+0000 7fdc27fff700 1 --2- 172.21.15.150:0/1974524801 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fdc28038700 0x7fdc2803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.584+0000 7fdc27fff700 1 -- 172.21.15.150:0/1974524801 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdc3004d860 con 0x7fdc40107840 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.584+0000 7fdc3e7fc700 1 --2- 172.21.15.150:0/1974524801 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fdc28038700 0x7fdc2803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.584+0000 7fdc3e7fc700 1 --2- 172.21.15.150:0/1974524801 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fdc28038700 0x7fdc2803ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdc34006fd0 tx=0x7fdc34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.589+0000 7fdc27fff700 1 -- 172.21.15.150:0/1974524801 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdc30010380 con 0x7fdc40107840 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.827+0000 7fdc4543e700 1 -- 172.21.15.150:0/1974524801 --> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fdc401a0a80 con 0x7fdc28038700 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.833+0000 7fdc27fff700 1 -- 172.21.15.150:0/1974524801 <== mgr.14120 v2:172.21.15.150:6800/999775795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fdc401a0a80 con 0x7fdc28038700 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.835+0000 7fdc4543e700 1 -- 172.21.15.150:0/1974524801 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fdc28038700 msgr2=0x7fdc2803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.835+0000 7fdc4543e700 1 --2- 172.21.15.150:0/1974524801 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fdc28038700 0x7fdc2803ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdc34006fd0 tx=0x7fdc34006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:19.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.835+0000 7fdc4543e700 1 -- 172.21.15.150:0/1974524801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 msgr2=0x7fdc4019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:19.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.835+0000 7fdc4543e700 1 --2- 172.21.15.150:0/1974524801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 0x7fdc4019d6a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc30011670 tx=0x7fdc30011a30 comp rx=0 tx=0).stop 2024-05-10T03:57:19.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.835+0000 7fdc4543e700 1 -- 172.21.15.150:0/1974524801 shutdown_connections 2024-05-10T03:57:19.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.835+0000 7fdc4543e700 1 --2- 172.21.15.150:0/1974524801 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7fdc28038700 0x7fdc2803ac10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:19.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.835+0000 7fdc4543e700 1 --2- 172.21.15.150:0/1974524801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc40107840 0x7fdc4019d6a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:19.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.835+0000 7fdc4543e700 1 -- 172.21.15.150:0/1974524801 >> 172.21.15.150:0/1974524801 conn(0x7fdc40101030 msgr2=0x7fdc40102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:19.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.836+0000 7fdc4543e700 1 -- 172.21.15.150:0/1974524801 shutdown_connections 2024-05-10T03:57:19.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:18.836+0000 7fdc4543e700 1 -- 172.21.15.150:0/1974524801 wait complete. 2024-05-10T03:57:20.138 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:19 smithi150 ceph-mon[42477]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:20.139 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:19 smithi150 ceph-mon[42477]: Saving service mgr spec with placement count:2 2024-05-10T03:57:20.139 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:19 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:20.344 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.742+0000 7f654f8f6700 1 Processor -- start 2024-05-10T03:57:20.344 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.742+0000 7f654f8f6700 1 -- start start 2024-05-10T03:57:20.344 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.742+0000 7f654f8f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 0x7f6548107bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:20.344 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.742+0000 7f654f8f6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65481081c0 con 0x7f65481077a0 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.743+0000 7f654d692700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 0x7f6548107bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.743+0000 7f654d692700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 0x7f6548107bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58322/0 (socket says 172.21.15.150:58322) 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.743+0000 7f654d692700 1 -- 172.21.15.150:0/177115791 learned_addr learned my addr 172.21.15.150:0/177115791 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.743+0000 7f654d692700 1 -- 172.21.15.150:0/177115791 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65481063d0 con 0x7f65481077a0 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.743+0000 7f654d692700 1 --2- 172.21.15.150:0/177115791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 0x7f6548107bb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f65440098a0 tx=0x7f6544009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15a92931d325a9a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.744+0000 7f653ffff700 1 -- 172.21.15.150:0/177115791 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6544004160 con 0x7f65481077a0 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.744+0000 7f653ffff700 1 -- 172.21.15.150:0/177115791 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65440042c0 con 0x7f65481077a0 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.744+0000 7f653ffff700 1 -- 172.21.15.150:0/177115791 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65440119b0 con 0x7f65481077a0 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.745+0000 7f654f8f6700 1 -- 172.21.15.150:0/177115791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 msgr2=0x7f6548107bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.745+0000 7f654f8f6700 1 --2- 172.21.15.150:0/177115791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 0x7f6548107bb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f65440098a0 tx=0x7f6544009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.745+0000 7f654f8f6700 1 -- 172.21.15.150:0/177115791 shutdown_connections 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.745+0000 7f654f8f6700 1 --2- 172.21.15.150:0/177115791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 0x7f6548107bb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.745+0000 7f654f8f6700 1 -- 172.21.15.150:0/177115791 >> 172.21.15.150:0/177115791 conn(0x7f6548100ff0 msgr2=0x7f6548103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.745+0000 7f654f8f6700 1 -- 172.21.15.150:0/177115791 shutdown_connections 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.745+0000 7f654f8f6700 1 -- 172.21.15.150:0/177115791 wait complete. 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.746+0000 7f654f8f6700 1 Processor -- start 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.746+0000 7f654f8f6700 1 -- start start 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.746+0000 7f654f8f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 0x7f654819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.746+0000 7f654f8f6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654819a900 con 0x7f65481077a0 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.746+0000 7f654d692700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 0x7f654819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.746+0000 7f654d692700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 0x7f654819a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58328/0 (socket says 172.21.15.150:58328) 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.746+0000 7f654d692700 1 -- 172.21.15.150:0/3749310298 learned_addr learned my addr 172.21.15.150:0/3749310298 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.746+0000 7f654d692700 1 -- 172.21.15.150:0/3749310298 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6544009550 con 0x7f65481077a0 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.747+0000 7f654d692700 1 --2- 172.21.15.150:0/3749310298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 0x7f654819a3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6544003e60 tx=0x7f6544011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.747+0000 7f653e7fc700 1 -- 172.21.15.150:0/3749310298 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65440038b0 con 0x7f65481077a0 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.747+0000 7f654f8f6700 1 -- 172.21.15.150:0/3749310298 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654819ab00 con 0x7f65481077a0 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.747+0000 7f653e7fc700 1 -- 172.21.15.150:0/3749310298 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6544003a10 con 0x7f65481077a0 2024-05-10T03:57:20.346 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.747+0000 7f653e7fc700 1 -- 172.21.15.150:0/3749310298 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6544019530 con 0x7f65481077a0 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.747+0000 7f654f8f6700 1 -- 172.21.15.150:0/3749310298 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654819afa0 con 0x7f65481077a0 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.748+0000 7f653e7fc700 1 -- 172.21.15.150:0/3749310298 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f6544019830 con 0x7f65481077a0 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.748+0000 7f653e7fc700 1 --2- 172.21.15.150:0/3749310298 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f65340386b0 0x7f653403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.748+0000 7f653e7fc700 1 -- 172.21.15.150:0/3749310298 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f654404d870 con 0x7f65481077a0 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.748+0000 7f654ce91700 1 --2- 172.21.15.150:0/3749310298 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f65340386b0 0x7f653403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.748+0000 7f654f8f6700 1 -- 172.21.15.150:0/3749310298 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654810a610 con 0x7f65481077a0 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.748+0000 7f654ce91700 1 --2- 172.21.15.150:0/3749310298 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f65340386b0 0x7f653403abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6538006fd0 tx=0x7f6538006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.753+0000 7f653e7fc700 1 -- 172.21.15.150:0/3749310298 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6544019b30 con 0x7f65481077a0 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.971+0000 7f654f8f6700 1 -- 172.21.15.150:0/3749310298 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f654819d8e0 con 0x7f65481077a0 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.976+0000 7f653e7fc700 1 -- 172.21.15.150:0/3749310298 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f6544018350 con 0x7f65481077a0 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.979+0000 7f654f8f6700 1 -- 172.21.15.150:0/3749310298 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f65340386b0 msgr2=0x7f653403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.979+0000 7f654f8f6700 1 --2- 172.21.15.150:0/3749310298 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f65340386b0 0x7f653403abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6538006fd0 tx=0x7f6538006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.979+0000 7f654f8f6700 1 -- 172.21.15.150:0/3749310298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 msgr2=0x7f654819a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.979+0000 7f654f8f6700 1 --2- 172.21.15.150:0/3749310298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 0x7f654819a3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6544003e60 tx=0x7f6544011a30 comp rx=0 tx=0).stop 2024-05-10T03:57:20.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.979+0000 7f654f8f6700 1 -- 172.21.15.150:0/3749310298 shutdown_connections 2024-05-10T03:57:20.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.979+0000 7f654f8f6700 1 --2- 172.21.15.150:0/3749310298 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f65340386b0 0x7f653403abc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:20.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.979+0000 7f654f8f6700 1 --2- 172.21.15.150:0/3749310298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65481077a0 0x7f654819a3c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:20.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.979+0000 7f654f8f6700 1 -- 172.21.15.150:0/3749310298 >> 172.21.15.150:0/3749310298 conn(0x7f6548100ff0 msgr2=0x7f65481017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:20.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.979+0000 7f654f8f6700 1 -- 172.21.15.150:0/3749310298 shutdown_connections 2024-05-10T03:57:20.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:19.979+0000 7f654f8f6700 1 -- 172.21.15.150:0/3749310298 wait complete. 2024-05-10T03:57:21.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:20 smithi150 ceph-mon[42477]: from='mgr.14120 172.21.15.150:0/720668380' entity='mgr.x' 2024-05-10T03:57:21.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:20 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3749310298' entity='client.admin' 2024-05-10T03:57:21.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.825+0000 7f084d1a1700 1 Processor -- start 2024-05-10T03:57:21.387 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.825+0000 7f084d1a1700 1 -- start start 2024-05-10T03:57:21.387 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.825+0000 7f084d1a1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 0x7f084810a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:21.387 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.825+0000 7f084d1a1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084807adc0 con 0x7f0848107e40 2024-05-10T03:57:21.387 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.825+0000 7f0846d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 0x7f084810a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:21.387 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.825+0000 7f0846d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 0x7f084810a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58332/0 (socket says 172.21.15.150:58332) 2024-05-10T03:57:21.387 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.825+0000 7f0846d9d700 1 -- 172.21.15.150:0/1387606063 learned_addr learned my addr 172.21.15.150:0/1387606063 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:21.387 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.825+0000 7f0846d9d700 1 -- 172.21.15.150:0/1387606063 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084810a7a0 con 0x7f0848107e40 2024-05-10T03:57:21.387 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.826+0000 7f0846d9d700 1 --2- 172.21.15.150:0/1387606063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 0x7f084810a260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08380098a0 tx=0x7f0838009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f485a87913e250e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:21.388 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.826+0000 7f0845d9b700 1 -- 172.21.15.150:0/1387606063 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0838004160 con 0x7f0848107e40 2024-05-10T03:57:21.388 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.826+0000 7f0845d9b700 1 -- 172.21.15.150:0/1387606063 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08380042c0 con 0x7f0848107e40 2024-05-10T03:57:21.388 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.826+0000 7f0845d9b700 1 -- 172.21.15.150:0/1387606063 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08380119b0 con 0x7f0848107e40 2024-05-10T03:57:21.388 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.827+0000 7f084d1a1700 1 -- 172.21.15.150:0/1387606063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 msgr2=0x7f084810a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:21.388 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.827+0000 7f084d1a1700 1 --2- 172.21.15.150:0/1387606063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 0x7f084810a260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08380098a0 tx=0x7f0838009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:21.388 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.827+0000 7f084d1a1700 1 -- 172.21.15.150:0/1387606063 shutdown_connections 2024-05-10T03:57:21.388 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.827+0000 7f084d1a1700 1 --2- 172.21.15.150:0/1387606063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 0x7f084810a260 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:21.388 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.827+0000 7f084d1a1700 1 -- 172.21.15.150:0/1387606063 >> 172.21.15.150:0/1387606063 conn(0x7f0848101820 msgr2=0x7f0848103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:21.388 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.827+0000 7f084d1a1700 1 -- 172.21.15.150:0/1387606063 shutdown_connections 2024-05-10T03:57:21.388 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.827+0000 7f084d1a1700 1 -- 172.21.15.150:0/1387606063 wait complete. 2024-05-10T03:57:21.388 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.828+0000 7f084d1a1700 1 Processor -- start 2024-05-10T03:57:21.388 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.828+0000 7f084d1a1700 1 -- start start 2024-05-10T03:57:21.388 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.828+0000 7f084d1a1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 0x7f084819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.828+0000 7f084d1a1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084819b9c0 con 0x7f0848107e40 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.828+0000 7f0846d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 0x7f084819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.828+0000 7f0846d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 0x7f084819b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58344/0 (socket says 172.21.15.150:58344) 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.828+0000 7f0846d9d700 1 -- 172.21.15.150:0/1625308729 learned_addr learned my addr 172.21.15.150:0/1625308729 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.829+0000 7f0846d9d700 1 -- 172.21.15.150:0/1625308729 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0838009550 con 0x7f0848107e40 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.829+0000 7f0846d9d700 1 --2- 172.21.15.150:0/1625308729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 0x7f084819b480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0838003e60 tx=0x7f0838011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.829+0000 7f082ffff700 1 -- 172.21.15.150:0/1625308729 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08380038b0 con 0x7f0848107e40 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.829+0000 7f082ffff700 1 -- 172.21.15.150:0/1625308729 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0838003a10 con 0x7f0848107e40 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.829+0000 7f082ffff700 1 -- 172.21.15.150:0/1625308729 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0838019530 con 0x7f0848107e40 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.829+0000 7f084d1a1700 1 -- 172.21.15.150:0/1625308729 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084819bbc0 con 0x7f0848107e40 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.829+0000 7f084d1a1700 1 -- 172.21.15.150:0/1625308729 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084819c060 con 0x7f0848107e40 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.830+0000 7f082ffff700 1 -- 172.21.15.150:0/1625308729 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f08380197f0 con 0x7f0848107e40 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.830+0000 7f084d1a1700 1 -- 172.21.15.150:0/1625308729 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084810b520 con 0x7f0848107e40 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.830+0000 7f082ffff700 1 --2- 172.21.15.150:0/1625308729 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f08300386b0 0x7f083003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:21.389 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.831+0000 7f082ffff700 1 -- 172.21.15.150:0/1625308729 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f083804d820 con 0x7f0848107e40 2024-05-10T03:57:21.390 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.832+0000 7f084659c700 1 --2- 172.21.15.150:0/1625308729 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f08300386b0 0x7f083003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:21.390 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.832+0000 7f084659c700 1 --2- 172.21.15.150:0/1625308729 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f08300386b0 0x7f083003abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f083c006fd0 tx=0x7f083c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:21.390 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:20.837+0000 7f082ffff700 1 -- 172.21.15.150:0/1625308729 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f083802b340 con 0x7f0848107e40 2024-05-10T03:57:21.390 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.037+0000 7f084d1a1700 1 -- 172.21.15.150:0/1625308729 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f084819e860 con 0x7f0848107e40 2024-05-10T03:57:21.390 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.042+0000 7f082ffff700 1 -- 172.21.15.150:0/1625308729 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f083804d190 con 0x7f0848107e40 2024-05-10T03:57:21.390 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.044+0000 7f084d1a1700 1 -- 172.21.15.150:0/1625308729 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f08300386b0 msgr2=0x7f083003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:21.390 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.044+0000 7f084d1a1700 1 --2- 172.21.15.150:0/1625308729 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f08300386b0 0x7f083003abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f083c006fd0 tx=0x7f083c006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:21.390 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.044+0000 7f084d1a1700 1 -- 172.21.15.150:0/1625308729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 msgr2=0x7f084819b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:21.390 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.044+0000 7f084d1a1700 1 --2- 172.21.15.150:0/1625308729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 0x7f084819b480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0838003e60 tx=0x7f0838011a30 comp rx=0 tx=0).stop 2024-05-10T03:57:21.390 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.044+0000 7f084d1a1700 1 -- 172.21.15.150:0/1625308729 shutdown_connections 2024-05-10T03:57:21.390 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.044+0000 7f084d1a1700 1 --2- 172.21.15.150:0/1625308729 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f08300386b0 0x7f083003abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:21.390 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.044+0000 7f084d1a1700 1 --2- 172.21.15.150:0/1625308729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0848107e40 0x7f084819b480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:21.390 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.044+0000 7f084d1a1700 1 -- 172.21.15.150:0/1625308729 >> 172.21.15.150:0/1625308729 conn(0x7f0848101820 msgr2=0x7f0848102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:21.391 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.044+0000 7f084d1a1700 1 -- 172.21.15.150:0/1625308729 shutdown_connections 2024-05-10T03:57:21.391 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.044+0000 7f084d1a1700 1 -- 172.21.15.150:0/1625308729 wait complete. 2024-05-10T03:57:21.391 INFO:teuthology.orchestra.run.smithi150.stdout:Enabling the dashboard module... 2024-05-10T03:57:22.367 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:22 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1625308729' entity='client.admin' 2024-05-10T03:57:22.367 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:22 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1471872359' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-10T03:57:22.367 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:22 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: ignoring --setuser ceph since I am not root 2024-05-10T03:57:22.367 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:22 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: ignoring --setgroup ceph since I am not root 2024-05-10T03:57:22.436 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.730+0000 7f7b89532700 1 Processor -- start 2024-05-10T03:57:22.436 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.730+0000 7f7b89532700 1 -- start start 2024-05-10T03:57:22.436 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.730+0000 7f7b89532700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 0x7f7b84107be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:22.436 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.730+0000 7f7b89532700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b841081f0 con 0x7f7b841077d0 2024-05-10T03:57:22.436 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.730+0000 7f7b82ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 0x7f7b84107be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:22.436 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.730+0000 7f7b82ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 0x7f7b84107be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58356/0 (socket says 172.21.15.150:58356) 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.730+0000 7f7b82ffd700 1 -- 172.21.15.150:0/1630709467 learned_addr learned my addr 172.21.15.150:0/1630709467 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.730+0000 7f7b82ffd700 1 -- 172.21.15.150:0/1630709467 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b84106400 con 0x7f7b841077d0 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.731+0000 7f7b82ffd700 1 --2- 172.21.15.150:0/1630709467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 0x7f7b84107be0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7b740098a0 tx=0x7f7b74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b663caa5c9a7f32 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.731+0000 7f7b81ffb700 1 -- 172.21.15.150:0/1630709467 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b74004160 con 0x7f7b841077d0 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.731+0000 7f7b81ffb700 1 -- 172.21.15.150:0/1630709467 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b740042c0 con 0x7f7b841077d0 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.731+0000 7f7b81ffb700 1 -- 172.21.15.150:0/1630709467 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b740119b0 con 0x7f7b841077d0 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.731+0000 7f7b89532700 1 -- 172.21.15.150:0/1630709467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 msgr2=0x7f7b84107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.731+0000 7f7b89532700 1 --2- 172.21.15.150:0/1630709467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 0x7f7b84107be0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7b740098a0 tx=0x7f7b74009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.732+0000 7f7b89532700 1 -- 172.21.15.150:0/1630709467 shutdown_connections 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.732+0000 7f7b89532700 1 --2- 172.21.15.150:0/1630709467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 0x7f7b84107be0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.732+0000 7f7b89532700 1 -- 172.21.15.150:0/1630709467 >> 172.21.15.150:0/1630709467 conn(0x7f7b84101080 msgr2=0x7f7b841034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.732+0000 7f7b89532700 1 -- 172.21.15.150:0/1630709467 shutdown_connections 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.732+0000 7f7b89532700 1 -- 172.21.15.150:0/1630709467 wait complete. 2024-05-10T03:57:22.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.732+0000 7f7b89532700 1 Processor -- start 2024-05-10T03:57:22.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.732+0000 7f7b89532700 1 -- start start 2024-05-10T03:57:22.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.733+0000 7f7b89532700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 0x7f7b8419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:22.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.733+0000 7f7b89532700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8419b9e0 con 0x7f7b841077d0 2024-05-10T03:57:22.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.733+0000 7f7b82ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 0x7f7b8419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:22.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.733+0000 7f7b82ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 0x7f7b8419b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58370/0 (socket says 172.21.15.150:58370) 2024-05-10T03:57:22.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.733+0000 7f7b82ffd700 1 -- 172.21.15.150:0/1471872359 learned_addr learned my addr 172.21.15.150:0/1471872359 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:22.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.733+0000 7f7b82ffd700 1 -- 172.21.15.150:0/1471872359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b74009550 con 0x7f7b841077d0 2024-05-10T03:57:22.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.733+0000 7f7b82ffd700 1 --2- 172.21.15.150:0/1471872359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 0x7f7b8419b4a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7b74011670 tx=0x7f7b74011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:22.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.733+0000 7f7b6bfff700 1 -- 172.21.15.150:0/1471872359 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b74003af0 con 0x7f7b841077d0 2024-05-10T03:57:22.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.733+0000 7f7b6bfff700 1 -- 172.21.15.150:0/1471872359 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b74003c50 con 0x7f7b841077d0 2024-05-10T03:57:22.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.733+0000 7f7b89532700 1 -- 172.21.15.150:0/1471872359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b8419bbe0 con 0x7f7b841077d0 2024-05-10T03:57:22.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.733+0000 7f7b6bfff700 1 -- 172.21.15.150:0/1471872359 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b74024820 con 0x7f7b841077d0 2024-05-10T03:57:22.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.733+0000 7f7b89532700 1 -- 172.21.15.150:0/1471872359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8419c080 con 0x7f7b841077d0 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.734+0000 7f7b6bfff700 1 -- 172.21.15.150:0/1471872359 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f7b74024aa0 con 0x7f7b841077d0 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.734+0000 7f7b89532700 1 -- 172.21.15.150:0/1471872359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b84194b30 con 0x7f7b841077d0 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.734+0000 7f7b6bfff700 1 --2- 172.21.15.150:0/1471872359 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f7b6c038700 0x7f7b6c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.734+0000 7f7b6bfff700 1 -- 172.21.15.150:0/1471872359 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7b7404c930 con 0x7f7b841077d0 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.734+0000 7f7b827fc700 1 --2- 172.21.15.150:0/1471872359 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f7b6c038700 0x7f7b6c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.735+0000 7f7b827fc700 1 --2- 172.21.15.150:0/1471872359 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f7b6c038700 0x7f7b6c03ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7b78006fd0 tx=0x7f7b78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.739+0000 7f7b6bfff700 1 -- 172.21.15.150:0/1471872359 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7b74010380 con 0x7f7b841077d0 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:21.985+0000 7f7b89532700 1 -- 172.21.15.150:0/1471872359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f7b8410a640 con 0x7f7b841077d0 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.043+0000 7f7b6bfff700 1 -- 172.21.15.150:0/1471872359 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f7b74024d50 con 0x7f7b841077d0 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.045+0000 7f7b6bfff700 1 -- 172.21.15.150:0/1471872359 <== mon.0 v2:172.21.15.150:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f7b7400f2e0 con 0x7f7b841077d0 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.047+0000 7f7b89532700 1 -- 172.21.15.150:0/1471872359 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f7b6c038700 msgr2=0x7f7b6c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.047+0000 7f7b89532700 1 --2- 172.21.15.150:0/1471872359 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f7b6c038700 0x7f7b6c03ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7b78006fd0 tx=0x7f7b78006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.047+0000 7f7b89532700 1 -- 172.21.15.150:0/1471872359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 msgr2=0x7f7b8419b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.047+0000 7f7b89532700 1 --2- 172.21.15.150:0/1471872359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 0x7f7b8419b4a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7b74011670 tx=0x7f7b74011a30 comp rx=0 tx=0).stop 2024-05-10T03:57:22.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.047+0000 7f7b89532700 1 -- 172.21.15.150:0/1471872359 shutdown_connections 2024-05-10T03:57:22.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.047+0000 7f7b89532700 1 --2- 172.21.15.150:0/1471872359 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f7b6c038700 0x7f7b6c03ac10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:22.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.047+0000 7f7b89532700 1 --2- 172.21.15.150:0/1471872359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b841077d0 0x7f7b8419b4a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:22.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.047+0000 7f7b89532700 1 -- 172.21.15.150:0/1471872359 >> 172.21.15.150:0/1471872359 conn(0x7f7b84101080 msgr2=0x7f7b84191c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:22.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.048+0000 7f7b89532700 1 -- 172.21.15.150:0/1471872359 shutdown_connections 2024-05-10T03:57:22.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.048+0000 7f7b89532700 1 -- 172.21.15.150:0/1471872359 wait complete. 2024-05-10T03:57:22.619 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:22 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:22.536+0000 7f4e69fc8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T03:57:22.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:22 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:22.641+0000 7f4e69fc8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T03:57:22.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:22 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:22.733+0000 7f4e69fc8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T03:57:22.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:22 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:22.825+0000 7f4e69fc8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T03:57:23.338 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:23 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1471872359' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-10T03:57:23.338 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:23 smithi150 ceph-mon[42477]: mgrmap e9: x(active, since 13s) 2024-05-10T03:57:23.470 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T03:57:23.470 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-10T03:57:23.470 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T03:57:23.470 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-10T03:57:23.470 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-10T03:57:23.470 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T03:57:23.470 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.822+0000 7f6ca8134700 1 Processor -- start 2024-05-10T03:57:23.471 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.822+0000 7f6ca8134700 1 -- start start 2024-05-10T03:57:23.471 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.822+0000 7f6ca8134700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 0x7f6ca0107be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:23.471 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.822+0000 7f6ca8134700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01081d0 con 0x7f6ca01077d0 2024-05-10T03:57:23.471 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.823+0000 7f6ca5ed0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 0x7f6ca0107be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:23.471 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.823+0000 7f6ca5ed0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 0x7f6ca0107be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58380/0 (socket says 172.21.15.150:58380) 2024-05-10T03:57:23.471 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.823+0000 7f6ca5ed0700 1 -- 172.21.15.150:0/1128322868 learned_addr learned my addr 172.21.15.150:0/1128322868 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:23.471 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.823+0000 7f6ca5ed0700 1 -- 172.21.15.150:0/1128322868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca0106450 con 0x7f6ca01077d0 2024-05-10T03:57:23.471 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.823+0000 7f6ca5ed0700 1 --2- 172.21.15.150:0/1128322868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 0x7f6ca0107be0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0098a0 tx=0x7f6c9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb105a44f0ef3a9c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:23.471 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.823+0000 7f6ca4ece700 1 -- 172.21.15.150:0/1128322868 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c9c004160 con 0x7f6ca01077d0 2024-05-10T03:57:23.471 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.823+0000 7f6ca4ece700 1 -- 172.21.15.150:0/1128322868 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c9c0042c0 con 0x7f6ca01077d0 2024-05-10T03:57:23.471 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.823+0000 7f6ca4ece700 1 -- 172.21.15.150:0/1128322868 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c9c0119b0 con 0x7f6ca01077d0 2024-05-10T03:57:23.471 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.824+0000 7f6ca8134700 1 -- 172.21.15.150:0/1128322868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 msgr2=0x7f6ca0107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:23.471 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.824+0000 7f6ca8134700 1 --2- 172.21.15.150:0/1128322868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 0x7f6ca0107be0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0098a0 tx=0x7f6c9c009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:23.472 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.824+0000 7f6ca8134700 1 -- 172.21.15.150:0/1128322868 shutdown_connections 2024-05-10T03:57:23.472 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.824+0000 7f6ca8134700 1 --2- 172.21.15.150:0/1128322868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 0x7f6ca0107be0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:23.472 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.824+0000 7f6ca8134700 1 -- 172.21.15.150:0/1128322868 >> 172.21.15.150:0/1128322868 conn(0x7f6ca0100ff0 msgr2=0x7f6ca0103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:23.472 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.824+0000 7f6ca8134700 1 -- 172.21.15.150:0/1128322868 shutdown_connections 2024-05-10T03:57:23.472 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.824+0000 7f6ca8134700 1 -- 172.21.15.150:0/1128322868 wait complete. 2024-05-10T03:57:23.472 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.825+0000 7f6ca8134700 1 Processor -- start 2024-05-10T03:57:23.472 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.825+0000 7f6ca8134700 1 -- start start 2024-05-10T03:57:23.472 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.825+0000 7f6ca8134700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 0x7f6ca019b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:23.472 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.825+0000 7f6ca8134700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca019bab0 con 0x7f6ca01077d0 2024-05-10T03:57:23.472 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.825+0000 7f6ca5ed0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 0x7f6ca019b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:23.472 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.825+0000 7f6ca5ed0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 0x7f6ca019b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58396/0 (socket says 172.21.15.150:58396) 2024-05-10T03:57:23.472 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.825+0000 7f6ca5ed0700 1 -- 172.21.15.150:0/3086228937 learned_addr learned my addr 172.21.15.150:0/3086228937 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:23.472 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.826+0000 7f6ca5ed0700 1 -- 172.21.15.150:0/3086228937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c009550 con 0x7f6ca01077d0 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.826+0000 7f6ca5ed0700 1 --2- 172.21.15.150:0/3086228937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 0x7f6ca019b570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c011980 tx=0x7f6c9c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.826+0000 7f6c96ffd700 1 -- 172.21.15.150:0/3086228937 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c9c003aa0 con 0x7f6ca01077d0 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.826+0000 7f6ca8134700 1 -- 172.21.15.150:0/3086228937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca019bcb0 con 0x7f6ca01077d0 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.826+0000 7f6c96ffd700 1 -- 172.21.15.150:0/3086228937 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c9c003c00 con 0x7f6ca01077d0 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.826+0000 7f6c96ffd700 1 -- 172.21.15.150:0/3086228937 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c9c0196c0 con 0x7f6ca01077d0 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.826+0000 7f6ca8134700 1 -- 172.21.15.150:0/3086228937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca019c150 con 0x7f6ca01077d0 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.827+0000 7f6c96ffd700 1 -- 172.21.15.150:0/3086228937 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f6c9c019980 con 0x7f6ca01077d0 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.827+0000 7f6c96ffd700 1 --2- 172.21.15.150:0/3086228937 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f6c8c038750 0x7f6c8c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.827+0000 7f6c96ffd700 1 -- 172.21.15.150:0/3086228937 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6c9c04ca00 con 0x7f6ca01077d0 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.827+0000 7f6ca56cf700 1 -- 172.21.15.150:0/3086228937 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f6c8c038750 msgr2=0x7f6c8c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/999775795 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.827+0000 7f6ca56cf700 1 --2- 172.21.15.150:0/3086228937 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f6c8c038750 0x7f6c8c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.828+0000 7f6ca8134700 1 -- 172.21.15.150:0/3086228937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca0194c10 con 0x7f6ca01077d0 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:22.833+0000 7f6c96ffd700 1 -- 172.21.15.150:0/3086228937 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6c9c018db0 con 0x7f6ca01077d0 2024-05-10T03:57:23.473 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.028+0000 7f6ca56cf700 1 -- 172.21.15.150:0/3086228937 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f6c8c038750 msgr2=0x7f6c8c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/999775795 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.028+0000 7f6ca56cf700 1 --2- 172.21.15.150:0/3086228937 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f6c8c038750 0x7f6c8c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.088+0000 7f6ca8134700 1 -- 172.21.15.150:0/3086228937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f6ca010a660 con 0x7f6ca01077d0 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.088+0000 7f6c96ffd700 1 -- 172.21.15.150:0/3086228937 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f6c9c018350 con 0x7f6ca01077d0 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.090+0000 7f6ca8134700 1 -- 172.21.15.150:0/3086228937 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f6c8c038750 msgr2=0x7f6c8c03ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.090+0000 7f6ca8134700 1 --2- 172.21.15.150:0/3086228937 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f6c8c038750 0x7f6c8c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.090+0000 7f6ca8134700 1 -- 172.21.15.150:0/3086228937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 msgr2=0x7f6ca019b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.090+0000 7f6ca8134700 1 --2- 172.21.15.150:0/3086228937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 0x7f6ca019b570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c011980 tx=0x7f6c9c011a60 comp rx=0 tx=0).stop 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.090+0000 7f6ca8134700 1 -- 172.21.15.150:0/3086228937 shutdown_connections 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.090+0000 7f6ca8134700 1 --2- 172.21.15.150:0/3086228937 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f6c8c038750 0x7f6c8c03ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.090+0000 7f6ca8134700 1 --2- 172.21.15.150:0/3086228937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca01077d0 0x7f6ca019b570 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.090+0000 7f6ca8134700 1 -- 172.21.15.150:0/3086228937 >> 172.21.15.150:0/3086228937 conn(0x7f6ca0100ff0 msgr2=0x7f6ca0191d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.090+0000 7f6ca8134700 1 -- 172.21.15.150:0/3086228937 shutdown_connections 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.091+0000 7f6ca8134700 1 -- 172.21.15.150:0/3086228937 wait complete. 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for the mgr to restart... 2024-05-10T03:57:23.474 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for mgr epoch 9... 2024-05-10T03:57:23.590 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:23 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:23.483+0000 7f4e69fc8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:57:23.855 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:23 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:23.756+0000 7f4e69fc8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T03:57:24.129 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:24 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3086228937' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-10T03:57:24.130 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:23 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:23.852+0000 7f4e69fc8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T03:57:24.130 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:23 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:23.964+0000 7f4e69fc8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T03:57:24.464 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:24 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:24.126+0000 7f4e69fc8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T03:57:24.464 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:24 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:24.255+0000 7f4e69fc8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T03:57:24.464 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:24 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:24.342+0000 7f4e69fc8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T03:57:25.118 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:24 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:24.721+0000 7f4e69fc8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T03:57:25.411 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:25 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:25.114+0000 7f4e69fc8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T03:57:25.412 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:25 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:25.314+0000 7f4e69fc8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T03:57:25.714 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:25 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:25.408+0000 7f4e69fc8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T03:57:26.464 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:26 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:26.185+0000 7f4e69fc8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T03:57:26.464 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:26 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:26.283+0000 7f4e69fc8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T03:57:26.464 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:26 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:26.434+0000 7f4e69fc8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T03:57:27.464 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:27 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:27.078+0000 7f4e69fc8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T03:57:27.464 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:27 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:27.265+0000 7f4e69fc8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T03:57:27.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:27 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:27.463+0000 7f4e69fc8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T03:57:28.711 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:28 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:28.408+0000 7f4e69fc8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T03:57:28.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:28 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:28.707+0000 7f4e69fc8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:57:28.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:28 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:28.799+0000 7f4e69fc8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T03:57:29.464 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 03:57:29 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T03:57:28.999+0000 7f4e69fc8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T03:57:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:29 smithi150 ceph-mon[42477]: Active manager daemon x restarted 2024-05-10T03:57:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:29 smithi150 ceph-mon[42477]: Activating manager daemon x 2024-05-10T03:57:29.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:29 smithi150 ceph-mon[42477]: osdmap e3: 0 total, 0 up, 0 in 2024-05-10T03:57:29.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:29 smithi150 ceph-mon[42477]: mgrmap e10: x(active, starting, since 0.0039315s) 2024-05-10T03:57:29.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:57:29.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T03:57:29.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T03:57:29.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T03:57:29.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T03:57:29.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:29 smithi150 ceph-mon[42477]: Manager daemon x is now available 2024-05-10T03:57:29.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:57:30.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T03:57:30.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T03:57:30.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:30 smithi150 ceph-mon[42477]: mgrmap e11: x(active, since 1.00638s) 2024-05-10T03:57:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T03:57:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-10T03:57:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-10T03:57:31.038 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T03:57:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.832+0000 7f8ed1acc700 1 Processor -- start 2024-05-10T03:57:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.832+0000 7f8ed1acc700 1 -- start start 2024-05-10T03:57:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.832+0000 7f8ed1acc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 0x7f8ecc106600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.832+0000 7f8ed1acc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc106bf0 con 0x7f8ecc1061f0 2024-05-10T03:57:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.832+0000 7f8ecb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 0x7f8ecc106600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.832+0000 7f8ecb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 0x7f8ecc106600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58400/0 (socket says 172.21.15.150:58400) 2024-05-10T03:57:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.832+0000 7f8ecb7fe700 1 -- 172.21.15.150:0/1589944402 learned_addr learned my addr 172.21.15.150:0/1589944402 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.833+0000 7f8ecb7fe700 1 -- 172.21.15.150:0/1589944402 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc1073c0 con 0x7f8ecc1061f0 2024-05-10T03:57:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.833+0000 7f8ecb7fe700 1 --2- 172.21.15.150:0/1589944402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 0x7f8ecc106600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0098a0 tx=0x7f8ebc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8cdb4d3ef28a77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.833+0000 7f8eca7fc700 1 -- 172.21.15.150:0/1589944402 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ebc004160 con 0x7f8ecc1061f0 2024-05-10T03:57:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.833+0000 7f8eca7fc700 1 -- 172.21.15.150:0/1589944402 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ebc0042c0 con 0x7f8ecc1061f0 2024-05-10T03:57:31.039 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.833+0000 7f8eca7fc700 1 -- 172.21.15.150:0/1589944402 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ebc0119b0 con 0x7f8ecc1061f0 2024-05-10T03:57:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.834+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1589944402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 msgr2=0x7f8ecc106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.834+0000 7f8ed1acc700 1 --2- 172.21.15.150:0/1589944402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 0x7f8ecc106600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0098a0 tx=0x7f8ebc009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.834+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1589944402 shutdown_connections 2024-05-10T03:57:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.834+0000 7f8ed1acc700 1 --2- 172.21.15.150:0/1589944402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 0x7f8ecc106600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.834+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1589944402 >> 172.21.15.150:0/1589944402 conn(0x7f8ecc101850 msgr2=0x7f8ecc103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.834+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1589944402 shutdown_connections 2024-05-10T03:57:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.834+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1589944402 wait complete. 2024-05-10T03:57:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.835+0000 7f8ed1acc700 1 Processor -- start 2024-05-10T03:57:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.835+0000 7f8ed1acc700 1 -- start start 2024-05-10T03:57:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.835+0000 7f8ed1acc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 0x7f8ecc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.835+0000 7f8ed1acc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc19a980 con 0x7f8ecc1061f0 2024-05-10T03:57:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.835+0000 7f8ecb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 0x7f8ecc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.835+0000 7f8ecb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 0x7f8ecc19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58406/0 (socket says 172.21.15.150:58406) 2024-05-10T03:57:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.835+0000 7f8ecb7fe700 1 -- 172.21.15.150:0/1313760700 learned_addr learned my addr 172.21.15.150:0/1313760700 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.836+0000 7f8ecb7fe700 1 -- 172.21.15.150:0/1313760700 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc009550 con 0x7f8ecc1061f0 2024-05-10T03:57:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.836+0000 7f8ecb7fe700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 0x7f8ecc19a440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc011980 tx=0x7f8ebc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.836+0000 7f8ec8ff9700 1 -- 172.21.15.150:0/1313760700 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ebc003aa0 con 0x7f8ecc1061f0 2024-05-10T03:57:31.041 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.836+0000 7f8ec8ff9700 1 -- 172.21.15.150:0/1313760700 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ebc003c00 con 0x7f8ecc1061f0 2024-05-10T03:57:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.836+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1313760700 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc19ab80 con 0x7f8ecc1061f0 2024-05-10T03:57:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.836+0000 7f8ec8ff9700 1 -- 172.21.15.150:0/1313760700 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ebc019670 con 0x7f8ecc1061f0 2024-05-10T03:57:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.836+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1313760700 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc19b020 con 0x7f8ecc1061f0 2024-05-10T03:57:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.837+0000 7f8ec8ff9700 1 -- 172.21.15.150:0/1313760700 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f8ebc019930 con 0x7f8ecc1061f0 2024-05-10T03:57:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.837+0000 7f8ec8ff9700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 0x7f8eb403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.837+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1313760700 --> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ecc10a5f0 con 0x7f8eb4038750 2024-05-10T03:57:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.837+0000 7f8ec8ff9700 1 -- 172.21.15.150:0/1313760700 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8ebc01a840 con 0x7f8ecc1061f0 2024-05-10T03:57:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.837+0000 7f8ecaffd700 1 -- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 msgr2=0x7f8eb403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/999775795 2024-05-10T03:57:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:23.837+0000 7f8ecaffd700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 0x7f8eb403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T03:57:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:24.038+0000 7f8ecaffd700 1 -- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 msgr2=0x7f8eb403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/999775795 2024-05-10T03:57:31.042 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:24.038+0000 7f8ecaffd700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 0x7f8eb403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T03:57:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:24.439+0000 7f8ecaffd700 1 -- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 msgr2=0x7f8eb403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/999775795 2024-05-10T03:57:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:24.439+0000 7f8ecaffd700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 0x7f8eb403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T03:57:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:25.240+0000 7f8ecaffd700 1 -- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 msgr2=0x7f8eb403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/999775795 2024-05-10T03:57:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:25.240+0000 7f8ecaffd700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 0x7f8eb403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T03:57:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:26.841+0000 7f8ecaffd700 1 -- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 msgr2=0x7f8eb403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/999775795 2024-05-10T03:57:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:26.841+0000 7f8ecaffd700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 0x7f8eb403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-10T03:57:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:29.014+0000 7f8ec8ff9700 1 -- 172.21.15.150:0/1313760700 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mgrmap(e 10) v1 ==== 45321+0+0 (secure 0 0 0) 0x7f8ebc00b900 con 0x7f8ecc1061f0 2024-05-10T03:57:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:29.014+0000 7f8ec8ff9700 1 -- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 msgr2=0x7f8eb403ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T03:57:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:29.015+0000 7f8ec8ff9700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 0x7f8eb403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.016+0000 7f8ec8ff9700 1 -- 172.21.15.150:0/1313760700 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7f8ebc04dc80 con 0x7f8ecc1061f0 2024-05-10T03:57:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.016+0000 7f8ec8ff9700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8ebc01a430 0x7f8eb403dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:31.043 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.016+0000 7f8ec8ff9700 1 -- 172.21.15.150:0/1313760700 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ecc10a5f0 con 0x7f8ebc01a430 2024-05-10T03:57:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.016+0000 7f8ecaffd700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8ebc01a430 0x7f8eb403dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.017+0000 7f8ecaffd700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8ebc01a430 0x7f8eb403dfd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0003a10 tx=0x7f8ec0009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.018+0000 7f8ec8ff9700 1 -- 172.21.15.150:0/1313760700 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f8ecc10a5f0 con 0x7f8ebc01a430 2024-05-10T03:57:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.024+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1313760700 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f8ecc10a760 con 0x7f8ebc01a430 2024-05-10T03:57:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.025+0000 7f8ec8ff9700 1 -- 172.21.15.150:0/1313760700 <== mgr.14152 v2:172.21.15.150:6800/1954703650 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f8ecc10a760 con 0x7f8ebc01a430 2024-05-10T03:57:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.025+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8ebc01a430 msgr2=0x7f8eb403dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.025+0000 7f8ed1acc700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8ebc01a430 0x7f8eb403dfd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0003a10 tx=0x7f8ec0009270 comp rx=0 tx=0).stop 2024-05-10T03:57:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.025+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 msgr2=0x7f8ecc19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.025+0000 7f8ed1acc700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 0x7f8ecc19a440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc011980 tx=0x7f8ebc011a60 comp rx=0 tx=0).stop 2024-05-10T03:57:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.025+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1313760700 shutdown_connections 2024-05-10T03:57:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.025+0000 7f8ed1acc700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8ebc01a430 0x7f8eb403dfd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.025+0000 7f8ed1acc700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc1061f0 0x7f8ecc19a440 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:31.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.025+0000 7f8ed1acc700 1 --2- 172.21.15.150:0/1313760700 >> [v2:172.21.15.150:6800/999775795,v1:172.21.15.150:6801/999775795] conn(0x7f8eb4038750 0x7f8eb403ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.025+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1313760700 >> 172.21.15.150:0/1313760700 conn(0x7f8ecc101850 msgr2=0x7f8ecc102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.025+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1313760700 shutdown_connections 2024-05-10T03:57:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:30.025+0000 7f8ed1acc700 1 -- 172.21.15.150:0/1313760700 wait complete. 2024-05-10T03:57:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:mgr epoch 9 is available 2024-05-10T03:57:31.045 INFO:teuthology.orchestra.run.smithi150.stdout:Generating a dashboard self-signed certificate... 2024-05-10T03:57:31.157 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:31 smithi150 ceph-mon[42477]: [10/May/2024:03:57:29] ENGINE Bus STARTING 2024-05-10T03:57:31.157 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:31 smithi150 ceph-mon[42477]: [10/May/2024:03:57:29] ENGINE Serving on http://172.21.15.150:8765 2024-05-10T03:57:31.157 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:31 smithi150 ceph-mon[42477]: [10/May/2024:03:57:29] ENGINE Serving on https://172.21.15.150:7150 2024-05-10T03:57:31.157 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:31 smithi150 ceph-mon[42477]: [10/May/2024:03:57:29] ENGINE Bus STARTED 2024-05-10T03:57:31.157 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:31 smithi150 ceph-mon[42477]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-10T03:57:31.157 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:31 smithi150 ceph-mon[42477]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-10T03:57:32.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-10T03:57:32.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.436+0000 7f79cae64700 1 Processor -- start 2024-05-10T03:57:32.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.436+0000 7f79cae64700 1 -- start start 2024-05-10T03:57:32.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.436+0000 7f79cae64700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 0x7f79c410a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:32.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.436+0000 7f79cae64700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c407ade0 con 0x7f79c4107e10 2024-05-10T03:57:32.102 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.436+0000 7f79c8c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 0x7f79c410a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:32.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.436+0000 7f79c8c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 0x7f79c410a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54876/0 (socket says 172.21.15.150:54876) 2024-05-10T03:57:32.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.436+0000 7f79c8c00700 1 -- 172.21.15.150:0/4255670337 learned_addr learned my addr 172.21.15.150:0/4255670337 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:32.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.437+0000 7f79c8c00700 1 -- 172.21.15.150:0/4255670337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c410a770 con 0x7f79c4107e10 2024-05-10T03:57:32.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.437+0000 7f79c8c00700 1 --2- 172.21.15.150:0/4255670337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 0x7f79c410a230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79b80098a0 tx=0x7f79b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f666456c0f1a2419 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:32.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.437+0000 7f79c37fe700 1 -- 172.21.15.150:0/4255670337 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79b8004160 con 0x7f79c4107e10 2024-05-10T03:57:32.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.437+0000 7f79c37fe700 1 -- 172.21.15.150:0/4255670337 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79b80042c0 con 0x7f79c4107e10 2024-05-10T03:57:32.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.437+0000 7f79c37fe700 1 -- 172.21.15.150:0/4255670337 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79b80119b0 con 0x7f79c4107e10 2024-05-10T03:57:32.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.438+0000 7f79cae64700 1 -- 172.21.15.150:0/4255670337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 msgr2=0x7f79c410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:32.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.438+0000 7f79cae64700 1 --2- 172.21.15.150:0/4255670337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 0x7f79c410a230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79b80098a0 tx=0x7f79b8009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:32.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.438+0000 7f79cae64700 1 -- 172.21.15.150:0/4255670337 shutdown_connections 2024-05-10T03:57:32.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.438+0000 7f79cae64700 1 --2- 172.21.15.150:0/4255670337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 0x7f79c410a230 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:32.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.438+0000 7f79cae64700 1 -- 172.21.15.150:0/4255670337 >> 172.21.15.150:0/4255670337 conn(0x7f79c4101850 msgr2=0x7f79c4103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:32.103 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.438+0000 7f79cae64700 1 -- 172.21.15.150:0/4255670337 shutdown_connections 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.438+0000 7f79cae64700 1 -- 172.21.15.150:0/4255670337 wait complete. 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.439+0000 7f79cae64700 1 Processor -- start 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.439+0000 7f79cae64700 1 -- start start 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.439+0000 7f79cae64700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 0x7f79c419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.439+0000 7f79cae64700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c419ba20 con 0x7f79c4107e10 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.439+0000 7f79c8c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 0x7f79c419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.439+0000 7f79c8c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 0x7f79c419b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54886/0 (socket says 172.21.15.150:54886) 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.439+0000 7f79c8c00700 1 -- 172.21.15.150:0/2697517339 learned_addr learned my addr 172.21.15.150:0/2697517339 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.439+0000 7f79c8c00700 1 -- 172.21.15.150:0/2697517339 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b8009550 con 0x7f79c4107e10 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.440+0000 7f79c8c00700 1 --2- 172.21.15.150:0/2697517339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 0x7f79c419b4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f79b8011670 tx=0x7f79b8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.440+0000 7f79c1ffb700 1 -- 172.21.15.150:0/2697517339 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79b8003af0 con 0x7f79c4107e10 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.440+0000 7f79c1ffb700 1 -- 172.21.15.150:0/2697517339 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79b8003c50 con 0x7f79c4107e10 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.440+0000 7f79cae64700 1 -- 172.21.15.150:0/2697517339 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c419bc20 con 0x7f79c4107e10 2024-05-10T03:57:32.104 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.440+0000 7f79c1ffb700 1 -- 172.21.15.150:0/2697517339 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79b8024730 con 0x7f79c4107e10 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.440+0000 7f79cae64700 1 -- 172.21.15.150:0/2697517339 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c419c0c0 con 0x7f79c4107e10 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.441+0000 7f79c1ffb700 1 -- 172.21.15.150:0/2697517339 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f79b8024970 con 0x7f79c4107e10 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.441+0000 7f79c1ffb700 1 --2- 172.21.15.150:0/2697517339 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f79b4038750 0x7f79b403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.441+0000 7f79c1ffb700 1 -- 172.21.15.150:0/2697517339 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f79b804d7a0 con 0x7f79c4107e10 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.441+0000 7f79cae64700 1 -- 172.21.15.150:0/2697517339 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c410b550 con 0x7f79c4107e10 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.441+0000 7f79c3fff700 1 --2- 172.21.15.150:0/2697517339 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f79b4038750 0x7f79b403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.442+0000 7f79c3fff700 1 --2- 172.21.15.150:0/2697517339 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f79b4038750 0x7f79b403ac60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79b0006fd0 tx=0x7f79b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.447+0000 7f79c1ffb700 1 -- 172.21.15.150:0/2697517339 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f79b8010380 con 0x7f79c4107e10 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.661+0000 7f79cae64700 1 -- 172.21.15.150:0/2697517339 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f79c41029a0 con 0x7f79b4038750 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.765+0000 7f79c1ffb700 1 -- 172.21.15.150:0/2697517339 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f79c41029a0 con 0x7f79b4038750 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.767+0000 7f79cae64700 1 -- 172.21.15.150:0/2697517339 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f79b4038750 msgr2=0x7f79b403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.767+0000 7f79cae64700 1 --2- 172.21.15.150:0/2697517339 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f79b4038750 0x7f79b403ac60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79b0006fd0 tx=0x7f79b0006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.767+0000 7f79cae64700 1 -- 172.21.15.150:0/2697517339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 msgr2=0x7f79c419b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.767+0000 7f79cae64700 1 --2- 172.21.15.150:0/2697517339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 0x7f79c419b4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f79b8011670 tx=0x7f79b8011a30 comp rx=0 tx=0).stop 2024-05-10T03:57:32.106 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.767+0000 7f79cae64700 1 -- 172.21.15.150:0/2697517339 shutdown_connections 2024-05-10T03:57:32.106 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.767+0000 7f79cae64700 1 --2- 172.21.15.150:0/2697517339 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f79b4038750 0x7f79b403ac60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:32.106 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.767+0000 7f79cae64700 1 --2- 172.21.15.150:0/2697517339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79c4107e10 0x7f79c419b4e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:32.106 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.767+0000 7f79cae64700 1 -- 172.21.15.150:0/2697517339 >> 172.21.15.150:0/2697517339 conn(0x7f79c4101850 msgr2=0x7f79c4102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:32.106 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.767+0000 7f79cae64700 1 -- 172.21.15.150:0/2697517339 shutdown_connections 2024-05-10T03:57:32.106 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:31.767+0000 7f79cae64700 1 -- 172.21.15.150:0/2697517339 wait complete. 2024-05-10T03:57:32.106 INFO:teuthology.orchestra.run.smithi150.stdout:Creating initial admin user... 2024-05-10T03:57:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:32 smithi150 ceph-mon[42477]: mgrmap e12: x(active, since 2s) 2024-05-10T03:57:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:33.183 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:33 smithi150 ceph-mon[42477]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:33.183 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:33.368 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$fgvG4NxJglsgYbfORRU6i.DaT.axU8PcnYYg3sccofFGelwTIOBqC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715313452, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-10T03:57:33.368 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.477+0000 7f04124ba700 1 Processor -- start 2024-05-10T03:57:33.368 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.477+0000 7f04124ba700 1 -- start start 2024-05-10T03:57:33.368 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.477+0000 7f04124ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 0x7f040c108d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:33.368 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.477+0000 7f04124ba700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c109310 con 0x7f040c108930 2024-05-10T03:57:33.369 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.477+0000 7f040bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 0x7f040c108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:33.369 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.477+0000 7f040bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 0x7f040c108d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54890/0 (socket says 172.21.15.150:54890) 2024-05-10T03:57:33.369 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.477+0000 7f040bfff700 1 -- 172.21.15.150:0/415414562 learned_addr learned my addr 172.21.15.150:0/415414562 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:33.369 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.477+0000 7f040bfff700 1 -- 172.21.15.150:0/415414562 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c109450 con 0x7f040c108930 2024-05-10T03:57:33.369 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.478+0000 7f040bfff700 1 --2- 172.21.15.150:0/415414562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 0x7f040c108d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0098a0 tx=0x7f03fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab96da8fcb77a81 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:33.369 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.478+0000 7f040affd700 1 -- 172.21.15.150:0/415414562 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03fc004160 con 0x7f040c108930 2024-05-10T03:57:33.369 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.478+0000 7f040affd700 1 -- 172.21.15.150:0/415414562 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03fc0042c0 con 0x7f040c108930 2024-05-10T03:57:33.369 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.478+0000 7f040affd700 1 -- 172.21.15.150:0/415414562 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03fc0119b0 con 0x7f040c108930 2024-05-10T03:57:33.369 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.478+0000 7f04124ba700 1 -- 172.21.15.150:0/415414562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 msgr2=0x7f040c108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:33.369 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.478+0000 7f04124ba700 1 --2- 172.21.15.150:0/415414562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 0x7f040c108d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0098a0 tx=0x7f03fc009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:33.369 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.479+0000 7f04124ba700 1 -- 172.21.15.150:0/415414562 shutdown_connections 2024-05-10T03:57:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.479+0000 7f04124ba700 1 --2- 172.21.15.150:0/415414562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 0x7f040c108d40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.479+0000 7f04124ba700 1 -- 172.21.15.150:0/415414562 >> 172.21.15.150:0/415414562 conn(0x7f040c100ff0 msgr2=0x7f040c103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.479+0000 7f04124ba700 1 -- 172.21.15.150:0/415414562 shutdown_connections 2024-05-10T03:57:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.479+0000 7f04124ba700 1 -- 172.21.15.150:0/415414562 wait complete. 2024-05-10T03:57:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.479+0000 7f04124ba700 1 Processor -- start 2024-05-10T03:57:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.479+0000 7f04124ba700 1 -- start start 2024-05-10T03:57:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.480+0000 7f04124ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 0x7f040c19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.480+0000 7f04124ba700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c19b9c0 con 0x7f040c108930 2024-05-10T03:57:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.480+0000 7f040bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 0x7f040c19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.480+0000 7f040bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 0x7f040c19b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54892/0 (socket says 172.21.15.150:54892) 2024-05-10T03:57:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.480+0000 7f040bfff700 1 -- 172.21.15.150:0/3072805744 learned_addr learned my addr 172.21.15.150:0/3072805744 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.480+0000 7f040bfff700 1 -- 172.21.15.150:0/3072805744 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc009550 con 0x7f040c108930 2024-05-10T03:57:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.480+0000 7f040bfff700 1 --2- 172.21.15.150:0/3072805744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 0x7f040c19b480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f03fc006af0 tx=0x7f03fc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.480+0000 7f04097fa700 1 -- 172.21.15.150:0/3072805744 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03fc0038b0 con 0x7f040c108930 2024-05-10T03:57:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.480+0000 7f04124ba700 1 -- 172.21.15.150:0/3072805744 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040c19bc20 con 0x7f040c108930 2024-05-10T03:57:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.480+0000 7f04097fa700 1 -- 172.21.15.150:0/3072805744 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03fc003a10 con 0x7f040c108930 2024-05-10T03:57:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.480+0000 7f04097fa700 1 -- 172.21.15.150:0/3072805744 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03fc019550 con 0x7f040c108930 2024-05-10T03:57:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.480+0000 7f04124ba700 1 -- 172.21.15.150:0/3072805744 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040c19e860 con 0x7f040c108930 2024-05-10T03:57:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.481+0000 7f04097fa700 1 -- 172.21.15.150:0/3072805744 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f03fc019810 con 0x7f040c108930 2024-05-10T03:57:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.481+0000 7f04124ba700 1 -- 172.21.15.150:0/3072805744 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040c062840 con 0x7f040c108930 2024-05-10T03:57:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.482+0000 7f04097fa700 1 --2- 172.21.15.150:0/3072805744 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03f4038700 0x7f03f403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.482+0000 7f04097fa700 1 -- 172.21.15.150:0/3072805744 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f03fc04c990 con 0x7f040c108930 2024-05-10T03:57:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.482+0000 7f040b7fe700 1 --2- 172.21.15.150:0/3072805744 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03f4038700 0x7f03f403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.482+0000 7f040b7fe700 1 --2- 172.21.15.150:0/3072805744 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03f4038700 0x7f03f403ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0400006fd0 tx=0x7f0400006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.487+0000 7f04097fa700 1 -- 172.21.15.150:0/3072805744 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f03fc0189a0 con 0x7f040c108930 2024-05-10T03:57:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.708+0000 7f04124ba700 1 -- 172.21.15.150:0/3072805744 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f040c106030 con 0x7f03f4038700 2024-05-10T03:57:33.372 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.934+0000 7f04097fa700 1 -- 172.21.15.150:0/3072805744 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f040c106030 con 0x7f03f4038700 2024-05-10T03:57:33.372 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.936+0000 7f04124ba700 1 -- 172.21.15.150:0/3072805744 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03f4038700 msgr2=0x7f03f403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:33.372 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.936+0000 7f04124ba700 1 --2- 172.21.15.150:0/3072805744 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03f4038700 0x7f03f403ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0400006fd0 tx=0x7f0400006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:33.372 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.936+0000 7f04124ba700 1 -- 172.21.15.150:0/3072805744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 msgr2=0x7f040c19b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:33.372 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.936+0000 7f04124ba700 1 --2- 172.21.15.150:0/3072805744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 0x7f040c19b480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f03fc006af0 tx=0x7f03fc011a30 comp rx=0 tx=0).stop 2024-05-10T03:57:33.372 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.937+0000 7f04124ba700 1 -- 172.21.15.150:0/3072805744 shutdown_connections 2024-05-10T03:57:33.372 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.937+0000 7f04124ba700 1 --2- 172.21.15.150:0/3072805744 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03f4038700 0x7f03f403ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:33.372 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.937+0000 7f04124ba700 1 --2- 172.21.15.150:0/3072805744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f040c108930 0x7f040c19b480 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:33.372 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.937+0000 7f04124ba700 1 -- 172.21.15.150:0/3072805744 >> 172.21.15.150:0/3072805744 conn(0x7f040c100ff0 msgr2=0x7f040c101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:33.372 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.937+0000 7f04124ba700 1 -- 172.21.15.150:0/3072805744 shutdown_connections 2024-05-10T03:57:33.372 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:32.937+0000 7f04124ba700 1 -- 172.21.15.150:0/3072805744 wait complete. 2024-05-10T03:57:33.372 INFO:teuthology.orchestra.run.smithi150.stdout:Fetching dashboard port number... 2024-05-10T03:57:34.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:34 smithi150 ceph-mon[42477]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:34.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:34 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2513088699' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-10T03:57:34.287 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 8443 2024-05-10T03:57:34.293 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.729+0000 7f190bb62700 1 Processor -- start 2024-05-10T03:57:34.293 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.729+0000 7f190bb62700 1 -- start start 2024-05-10T03:57:34.293 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.729+0000 7f190bb62700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 0x7f1904105a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:34.293 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.729+0000 7f190bb62700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1904105ff0 con 0x7f1904105610 2024-05-10T03:57:34.293 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.729+0000 7f19098fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 0x7f1904105a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:34.293 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.729+0000 7f19098fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 0x7f1904105a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54894/0 (socket says 172.21.15.150:54894) 2024-05-10T03:57:34.293 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.729+0000 7f19098fe700 1 -- 172.21.15.150:0/728377577 learned_addr learned my addr 172.21.15.150:0/728377577 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:34.293 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.729+0000 7f19098fe700 1 -- 172.21.15.150:0/728377577 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1904106130 con 0x7f1904105610 2024-05-10T03:57:34.294 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.730+0000 7f19098fe700 1 --2- 172.21.15.150:0/728377577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 0x7f1904105a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19000098a0 tx=0x7f1900009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52491252ad80de59 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:34.294 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.730+0000 7f19088fc700 1 -- 172.21.15.150:0/728377577 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1900004160 con 0x7f1904105610 2024-05-10T03:57:34.294 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.730+0000 7f19088fc700 1 -- 172.21.15.150:0/728377577 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19000042c0 con 0x7f1904105610 2024-05-10T03:57:34.294 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.730+0000 7f19088fc700 1 -- 172.21.15.150:0/728377577 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19000119b0 con 0x7f1904105610 2024-05-10T03:57:34.294 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.730+0000 7f190bb62700 1 -- 172.21.15.150:0/728377577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 msgr2=0x7f1904105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:34.294 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.730+0000 7f190bb62700 1 --2- 172.21.15.150:0/728377577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 0x7f1904105a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19000098a0 tx=0x7f1900009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:34.294 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.731+0000 7f190bb62700 1 -- 172.21.15.150:0/728377577 shutdown_connections 2024-05-10T03:57:34.294 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.731+0000 7f190bb62700 1 --2- 172.21.15.150:0/728377577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 0x7f1904105a20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:34.294 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.731+0000 7f190bb62700 1 -- 172.21.15.150:0/728377577 >> 172.21.15.150:0/728377577 conn(0x7f1904101080 msgr2=0x7f19041034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:34.294 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.731+0000 7f190bb62700 1 -- 172.21.15.150:0/728377577 shutdown_connections 2024-05-10T03:57:34.294 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.731+0000 7f190bb62700 1 -- 172.21.15.150:0/728377577 wait complete. 2024-05-10T03:57:34.294 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.731+0000 7f190bb62700 1 Processor -- start 2024-05-10T03:57:34.294 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.732+0000 7f190bb62700 1 -- start start 2024-05-10T03:57:34.295 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.732+0000 7f190bb62700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 0x7f190419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:34.295 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.732+0000 7f190bb62700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190419ba20 con 0x7f1904105610 2024-05-10T03:57:34.295 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.732+0000 7f19098fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 0x7f190419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:34.295 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.732+0000 7f19098fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 0x7f190419b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54896/0 (socket says 172.21.15.150:54896) 2024-05-10T03:57:34.295 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.732+0000 7f19098fe700 1 -- 172.21.15.150:0/2513088699 learned_addr learned my addr 172.21.15.150:0/2513088699 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:34.295 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.732+0000 7f19098fe700 1 -- 172.21.15.150:0/2513088699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1900009550 con 0x7f1904105610 2024-05-10T03:57:34.295 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.732+0000 7f19098fe700 1 --2- 172.21.15.150:0/2513088699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 0x7f190419b4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1900011980 tx=0x7f1900011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:34.295 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.732+0000 7f18f6ffd700 1 -- 172.21.15.150:0/2513088699 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1900003aa0 con 0x7f1904105610 2024-05-10T03:57:34.295 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.732+0000 7f18f6ffd700 1 -- 172.21.15.150:0/2513088699 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1900003c00 con 0x7f1904105610 2024-05-10T03:57:34.295 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.732+0000 7f190bb62700 1 -- 172.21.15.150:0/2513088699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190419bc20 con 0x7f1904105610 2024-05-10T03:57:34.295 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.732+0000 7f18f6ffd700 1 -- 172.21.15.150:0/2513088699 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19000196c0 con 0x7f1904105610 2024-05-10T03:57:34.295 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.733+0000 7f190bb62700 1 -- 172.21.15.150:0/2513088699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190419c0c0 con 0x7f1904105610 2024-05-10T03:57:34.295 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.733+0000 7f190bb62700 1 -- 172.21.15.150:0/2513088699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190410b840 con 0x7f1904105610 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.734+0000 7f18f6ffd700 1 -- 172.21.15.150:0/2513088699 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f190001aa80 con 0x7f1904105610 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.734+0000 7f18f6ffd700 1 --2- 172.21.15.150:0/2513088699 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f18f0038750 0x7f18f003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.734+0000 7f18f6ffd700 1 -- 172.21.15.150:0/2513088699 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f190004d0f0 con 0x7f1904105610 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.734+0000 7f19090fd700 1 --2- 172.21.15.150:0/2513088699 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f18f0038750 0x7f18f003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.735+0000 7f19090fd700 1 --2- 172.21.15.150:0/2513088699 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f18f0038750 0x7f18f003ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18f8006fd0 tx=0x7f18f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.739+0000 7f18f6ffd700 1 -- 172.21.15.150:0/2513088699 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1900025460 con 0x7f1904105610 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.930+0000 7f190bb62700 1 -- 172.21.15.150:0/2513088699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f190419ebb0 con 0x7f1904105610 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.931+0000 7f18f6ffd700 1 -- 172.21.15.150:0/2513088699 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f190000b3e0 con 0x7f1904105610 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.933+0000 7f190bb62700 1 -- 172.21.15.150:0/2513088699 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f18f0038750 msgr2=0x7f18f003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.933+0000 7f190bb62700 1 --2- 172.21.15.150:0/2513088699 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f18f0038750 0x7f18f003ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18f8006fd0 tx=0x7f18f8006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.933+0000 7f190bb62700 1 -- 172.21.15.150:0/2513088699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 msgr2=0x7f190419b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.933+0000 7f190bb62700 1 --2- 172.21.15.150:0/2513088699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 0x7f190419b4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1900011980 tx=0x7f1900011a60 comp rx=0 tx=0).stop 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.933+0000 7f190bb62700 1 -- 172.21.15.150:0/2513088699 shutdown_connections 2024-05-10T03:57:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.933+0000 7f190bb62700 1 --2- 172.21.15.150:0/2513088699 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f18f0038750 0x7f18f003ac60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:34.297 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.933+0000 7f190bb62700 1 --2- 172.21.15.150:0/2513088699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1904105610 0x7f190419b4e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:34.297 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.933+0000 7f190bb62700 1 -- 172.21.15.150:0/2513088699 >> 172.21.15.150:0/2513088699 conn(0x7f1904101080 msgr2=0x7f19041029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:34.297 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.933+0000 7f190bb62700 1 -- 172.21.15.150:0/2513088699 shutdown_connections 2024-05-10T03:57:34.297 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:33.933+0000 7f190bb62700 1 -- 172.21.15.150:0/2513088699 wait complete. 2024-05-10T03:57:34.299 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T03:57:34.299 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout disabled 2024-05-10T03:57:34.307 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T03:57:34.307 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout inactive 2024-05-10T03:57:34.307 INFO:teuthology.orchestra.run.smithi150.stdout:firewalld.service is not enabled 2024-05-10T03:57:34.307 INFO:teuthology.orchestra.run.smithi150.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-10T03:57:34.308 INFO:teuthology.orchestra.run.smithi150.stdout:Ceph Dashboard is now available at: 2024-05-10T03:57:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:57:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: URL: https://smithi150:8443/ 2024-05-10T03:57:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: User: admin 2024-05-10T03:57:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: Password: rbsj1mi3jm 2024-05-10T03:57:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:57:34.309 INFO:teuthology.orchestra.run.smithi150.stdout:Saving cluster configuration to /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config directory 2024-05-10T03:57:34.309 INFO:teuthology.orchestra.run.smithi150.stdout:Enabling autotune for osd_memory_target 2024-05-10T03:57:35.210 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.638+0000 7f4d4795d700 1 Processor -- start 2024-05-10T03:57:35.210 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.638+0000 7f4d4795d700 1 -- start start 2024-05-10T03:57:35.210 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.638+0000 7f4d4795d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 0x7f4d4010a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:35.211 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.638+0000 7f4d4795d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4007ae20 con 0x7f4d40107de0 2024-05-10T03:57:35.211 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.638+0000 7f4d456f9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 0x7f4d4010a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:35.211 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.638+0000 7f4d456f9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 0x7f4d4010a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54906/0 (socket says 172.21.15.150:54906) 2024-05-10T03:57:35.211 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.638+0000 7f4d456f9700 1 -- 172.21.15.150:0/745443188 learned_addr learned my addr 172.21.15.150:0/745443188 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:35.212 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.638+0000 7f4d456f9700 1 -- 172.21.15.150:0/745443188 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4010a740 con 0x7f4d40107de0 2024-05-10T03:57:35.212 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.639+0000 7f4d456f9700 1 --2- 172.21.15.150:0/745443188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 0x7f4d4010a200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d300098a0 tx=0x7f4d30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9975a6a191ec3e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:35.212 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.639+0000 7f4d37fff700 1 -- 172.21.15.150:0/745443188 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d30004160 con 0x7f4d40107de0 2024-05-10T03:57:35.212 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.639+0000 7f4d37fff700 1 -- 172.21.15.150:0/745443188 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d300042c0 con 0x7f4d40107de0 2024-05-10T03:57:35.212 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.639+0000 7f4d37fff700 1 -- 172.21.15.150:0/745443188 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d300119b0 con 0x7f4d40107de0 2024-05-10T03:57:35.212 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.639+0000 7f4d4795d700 1 -- 172.21.15.150:0/745443188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 msgr2=0x7f4d4010a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:35.212 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.639+0000 7f4d4795d700 1 --2- 172.21.15.150:0/745443188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 0x7f4d4010a200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d300098a0 tx=0x7f4d30009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:35.212 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.640+0000 7f4d4795d700 1 -- 172.21.15.150:0/745443188 shutdown_connections 2024-05-10T03:57:35.212 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.640+0000 7f4d4795d700 1 --2- 172.21.15.150:0/745443188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 0x7f4d4010a200 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:35.212 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.640+0000 7f4d4795d700 1 -- 172.21.15.150:0/745443188 >> 172.21.15.150:0/745443188 conn(0x7f4d40101820 msgr2=0x7f4d40103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:35.212 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.640+0000 7f4d4795d700 1 -- 172.21.15.150:0/745443188 shutdown_connections 2024-05-10T03:57:35.213 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.640+0000 7f4d4795d700 1 -- 172.21.15.150:0/745443188 wait complete. 2024-05-10T03:57:35.213 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.640+0000 7f4d4795d700 1 Processor -- start 2024-05-10T03:57:35.213 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.640+0000 7f4d4795d700 1 -- start start 2024-05-10T03:57:35.213 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.641+0000 7f4d4795d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 0x7f4d4019b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:35.213 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.641+0000 7f4d4795d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4019b9c0 con 0x7f4d40107de0 2024-05-10T03:57:35.213 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.641+0000 7f4d456f9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 0x7f4d4019b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:35.213 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.641+0000 7f4d456f9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 0x7f4d4019b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54910/0 (socket says 172.21.15.150:54910) 2024-05-10T03:57:35.213 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.641+0000 7f4d456f9700 1 -- 172.21.15.150:0/3979170860 learned_addr learned my addr 172.21.15.150:0/3979170860 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:35.213 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.641+0000 7f4d456f9700 1 -- 172.21.15.150:0/3979170860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d30009550 con 0x7f4d40107de0 2024-05-10T03:57:35.213 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.641+0000 7f4d456f9700 1 --2- 172.21.15.150:0/3979170860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 0x7f4d4019b480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d30011670 tx=0x7f4d30011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:35.214 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.641+0000 7f4d367fc700 1 -- 172.21.15.150:0/3979170860 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d30003af0 con 0x7f4d40107de0 2024-05-10T03:57:35.214 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.641+0000 7f4d367fc700 1 -- 172.21.15.150:0/3979170860 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d30003c50 con 0x7f4d40107de0 2024-05-10T03:57:35.214 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.641+0000 7f4d4795d700 1 -- 172.21.15.150:0/3979170860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4019bbc0 con 0x7f4d40107de0 2024-05-10T03:57:35.214 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.641+0000 7f4d367fc700 1 -- 172.21.15.150:0/3979170860 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d30024820 con 0x7f4d40107de0 2024-05-10T03:57:35.214 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.641+0000 7f4d4795d700 1 -- 172.21.15.150:0/3979170860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4019c060 con 0x7f4d40107de0 2024-05-10T03:57:35.214 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.642+0000 7f4d367fc700 1 -- 172.21.15.150:0/3979170860 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f4d30024a50 con 0x7f4d40107de0 2024-05-10T03:57:35.215 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.642+0000 7f4d367fc700 1 --2- 172.21.15.150:0/3979170860 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4d2c0387a0 0x7f4d2c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:35.215 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.642+0000 7f4d367fc700 1 -- 172.21.15.150:0/3979170860 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4d3004c820 con 0x7f4d40107de0 2024-05-10T03:57:35.215 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.642+0000 7f4d44ef8700 1 --2- 172.21.15.150:0/3979170860 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4d2c0387a0 0x7f4d2c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:35.215 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.643+0000 7f4d4795d700 1 -- 172.21.15.150:0/3979170860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4010b520 con 0x7f4d40107de0 2024-05-10T03:57:35.215 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.643+0000 7f4d44ef8700 1 --2- 172.21.15.150:0/3979170860 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4d2c0387a0 0x7f4d2c03acb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c006fd0 tx=0x7f4d3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:35.215 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.648+0000 7f4d367fc700 1 -- 172.21.15.150:0/3979170860 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4d30010790 con 0x7f4d40107de0 2024-05-10T03:57:35.215 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.841+0000 7f4d4795d700 1 -- 172.21.15.150:0/3979170860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f4d4019ea20 con 0x7f4d40107de0 2024-05-10T03:57:35.215 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.841+0000 7f4d367fc700 1 -- 172.21.15.150:0/3979170860 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f4d3004a020 con 0x7f4d40107de0 2024-05-10T03:57:35.215 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.844+0000 7f4d4795d700 1 -- 172.21.15.150:0/3979170860 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4d2c0387a0 msgr2=0x7f4d2c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:35.215 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.844+0000 7f4d4795d700 1 --2- 172.21.15.150:0/3979170860 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4d2c0387a0 0x7f4d2c03acb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c006fd0 tx=0x7f4d3c006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:35.215 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.844+0000 7f4d4795d700 1 -- 172.21.15.150:0/3979170860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 msgr2=0x7f4d4019b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:35.215 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.844+0000 7f4d4795d700 1 --2- 172.21.15.150:0/3979170860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 0x7f4d4019b480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d30011670 tx=0x7f4d30011a30 comp rx=0 tx=0).stop 2024-05-10T03:57:35.216 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.844+0000 7f4d4795d700 1 -- 172.21.15.150:0/3979170860 shutdown_connections 2024-05-10T03:57:35.216 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.844+0000 7f4d4795d700 1 --2- 172.21.15.150:0/3979170860 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4d2c0387a0 0x7f4d2c03acb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:35.216 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.844+0000 7f4d4795d700 1 --2- 172.21.15.150:0/3979170860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40107de0 0x7f4d4019b480 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:35.216 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.844+0000 7f4d4795d700 1 -- 172.21.15.150:0/3979170860 >> 172.21.15.150:0/3979170860 conn(0x7f4d40101820 msgr2=0x7f4d40102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:35.216 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.844+0000 7f4d4795d700 1 -- 172.21.15.150:0/3979170860 shutdown_connections 2024-05-10T03:57:35.216 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:34.844+0000 7f4d4795d700 1 -- 172.21.15.150:0/3979170860 wait complete. 2024-05-10T03:57:36.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:36 smithi150 ceph-mon[42477]: mgrmap e13: x(active, since 6s) 2024-05-10T03:57:36.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:36 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4238773619' entity='client.admin' 2024-05-10T03:57:36.251 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.570+0000 7fdb99887700 1 Processor -- start 2024-05-10T03:57:36.252 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.570+0000 7fdb99887700 1 -- start start 2024-05-10T03:57:36.252 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.570+0000 7fdb99887700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 0x7fdb94105a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:36.252 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.570+0000 7fdb99887700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb94105ff0 con 0x7fdb94105610 2024-05-10T03:57:36.252 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.570+0000 7fdb92ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 0x7fdb94105a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:36.252 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.570+0000 7fdb92ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 0x7fdb94105a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54920/0 (socket says 172.21.15.150:54920) 2024-05-10T03:57:36.252 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.570+0000 7fdb92ffd700 1 -- 172.21.15.150:0/735548447 learned_addr learned my addr 172.21.15.150:0/735548447 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:36.252 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.570+0000 7fdb92ffd700 1 -- 172.21.15.150:0/735548447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb94106130 con 0x7fdb94105610 2024-05-10T03:57:36.252 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.571+0000 7fdb92ffd700 1 --2- 172.21.15.150:0/735548447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 0x7fdb94105a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdb880098a0 tx=0x7fdb88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5015c8954bf1d476 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:36.252 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.571+0000 7fdb91ffb700 1 -- 172.21.15.150:0/735548447 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb88004160 con 0x7fdb94105610 2024-05-10T03:57:36.252 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.571+0000 7fdb91ffb700 1 -- 172.21.15.150:0/735548447 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb880042c0 con 0x7fdb94105610 2024-05-10T03:57:36.252 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.571+0000 7fdb91ffb700 1 -- 172.21.15.150:0/735548447 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb880119b0 con 0x7fdb94105610 2024-05-10T03:57:36.252 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.571+0000 7fdb99887700 1 -- 172.21.15.150:0/735548447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 msgr2=0x7fdb94105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.571+0000 7fdb99887700 1 --2- 172.21.15.150:0/735548447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 0x7fdb94105a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdb880098a0 tx=0x7fdb88009bb0 comp rx=0 tx=0).stop 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.571+0000 7fdb99887700 1 -- 172.21.15.150:0/735548447 shutdown_connections 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.571+0000 7fdb99887700 1 --2- 172.21.15.150:0/735548447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 0x7fdb94105a20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.571+0000 7fdb99887700 1 -- 172.21.15.150:0/735548447 >> 172.21.15.150:0/735548447 conn(0x7fdb94101080 msgr2=0x7fdb941034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.572+0000 7fdb99887700 1 -- 172.21.15.150:0/735548447 shutdown_connections 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.572+0000 7fdb99887700 1 -- 172.21.15.150:0/735548447 wait complete. 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.572+0000 7fdb99887700 1 Processor -- start 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.572+0000 7fdb99887700 1 -- start start 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.573+0000 7fdb99887700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 0x7fdb9419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.573+0000 7fdb99887700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9419ba20 con 0x7fdb94105610 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.573+0000 7fdb92ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 0x7fdb9419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.573+0000 7fdb92ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 0x7fdb9419b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54932/0 (socket says 172.21.15.150:54932) 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.573+0000 7fdb92ffd700 1 -- 172.21.15.150:0/4238773619 learned_addr learned my addr 172.21.15.150:0/4238773619 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:36.253 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.573+0000 7fdb92ffd700 1 -- 172.21.15.150:0/4238773619 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb88009550 con 0x7fdb94105610 2024-05-10T03:57:36.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.573+0000 7fdb92ffd700 1 --2- 172.21.15.150:0/4238773619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 0x7fdb9419b4e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdb88006af0 tx=0x7fdb88011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:36.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.573+0000 7fdb98885700 1 -- 172.21.15.150:0/4238773619 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb880038b0 con 0x7fdb94105610 2024-05-10T03:57:36.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.573+0000 7fdb99887700 1 -- 172.21.15.150:0/4238773619 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9419bc20 con 0x7fdb94105610 2024-05-10T03:57:36.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.573+0000 7fdb98885700 1 -- 172.21.15.150:0/4238773619 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb88003a10 con 0x7fdb94105610 2024-05-10T03:57:36.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.573+0000 7fdb98885700 1 -- 172.21.15.150:0/4238773619 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb88019550 con 0x7fdb94105610 2024-05-10T03:57:36.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.573+0000 7fdb99887700 1 -- 172.21.15.150:0/4238773619 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9419c0c0 con 0x7fdb94105610 2024-05-10T03:57:36.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.574+0000 7fdb99887700 1 -- 172.21.15.150:0/4238773619 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9410b840 con 0x7fdb94105610 2024-05-10T03:57:36.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.574+0000 7fdb98885700 1 -- 172.21.15.150:0/4238773619 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fdb88019810 con 0x7fdb94105610 2024-05-10T03:57:36.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.574+0000 7fdb98885700 1 --2- 172.21.15.150:0/4238773619 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdb7c0387a0 0x7fdb7c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:36.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.575+0000 7fdb98885700 1 -- 172.21.15.150:0/4238773619 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdb8804cab0 con 0x7fdb94105610 2024-05-10T03:57:36.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.575+0000 7fdb927fc700 1 --2- 172.21.15.150:0/4238773619 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdb7c0387a0 0x7fdb7c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:36.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.575+0000 7fdb927fc700 1 --2- 172.21.15.150:0/4238773619 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdb7c0387a0 0x7fdb7c03acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdb84006fd0 tx=0x7fdb84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:36.254 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.580+0000 7fdb98885700 1 -- 172.21.15.150:0/4238773619 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdb88018db0 con 0x7fdb94105610 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.879+0000 7fdb99887700 1 -- 172.21.15.150:0/4238773619 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fdb94193b60 con 0x7fdb94105610 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.882+0000 7fdb98885700 1 -- 172.21.15.150:0/4238773619 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fdb88018350 con 0x7fdb94105610 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.884+0000 7fdb99887700 1 -- 172.21.15.150:0/4238773619 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdb7c0387a0 msgr2=0x7fdb7c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.884+0000 7fdb99887700 1 --2- 172.21.15.150:0/4238773619 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdb7c0387a0 0x7fdb7c03acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdb84006fd0 tx=0x7fdb84006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.884+0000 7fdb99887700 1 -- 172.21.15.150:0/4238773619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 msgr2=0x7fdb9419b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.884+0000 7fdb99887700 1 --2- 172.21.15.150:0/4238773619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 0x7fdb9419b4e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdb88006af0 tx=0x7fdb88011a30 comp rx=0 tx=0).stop 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.884+0000 7fdb99887700 1 -- 172.21.15.150:0/4238773619 shutdown_connections 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.884+0000 7fdb99887700 1 --2- 172.21.15.150:0/4238773619 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdb7c0387a0 0x7fdb7c03acb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.884+0000 7fdb99887700 1 --2- 172.21.15.150:0/4238773619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb94105610 0x7fdb9419b4e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.884+0000 7fdb99887700 1 -- 172.21.15.150:0/4238773619 >> 172.21.15.150:0/4238773619 conn(0x7fdb94101080 msgr2=0x7fdb941029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.884+0000 7fdb99887700 1 -- 172.21.15.150:0/4238773619 shutdown_connections 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T03:57:35.884+0000 7fdb99887700 1 -- 172.21.15.150:0/4238773619 wait complete. 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-10T03:57:36.255 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout:Or, if you are only running a single cluster on this host: 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: ceph telemetry on 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout:For more information see: 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:57:36.256 INFO:teuthology.orchestra.run.smithi150.stdout:Bootstrap complete. 2024-05-10T03:57:36.286 INFO:tasks.cephadm:Fetching config... 2024-05-10T03:57:36.286 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:57:36.286 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-10T03:57:36.303 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-10T03:57:36.303 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:57:36.303 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-10T03:57:36.360 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-10T03:57:36.360 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:57:36.360 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-10T03:57:36.427 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-10T03:57:36.427 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:57:36.427 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-10T03:57:36.485 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-10T03:57:36.485 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCF8OsiFjLszF1SuH1SpsUyhZ8TstfczHwY/MdC5+ASFQMwKYIBR50HANADpJYs5vMIyfZD4wJpjnFc5foOY3pmIsSwfPglh3nU69tQG7plwF9g8Tn4DTPIaVHDHFHpELce0dayv1tk8WUOYTc8YiJoowGbyb2EZlNGPSU4ubI3pgf+ludW7luzVAp99LyTZAGqlxKOMoxsc+JTCAcWbmAsial8GcJrpmNOjbSA7H5KC3CneWuT07qxo0jHfAJn82SC4SxlP2bj9CzB9c1iC9BYajQfKxOkWxFv+iWlk2FruYmHkE3UJ8FhGnwMIgsUtz8UZkwZ5Z+fuXUD6NvPfVhKt9VoJWPU/NXnydNns2eF2yA1TTBA8aLEDqsbK5pLtSqRuSHahxDC5uOtCx19m/rd0+YNM7bQR0ZebexMOvTx0FFnIGek6eGuKASexTBuQfWhEnckQIpY6ar6MqZTTBaFyw6eZ3ivfDxwGHMhq8mYmGoxOFOsqKyH04kFZjSKt8= ceph-325b4c82-0e81-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T03:57:36.563 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCF8OsiFjLszF1SuH1SpsUyhZ8TstfczHwY/MdC5+ASFQMwKYIBR50HANADpJYs5vMIyfZD4wJpjnFc5foOY3pmIsSwfPglh3nU69tQG7plwF9g8Tn4DTPIaVHDHFHpELce0dayv1tk8WUOYTc8YiJoowGbyb2EZlNGPSU4ubI3pgf+ludW7luzVAp99LyTZAGqlxKOMoxsc+JTCAcWbmAsial8GcJrpmNOjbSA7H5KC3CneWuT07qxo0jHfAJn82SC4SxlP2bj9CzB9c1iC9BYajQfKxOkWxFv+iWlk2FruYmHkE3UJ8FhGnwMIgsUtz8UZkwZ5Z+fuXUD6NvPfVhKt9VoJWPU/NXnydNns2eF2yA1TTBA8aLEDqsbK5pLtSqRuSHahxDC5uOtCx19m/rd0+YNM7bQR0ZebexMOvTx0FFnIGek6eGuKASexTBuQfWhEnckQIpY6ar6MqZTTBaFyw6eZ3ivfDxwGHMhq8mYmGoxOFOsqKyH04kFZjSKt8= ceph-325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:57:36.576 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCF8OsiFjLszF1SuH1SpsUyhZ8TstfczHwY/MdC5+ASFQMwKYIBR50HANADpJYs5vMIyfZD4wJpjnFc5foOY3pmIsSwfPglh3nU69tQG7plwF9g8Tn4DTPIaVHDHFHpELce0dayv1tk8WUOYTc8YiJoowGbyb2EZlNGPSU4ubI3pgf+ludW7luzVAp99LyTZAGqlxKOMoxsc+JTCAcWbmAsial8GcJrpmNOjbSA7H5KC3CneWuT07qxo0jHfAJn82SC4SxlP2bj9CzB9c1iC9BYajQfKxOkWxFv+iWlk2FruYmHkE3UJ8FhGnwMIgsUtz8UZkwZ5Z+fuXUD6NvPfVhKt9VoJWPU/NXnydNns2eF2yA1TTBA8aLEDqsbK5pLtSqRuSHahxDC5uOtCx19m/rd0+YNM7bQR0ZebexMOvTx0FFnIGek6eGuKASexTBuQfWhEnckQIpY6ar6MqZTTBaFyw6eZ3ivfDxwGHMhq8mYmGoxOFOsqKyH04kFZjSKt8= ceph-325b4c82-0e81-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T03:57:36.620 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCF8OsiFjLszF1SuH1SpsUyhZ8TstfczHwY/MdC5+ASFQMwKYIBR50HANADpJYs5vMIyfZD4wJpjnFc5foOY3pmIsSwfPglh3nU69tQG7plwF9g8Tn4DTPIaVHDHFHpELce0dayv1tk8WUOYTc8YiJoowGbyb2EZlNGPSU4ubI3pgf+ludW7luzVAp99LyTZAGqlxKOMoxsc+JTCAcWbmAsial8GcJrpmNOjbSA7H5KC3CneWuT07qxo0jHfAJn82SC4SxlP2bj9CzB9c1iC9BYajQfKxOkWxFv+iWlk2FruYmHkE3UJ8FhGnwMIgsUtz8UZkwZ5Z+fuXUD6NvPfVhKt9VoJWPU/NXnydNns2eF2yA1TTBA8aLEDqsbK5pLtSqRuSHahxDC5uOtCx19m/rd0+YNM7bQR0ZebexMOvTx0FFnIGek6eGuKASexTBuQfWhEnckQIpY6ar6MqZTTBaFyw6eZ3ivfDxwGHMhq8mYmGoxOFOsqKyH04kFZjSKt8= ceph-325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:57:36.634 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCF8OsiFjLszF1SuH1SpsUyhZ8TstfczHwY/MdC5+ASFQMwKYIBR50HANADpJYs5vMIyfZD4wJpjnFc5foOY3pmIsSwfPglh3nU69tQG7plwF9g8Tn4DTPIaVHDHFHpELce0dayv1tk8WUOYTc8YiJoowGbyb2EZlNGPSU4ubI3pgf+ludW7luzVAp99LyTZAGqlxKOMoxsc+JTCAcWbmAsial8GcJrpmNOjbSA7H5KC3CneWuT07qxo0jHfAJn82SC4SxlP2bj9CzB9c1iC9BYajQfKxOkWxFv+iWlk2FruYmHkE3UJ8FhGnwMIgsUtz8UZkwZ5Z+fuXUD6NvPfVhKt9VoJWPU/NXnydNns2eF2yA1TTBA8aLEDqsbK5pLtSqRuSHahxDC5uOtCx19m/rd0+YNM7bQR0ZebexMOvTx0FFnIGek6eGuKASexTBuQfWhEnckQIpY6ar6MqZTTBaFyw6eZ3ivfDxwGHMhq8mYmGoxOFOsqKyH04kFZjSKt8= ceph-325b4c82-0e81-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T03:57:36.679 INFO:teuthology.orchestra.run.smithi181.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCF8OsiFjLszF1SuH1SpsUyhZ8TstfczHwY/MdC5+ASFQMwKYIBR50HANADpJYs5vMIyfZD4wJpjnFc5foOY3pmIsSwfPglh3nU69tQG7plwF9g8Tn4DTPIaVHDHFHpELce0dayv1tk8WUOYTc8YiJoowGbyb2EZlNGPSU4ubI3pgf+ludW7luzVAp99LyTZAGqlxKOMoxsc+JTCAcWbmAsial8GcJrpmNOjbSA7H5KC3CneWuT07qxo0jHfAJn82SC4SxlP2bj9CzB9c1iC9BYajQfKxOkWxFv+iWlk2FruYmHkE3UJ8FhGnwMIgsUtz8UZkwZ5Z+fuXUD6NvPfVhKt9VoJWPU/NXnydNns2eF2yA1TTBA8aLEDqsbK5pLtSqRuSHahxDC5uOtCx19m/rd0+YNM7bQR0ZebexMOvTx0FFnIGek6eGuKASexTBuQfWhEnckQIpY6ar6MqZTTBaFyw6eZ3ivfDxwGHMhq8mYmGoxOFOsqKyH04kFZjSKt8= ceph-325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:57:36.692 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-10T03:57:36.914 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T03:57:37.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.914+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2245665552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c8100430 msgr2=0x7f68c8100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:37.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.914+0000 7f68ce8d6700 1 --2- 172.21.15.150:0/2245665552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c8100430 0x7f68c8100840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009910 tx=0x7f68b8009c20 comp rx=0 tx=0).stop 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.915+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2245665552 shutdown_connections 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.915+0000 7f68ce8d6700 1 --2- 172.21.15.150:0/2245665552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c8100430 0x7f68c8100840 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.915+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2245665552 >> 172.21.15.150:0/2245665552 conn(0x7f68c80fba50 msgr2=0x7f68c80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.915+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2245665552 shutdown_connections 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.915+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2245665552 wait complete. 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.915+0000 7f68ce8d6700 1 Processor -- start 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.915+0000 7f68ce8d6700 1 -- start start 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.916+0000 7f68ce8d6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c8100430 0x7f68c8068ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.916+0000 7f68ce8d6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8197a20 con 0x7f68c8100430 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.916+0000 7f68c7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c8100430 0x7f68c8068ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.916+0000 7f68c7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c8100430 0x7f68c8068ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54956/0 (socket says 172.21.15.150:54956) 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.916+0000 7f68c7fff700 1 -- 172.21.15.150:0/2969481623 learned_addr learned my addr 172.21.15.150:0/2969481623 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.916+0000 7f68c7fff700 1 -- 172.21.15.150:0/2969481623 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b80095f0 con 0x7f68c8100430 2024-05-10T03:57:37.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.916+0000 7f68c7fff700 1 --2- 172.21.15.150:0/2969481623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c8100430 0x7f68c8068ef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68b8004f40 tx=0x7f68b8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:37.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.916+0000 7f68c57fa700 1 -- 172.21.15.150:0/2969481623 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68b801d070 con 0x7f68c8100430 2024-05-10T03:57:37.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.916+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2969481623 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c8069430 con 0x7f68c8100430 2024-05-10T03:57:37.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.916+0000 7f68c57fa700 1 -- 172.21.15.150:0/2969481623 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68b8022470 con 0x7f68c8100430 2024-05-10T03:57:37.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.916+0000 7f68c57fa700 1 -- 172.21.15.150:0/2969481623 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68b800f5f0 con 0x7f68c8100430 2024-05-10T03:57:37.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.916+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2969481623 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c819c200 con 0x7f68c8100430 2024-05-10T03:57:37.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.917+0000 7f68c57fa700 1 -- 172.21.15.150:0/2969481623 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f68b800f860 con 0x7f68c8100430 2024-05-10T03:57:37.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.917+0000 7f68c57fa700 1 --2- 172.21.15.150:0/2969481623 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f68b00387f0 0x7f68b003ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:37.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.917+0000 7f68c57fa700 1 -- 172.21.15.150:0/2969481623 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f68b804d7d0 con 0x7f68c8100430 2024-05-10T03:57:37.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.918+0000 7f68c77fe700 1 --2- 172.21.15.150:0/2969481623 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f68b00387f0 0x7f68b003ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:37.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.918+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2969481623 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c8062840 con 0x7f68c8100430 2024-05-10T03:57:37.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.918+0000 7f68c77fe700 1 --2- 172.21.15.150:0/2969481623 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f68b00387f0 0x7f68b003ad00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f68bc006fd0 tx=0x7f68bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:37.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:37.923+0000 7f68c57fa700 1 -- 172.21.15.150:0/2969481623 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f68b8027070 con 0x7f68c8100430 2024-05-10T03:57:38.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:38.127+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2969481623 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f68c8106210 con 0x7f68c8100430 2024-05-10T03:57:38.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:38.131+0000 7f68c57fa700 1 -- 172.21.15.150:0/2969481623 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f68b802a360 con 0x7f68c8100430 2024-05-10T03:57:38.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:38.133+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2969481623 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f68b00387f0 msgr2=0x7f68b003ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:38.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:38.133+0000 7f68ce8d6700 1 --2- 172.21.15.150:0/2969481623 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f68b00387f0 0x7f68b003ad00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f68bc006fd0 tx=0x7f68bc006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:38.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:38.133+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2969481623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c8100430 msgr2=0x7f68c8068ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:38.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:38.133+0000 7f68ce8d6700 1 --2- 172.21.15.150:0/2969481623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c8100430 0x7f68c8068ef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68b8004f40 tx=0x7f68b8009f00 comp rx=0 tx=0).stop 2024-05-10T03:57:38.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:38.133+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2969481623 shutdown_connections 2024-05-10T03:57:38.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:38.133+0000 7f68ce8d6700 1 --2- 172.21.15.150:0/2969481623 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f68b00387f0 0x7f68b003ad00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:38.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:38.133+0000 7f68ce8d6700 1 --2- 172.21.15.150:0/2969481623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c8100430 0x7f68c8068ef0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:38.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:38.133+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2969481623 >> 172.21.15.150:0/2969481623 conn(0x7f68c80fba50 msgr2=0x7f68c80fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:38.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:38.133+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2969481623 shutdown_connections 2024-05-10T03:57:38.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:38.133+0000 7f68ce8d6700 1 -- 172.21.15.150:0/2969481623 wait complete. 2024-05-10T03:57:38.559 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-10T03:57:38.560 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-10T03:57:38.790 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T03:57:39.194 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:39 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2969481623' entity='client.admin' 2024-05-10T03:57:39.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.866+0000 7f94d87ca700 1 -- 172.21.15.150:0/2699635351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d0101580 msgr2=0x7f94d0101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:39.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.866+0000 7f94d87ca700 1 --2- 172.21.15.150:0/2699635351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d0101580 0x7f94d0101990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009910 tx=0x7f94cc009c20 comp rx=0 tx=0).stop 2024-05-10T03:57:39.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.866+0000 7f94d87ca700 1 -- 172.21.15.150:0/2699635351 shutdown_connections 2024-05-10T03:57:39.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.866+0000 7f94d87ca700 1 --2- 172.21.15.150:0/2699635351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d0101580 0x7f94d0101990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:39.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.866+0000 7f94d87ca700 1 -- 172.21.15.150:0/2699635351 >> 172.21.15.150:0/2699635351 conn(0x7f94d00fcba0 msgr2=0x7f94d00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:39.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.866+0000 7f94d87ca700 1 -- 172.21.15.150:0/2699635351 shutdown_connections 2024-05-10T03:57:39.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.866+0000 7f94d87ca700 1 -- 172.21.15.150:0/2699635351 wait complete. 2024-05-10T03:57:39.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.867+0000 7f94d87ca700 1 Processor -- start 2024-05-10T03:57:39.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.867+0000 7f94d87ca700 1 -- start start 2024-05-10T03:57:39.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.867+0000 7f94d87ca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d0101580 0x7f94d0074dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:39.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.867+0000 7f94d87ca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d0075310 con 0x7f94d0101580 2024-05-10T03:57:39.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.867+0000 7f94d6566700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d0101580 0x7f94d0074dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:39.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.867+0000 7f94d6566700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d0101580 0x7f94d0074dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52668/0 (socket says 172.21.15.150:52668) 2024-05-10T03:57:39.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.867+0000 7f94d6566700 1 -- 172.21.15.150:0/1862677470 learned_addr learned my addr 172.21.15.150:0/1862677470 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:39.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.867+0000 7f94d6566700 1 -- 172.21.15.150:0/1862677470 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc0095f0 con 0x7f94d0101580 2024-05-10T03:57:39.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.868+0000 7f94d6566700 1 --2- 172.21.15.150:0/1862677470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d0101580 0x7f94d0074dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f94cc004f40 tx=0x7f94cc009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:39.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.868+0000 7f94c77fe700 1 -- 172.21.15.150:0/1862677470 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94cc01d070 con 0x7f94d0101580 2024-05-10T03:57:39.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.868+0000 7f94c77fe700 1 -- 172.21.15.150:0/1862677470 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94cc022470 con 0x7f94d0101580 2024-05-10T03:57:39.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.868+0000 7f94c77fe700 1 -- 172.21.15.150:0/1862677470 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94cc00f5f0 con 0x7f94d0101580 2024-05-10T03:57:39.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.868+0000 7f94d87ca700 1 -- 172.21.15.150:0/1862677470 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d0073560 con 0x7f94d0101580 2024-05-10T03:57:39.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.868+0000 7f94d87ca700 1 -- 172.21.15.150:0/1862677470 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d0073a00 con 0x7f94d0101580 2024-05-10T03:57:39.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.869+0000 7f94c77fe700 1 -- 172.21.15.150:0/1862677470 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f94cc00f8b0 con 0x7f94d0101580 2024-05-10T03:57:39.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.869+0000 7f94d87ca700 1 -- 172.21.15.150:0/1862677470 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d0074410 con 0x7f94d0101580 2024-05-10T03:57:39.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.869+0000 7f94c77fe700 1 --2- 172.21.15.150:0/1862677470 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f94bc0387a0 0x7f94bc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:39.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.869+0000 7f94c77fe700 1 -- 172.21.15.150:0/1862677470 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f94cc04c8b0 con 0x7f94d0101580 2024-05-10T03:57:39.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.869+0000 7f94d5d65700 1 --2- 172.21.15.150:0/1862677470 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f94bc0387a0 0x7f94bc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:39.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.869+0000 7f94d5d65700 1 --2- 172.21.15.150:0/1862677470 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f94bc0387a0 0x7f94bc03acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f94c0006fd0 tx=0x7f94c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:39.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:39.874+0000 7f94c77fe700 1 -- 172.21.15.150:0/1862677470 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f94cc02a360 con 0x7f94d0101580 2024-05-10T03:57:40.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:40.077+0000 7f94d87ca700 1 -- 172.21.15.150:0/1862677470 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f94d0073c40 con 0x7f94bc0387a0 2024-05-10T03:57:40.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:40.081+0000 7f94c77fe700 1 -- 172.21.15.150:0/1862677470 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f94d0073c40 con 0x7f94bc0387a0 2024-05-10T03:57:40.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:40.083+0000 7f94d87ca700 1 -- 172.21.15.150:0/1862677470 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f94bc0387a0 msgr2=0x7f94bc03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:40.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:40.083+0000 7f94d87ca700 1 --2- 172.21.15.150:0/1862677470 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f94bc0387a0 0x7f94bc03acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f94c0006fd0 tx=0x7f94c0006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:40.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:40.083+0000 7f94d87ca700 1 -- 172.21.15.150:0/1862677470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d0101580 msgr2=0x7f94d0074dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:40.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:40.083+0000 7f94d87ca700 1 --2- 172.21.15.150:0/1862677470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d0101580 0x7f94d0074dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f94cc004f40 tx=0x7f94cc009f00 comp rx=0 tx=0).stop 2024-05-10T03:57:40.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:40.083+0000 7f94d87ca700 1 -- 172.21.15.150:0/1862677470 shutdown_connections 2024-05-10T03:57:40.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:40.083+0000 7f94d87ca700 1 --2- 172.21.15.150:0/1862677470 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f94bc0387a0 0x7f94bc03acb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:40.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:40.083+0000 7f94d87ca700 1 --2- 172.21.15.150:0/1862677470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d0101580 0x7f94d0074dd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:40.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:40.083+0000 7f94d87ca700 1 -- 172.21.15.150:0/1862677470 >> 172.21.15.150:0/1862677470 conn(0x7f94d00fcba0 msgr2=0x7f94d00fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:40.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:40.083+0000 7f94d87ca700 1 -- 172.21.15.150:0/1862677470 shutdown_connections 2024-05-10T03:57:40.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:40.083+0000 7f94d87ca700 1 -- 172.21.15.150:0/1862677470 wait complete. 2024-05-10T03:57:40.419 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2024-05-10T03:57:40.419 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T03:57:40.419 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2024-05-10T03:57:40.440 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T03:57:40.440 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T03:57:40.500 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2024-05-10T03:57:40.500 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch host add smithi161 2024-05-10T03:57:40.730 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T03:57:41.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:41.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:41.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:57:41.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:41.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:57:41.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:41.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:40 smithi150 ceph-mon[42477]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:41.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:41.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:57:41.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:57:41.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:57:41.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:40 smithi150 ceph-mon[42477]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-10T03:57:41.749 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.745+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3438327000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7ec100610 msgr2=0x7fe7ec100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:41.749 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.746+0000 7fe7f0d9c700 1 --2- 172.21.15.150:0/3438327000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7ec100610 0x7fe7ec100a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0009910 tx=0x7fe7e0009c20 comp rx=0 tx=0).stop 2024-05-10T03:57:41.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.746+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3438327000 shutdown_connections 2024-05-10T03:57:41.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.746+0000 7fe7f0d9c700 1 --2- 172.21.15.150:0/3438327000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7ec100610 0x7fe7ec100a20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:41.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.746+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3438327000 >> 172.21.15.150:0/3438327000 conn(0x7fe7ec075f10 msgr2=0x7fe7ec076350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:41.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.746+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3438327000 shutdown_connections 2024-05-10T03:57:41.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.746+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3438327000 wait complete. 2024-05-10T03:57:41.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.747+0000 7fe7f0d9c700 1 Processor -- start 2024-05-10T03:57:41.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.747+0000 7fe7f0d9c700 1 -- start start 2024-05-10T03:57:41.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.747+0000 7fe7f0d9c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7ec100610 0x7fe7ec197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:41.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.747+0000 7fe7f0d9c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec069480 con 0x7fe7ec100610 2024-05-10T03:57:41.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.747+0000 7fe7ea59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7ec100610 0x7fe7ec197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:41.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.747+0000 7fe7ea59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7ec100610 0x7fe7ec197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52680/0 (socket says 172.21.15.150:52680) 2024-05-10T03:57:41.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.747+0000 7fe7ea59c700 1 -- 172.21.15.150:0/3488502844 learned_addr learned my addr 172.21.15.150:0/3488502844 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:41.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.748+0000 7fe7ea59c700 1 -- 172.21.15.150:0/3488502844 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e00095f0 con 0x7fe7ec100610 2024-05-10T03:57:41.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.748+0000 7fe7ea59c700 1 --2- 172.21.15.150:0/3488502844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7ec100610 0x7fe7ec197ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0004f40 tx=0x7fe7e0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:41.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.748+0000 7fe7db7fe700 1 -- 172.21.15.150:0/3488502844 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7e001d070 con 0x7fe7ec100610 2024-05-10T03:57:41.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.748+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3488502844 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ec198460 con 0x7fe7ec100610 2024-05-10T03:57:41.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.748+0000 7fe7db7fe700 1 -- 172.21.15.150:0/3488502844 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7e000bc10 con 0x7fe7ec100610 2024-05-10T03:57:41.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.748+0000 7fe7db7fe700 1 -- 172.21.15.150:0/3488502844 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7e00037d0 con 0x7fe7ec100610 2024-05-10T03:57:41.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.748+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3488502844 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ec1988e0 con 0x7fe7ec100610 2024-05-10T03:57:41.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.749+0000 7fe7db7fe700 1 -- 172.21.15.150:0/3488502844 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fe7e0003a90 con 0x7fe7ec100610 2024-05-10T03:57:41.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.749+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3488502844 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ec101ac0 con 0x7fe7ec100610 2024-05-10T03:57:41.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.749+0000 7fe7db7fe700 1 --2- 172.21.15.150:0/3488502844 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe7d40387a0 0x7fe7d403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:41.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.749+0000 7fe7db7fe700 1 -- 172.21.15.150:0/3488502844 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe7e004c5f0 con 0x7fe7ec100610 2024-05-10T03:57:41.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.749+0000 7fe7e9d9b700 1 --2- 172.21.15.150:0/3488502844 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe7d40387a0 0x7fe7d403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:41.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.750+0000 7fe7e9d9b700 1 --2- 172.21.15.150:0/3488502844 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe7d40387a0 0x7fe7d403acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc006fd0 tx=0x7fe7dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:41.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.754+0000 7fe7db7fe700 1 -- 172.21.15.150:0/3488502844 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe7e0029770 con 0x7fe7ec100610 2024-05-10T03:57:41.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:41.949+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3488502844 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7ec198b80 con 0x7fe7d40387a0 2024-05-10T03:57:42.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:41 smithi150 ceph-mon[42477]: Updating smithi150:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:57:42.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:41 smithi150 ceph-mon[42477]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-10T03:57:42.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:41 smithi150 ceph-mon[42477]: Updating smithi150:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T03:57:42.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:41 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:42.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:41 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:42.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:41 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:43.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:42 smithi150 ceph-mon[42477]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:44.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:44.119+0000 7fe7db7fe700 1 -- 172.21.15.150:0/3488502844 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe7ec198b80 con 0x7fe7d40387a0 2024-05-10T03:57:44.123 INFO:teuthology.orchestra.run.smithi150.stdout:Added host 'smithi161' with addr '172.21.15.161' 2024-05-10T03:57:44.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:44.121+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3488502844 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe7d40387a0 msgr2=0x7fe7d403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:44.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:44.121+0000 7fe7f0d9c700 1 --2- 172.21.15.150:0/3488502844 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe7d40387a0 0x7fe7d403acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc006fd0 tx=0x7fe7dc006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:44.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:44.121+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3488502844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7ec100610 msgr2=0x7fe7ec197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:44.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:44.121+0000 7fe7f0d9c700 1 --2- 172.21.15.150:0/3488502844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7ec100610 0x7fe7ec197ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0004f40 tx=0x7fe7e0005020 comp rx=0 tx=0).stop 2024-05-10T03:57:44.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:44.121+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3488502844 shutdown_connections 2024-05-10T03:57:44.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:44.121+0000 7fe7f0d9c700 1 --2- 172.21.15.150:0/3488502844 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe7d40387a0 0x7fe7d403acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:44.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:44.121+0000 7fe7f0d9c700 1 --2- 172.21.15.150:0/3488502844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7ec100610 0x7fe7ec197ec0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:44.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:44.121+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3488502844 >> 172.21.15.150:0/3488502844 conn(0x7fe7ec075f10 msgr2=0x7fe7ec101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:44.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:44.121+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3488502844 shutdown_connections 2024-05-10T03:57:44.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:44.121+0000 7fe7f0d9c700 1 -- 172.21.15.150:0/3488502844 wait complete. 2024-05-10T03:57:44.293 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:43 smithi150 ceph-mon[42477]: Deploying cephadm binary to smithi161 2024-05-10T03:57:44.515 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-10T03:57:44.733 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T03:57:45.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:45 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:45.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:45 smithi150 ceph-mon[42477]: Added host smithi161 2024-05-10T03:57:45.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:45 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:57:45.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:45 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:45.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:45 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:45.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.661+0000 7f15e7318700 1 -- 172.21.15.150:0/919372681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15e0107630 msgr2=0x7f15e0107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:45.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.661+0000 7f15e7318700 1 --2- 172.21.15.150:0/919372681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15e0107630 0x7f15e0107a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009910 tx=0x7f15dc009c20 comp rx=0 tx=0).stop 2024-05-10T03:57:45.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.661+0000 7f15e7318700 1 -- 172.21.15.150:0/919372681 shutdown_connections 2024-05-10T03:57:45.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.661+0000 7f15e7318700 1 --2- 172.21.15.150:0/919372681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15e0107630 0x7f15e0107a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:45.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.661+0000 7f15e7318700 1 -- 172.21.15.150:0/919372681 >> 172.21.15.150:0/919372681 conn(0x7f15e0075670 msgr2=0x7f15e0075ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:45.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.661+0000 7f15e7318700 1 -- 172.21.15.150:0/919372681 shutdown_connections 2024-05-10T03:57:45.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.661+0000 7f15e7318700 1 -- 172.21.15.150:0/919372681 wait complete. 2024-05-10T03:57:45.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.662+0000 7f15e7318700 1 Processor -- start 2024-05-10T03:57:45.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.662+0000 7f15e7318700 1 -- start start 2024-05-10T03:57:45.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.662+0000 7f15e7318700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15e0107630 0x7f15e0197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:45.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.662+0000 7f15e7318700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e0197660 con 0x7f15e0107630 2024-05-10T03:57:45.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.662+0000 7f15e50b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15e0107630 0x7f15e0197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:45.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.662+0000 7f15e50b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15e0107630 0x7f15e0197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52692/0 (socket says 172.21.15.150:52692) 2024-05-10T03:57:45.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.662+0000 7f15e50b4700 1 -- 172.21.15.150:0/736705098 learned_addr learned my addr 172.21.15.150:0/736705098 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:45.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.662+0000 7f15e50b4700 1 -- 172.21.15.150:0/736705098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15dc0095f0 con 0x7f15e0107630 2024-05-10T03:57:45.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.662+0000 7f15e50b4700 1 --2- 172.21.15.150:0/736705098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15e0107630 0x7f15e0197120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00b5c0 tx=0x7f15dc004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:45.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.663+0000 7f15d67fc700 1 -- 172.21.15.150:0/736705098 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15dc01d070 con 0x7f15e0107630 2024-05-10T03:57:45.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.663+0000 7f15d67fc700 1 -- 172.21.15.150:0/736705098 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15dc00b750 con 0x7f15e0107630 2024-05-10T03:57:45.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.663+0000 7f15e7318700 1 -- 172.21.15.150:0/736705098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e0197860 con 0x7f15e0107630 2024-05-10T03:57:45.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.663+0000 7f15d67fc700 1 -- 172.21.15.150:0/736705098 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15dc003810 con 0x7f15e0107630 2024-05-10T03:57:45.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.663+0000 7f15e7318700 1 -- 172.21.15.150:0/736705098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e0197d00 con 0x7f15e0107630 2024-05-10T03:57:45.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.664+0000 7f15d67fc700 1 -- 172.21.15.150:0/736705098 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f15dc003a80 con 0x7f15e0107630 2024-05-10T03:57:45.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.664+0000 7f15e7318700 1 -- 172.21.15.150:0/736705098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e0102c10 con 0x7f15e0107630 2024-05-10T03:57:45.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.664+0000 7f15d67fc700 1 --2- 172.21.15.150:0/736705098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f15cc0387f0 0x7f15cc03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:45.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.664+0000 7f15d67fc700 1 -- 172.21.15.150:0/736705098 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f15dc04c360 con 0x7f15e0107630 2024-05-10T03:57:45.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.664+0000 7f15e48b3700 1 --2- 172.21.15.150:0/736705098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f15cc0387f0 0x7f15cc03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:45.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.664+0000 7f15e48b3700 1 --2- 172.21.15.150:0/736705098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f15cc0387f0 0x7f15cc03ad00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f15d0006fd0 tx=0x7f15d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:45.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.669+0000 7f15d67fc700 1 -- 172.21.15.150:0/736705098 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f15dc029360 con 0x7f15e0107630 2024-05-10T03:57:45.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.867+0000 7f15e7318700 1 -- 172.21.15.150:0/736705098 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15e010ba10 con 0x7f15cc0387f0 2024-05-10T03:57:45.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.868+0000 7f15d67fc700 1 -- 172.21.15.150:0/736705098 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f15e010ba10 con 0x7f15cc0387f0 2024-05-10T03:57:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:57:45.872 INFO:teuthology.orchestra.run.smithi150.stdout:[{"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2024-05-10T03:57:45.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.871+0000 7f15e7318700 1 -- 172.21.15.150:0/736705098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f15cc0387f0 msgr2=0x7f15cc03ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:45.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.871+0000 7f15e7318700 1 --2- 172.21.15.150:0/736705098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f15cc0387f0 0x7f15cc03ad00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f15d0006fd0 tx=0x7f15d0006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:45.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.871+0000 7f15e7318700 1 -- 172.21.15.150:0/736705098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15e0107630 msgr2=0x7f15e0197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:45.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.871+0000 7f15e7318700 1 --2- 172.21.15.150:0/736705098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15e0107630 0x7f15e0197120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00b5c0 tx=0x7f15dc004ff0 comp rx=0 tx=0).stop 2024-05-10T03:57:45.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.871+0000 7f15e7318700 1 -- 172.21.15.150:0/736705098 shutdown_connections 2024-05-10T03:57:45.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.871+0000 7f15e7318700 1 --2- 172.21.15.150:0/736705098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f15cc0387f0 0x7f15cc03ad00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:45.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.871+0000 7f15e7318700 1 --2- 172.21.15.150:0/736705098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15e0107630 0x7f15e0197120 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:45.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.871+0000 7f15e7318700 1 -- 172.21.15.150:0/736705098 >> 172.21.15.150:0/736705098 conn(0x7f15e0075670 msgr2=0x7f15e00fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:45.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.871+0000 7f15e7318700 1 -- 172.21.15.150:0/736705098 shutdown_connections 2024-05-10T03:57:45.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:45.871+0000 7f15e7318700 1 -- 172.21.15.150:0/736705098 wait complete. 2024-05-10T03:57:46.473 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi181 2024-05-10T03:57:46.474 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T03:57:46.474 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.conf 2024-05-10T03:57:46.494 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T03:57:46.494 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T03:57:46.554 INFO:tasks.cephadm:Adding host smithi181 to orchestrator... 2024-05-10T03:57:46.554 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch host add smithi181 2024-05-10T03:57:46.771 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T03:57:46.874 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:46 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:46.875 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:46 smithi150 ceph-mon[42477]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T03:57:47.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.703+0000 7fec3c80f700 1 -- 172.21.15.150:0/890004210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec38102610 msgr2=0x7fec38102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:47.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.703+0000 7fec3c80f700 1 --2- 172.21.15.150:0/890004210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec38102610 0x7fec38102a20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fec20009910 tx=0x7fec20009c20 comp rx=0 tx=0).stop 2024-05-10T03:57:47.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.703+0000 7fec3c80f700 1 -- 172.21.15.150:0/890004210 shutdown_connections 2024-05-10T03:57:47.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.703+0000 7fec3c80f700 1 --2- 172.21.15.150:0/890004210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec38102610 0x7fec38102a20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:47.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.703+0000 7fec3c80f700 1 -- 172.21.15.150:0/890004210 >> 172.21.15.150:0/890004210 conn(0x7fec38075f10 msgr2=0x7fec38076350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:47.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.703+0000 7fec3c80f700 1 -- 172.21.15.150:0/890004210 shutdown_connections 2024-05-10T03:57:47.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.703+0000 7fec3c80f700 1 -- 172.21.15.150:0/890004210 wait complete. 2024-05-10T03:57:47.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.704+0000 7fec3c80f700 1 Processor -- start 2024-05-10T03:57:47.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.704+0000 7fec3c80f700 1 -- start start 2024-05-10T03:57:47.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.704+0000 7fec3c80f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec38102610 0x7fec381970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:47.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.704+0000 7fec3c80f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec38197600 con 0x7fec38102610 2024-05-10T03:57:47.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.704+0000 7fec3659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec38102610 0x7fec381970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:47.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.704+0000 7fec3659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec38102610 0x7fec381970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52706/0 (socket says 172.21.15.150:52706) 2024-05-10T03:57:47.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.704+0000 7fec3659c700 1 -- 172.21.15.150:0/2734845132 learned_addr learned my addr 172.21.15.150:0/2734845132 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:47.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.705+0000 7fec3659c700 1 -- 172.21.15.150:0/2734845132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec200095f0 con 0x7fec38102610 2024-05-10T03:57:47.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.705+0000 7fec3659c700 1 --2- 172.21.15.150:0/2734845132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec38102610 0x7fec381970c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fec20004f40 tx=0x7fec20005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:47.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.705+0000 7fec2f7fe700 1 -- 172.21.15.150:0/2734845132 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec2001d070 con 0x7fec38102610 2024-05-10T03:57:47.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.705+0000 7fec2f7fe700 1 -- 172.21.15.150:0/2734845132 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec20004370 con 0x7fec38102610 2024-05-10T03:57:47.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.705+0000 7fec2f7fe700 1 -- 172.21.15.150:0/2734845132 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec20003970 con 0x7fec38102610 2024-05-10T03:57:47.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.705+0000 7fec3c80f700 1 -- 172.21.15.150:0/2734845132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec38197800 con 0x7fec38102610 2024-05-10T03:57:47.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.705+0000 7fec3c80f700 1 -- 172.21.15.150:0/2734845132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec38197c20 con 0x7fec38102610 2024-05-10T03:57:47.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.706+0000 7fec2f7fe700 1 -- 172.21.15.150:0/2734845132 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fec20003c30 con 0x7fec38102610 2024-05-10T03:57:47.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.706+0000 7fec2f7fe700 1 --2- 172.21.15.150:0/2734845132 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fec240387a0 0x7fec2403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:47.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.706+0000 7fec2f7fe700 1 -- 172.21.15.150:0/2734845132 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fec200044e0 con 0x7fec38102610 2024-05-10T03:57:47.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.706+0000 7fec35d9b700 1 --2- 172.21.15.150:0/2734845132 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fec240387a0 0x7fec2403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:47.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.706+0000 7fec3c80f700 1 -- 172.21.15.150:0/2734845132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec38068dd0 con 0x7fec38102610 2024-05-10T03:57:47.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.707+0000 7fec35d9b700 1 --2- 172.21.15.150:0/2734845132 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fec240387a0 0x7fec2403acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fec28006fd0 tx=0x7fec28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:47.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.712+0000 7fec2f7fe700 1 -- 172.21.15.150:0/2734845132 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fec20026020 con 0x7fec38102610 2024-05-10T03:57:47.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:47.912+0000 7fec3c80f700 1 -- 172.21.15.150:0/2734845132 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}) v1 -- 0x7fec38198490 con 0x7fec240387a0 2024-05-10T03:57:49.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:49.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:49.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:49.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:49.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:57:49.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:57:49.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:57:49.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:48 smithi150 ceph-mon[42477]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-10T03:57:49.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:48 smithi150 ceph-mon[42477]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:49.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:49.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:49.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:50.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:50.023+0000 7fec2f7fe700 1 -- 172.21.15.150:0/2734845132 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fec38198490 con 0x7fec240387a0 2024-05-10T03:57:50.028 INFO:teuthology.orchestra.run.smithi150.stdout:Added host 'smithi181' with addr '172.21.15.181' 2024-05-10T03:57:50.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:50.025+0000 7fec3c80f700 1 -- 172.21.15.150:0/2734845132 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fec240387a0 msgr2=0x7fec2403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:50.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:50.025+0000 7fec3c80f700 1 --2- 172.21.15.150:0/2734845132 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fec240387a0 0x7fec2403acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fec28006fd0 tx=0x7fec28006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:50.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:50.026+0000 7fec3c80f700 1 -- 172.21.15.150:0/2734845132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec38102610 msgr2=0x7fec381970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:50.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:50.026+0000 7fec3c80f700 1 --2- 172.21.15.150:0/2734845132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec38102610 0x7fec381970c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fec20004f40 tx=0x7fec20005020 comp rx=0 tx=0).stop 2024-05-10T03:57:50.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:50.026+0000 7fec3c80f700 1 -- 172.21.15.150:0/2734845132 shutdown_connections 2024-05-10T03:57:50.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:50.026+0000 7fec3c80f700 1 --2- 172.21.15.150:0/2734845132 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fec240387a0 0x7fec2403acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:50.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:50.026+0000 7fec3c80f700 1 --2- 172.21.15.150:0/2734845132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec38102610 0x7fec381970c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:50.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:50.026+0000 7fec3c80f700 1 -- 172.21.15.150:0/2734845132 >> 172.21.15.150:0/2734845132 conn(0x7fec38075f10 msgr2=0x7fec38103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:50.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:50.026+0000 7fec3c80f700 1 -- 172.21.15.150:0/2734845132 shutdown_connections 2024-05-10T03:57:50.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:50.026+0000 7fec3c80f700 1 -- 172.21.15.150:0/2734845132 wait complete. 2024-05-10T03:57:50.321 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:50 smithi150 ceph-mon[42477]: Updating smithi161:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:57:50.321 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:50 smithi150 ceph-mon[42477]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-05-10T03:57:50.321 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:50 smithi150 ceph-mon[42477]: Deploying cephadm binary to smithi181 2024-05-10T03:57:50.322 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:50 smithi150 ceph-mon[42477]: Updating smithi161:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T03:57:50.322 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:50 smithi150 ceph-mon[42477]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:57:50.322 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:50 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:50.355 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-10T03:57:50.571 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T03:57:51.376 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:51 smithi150 ceph-mon[42477]: Added host smithi181 2024-05-10T03:57:51.376 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:51 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:57:51.376 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:51 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:51.376 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:51 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:51.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.554+0000 7fdbe0467700 1 -- 172.21.15.150:0/412868207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd8102610 msgr2=0x7fdbd8102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:51.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.554+0000 7fdbe0467700 1 --2- 172.21.15.150:0/412868207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd8102610 0x7fdbd8102a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009910 tx=0x7fdbc8009c20 comp rx=0 tx=0).stop 2024-05-10T03:57:51.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.554+0000 7fdbe0467700 1 -- 172.21.15.150:0/412868207 shutdown_connections 2024-05-10T03:57:51.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.554+0000 7fdbe0467700 1 --2- 172.21.15.150:0/412868207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd8102610 0x7fdbd8102a20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:51.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.554+0000 7fdbe0467700 1 -- 172.21.15.150:0/412868207 >> 172.21.15.150:0/412868207 conn(0x7fdbd8075f10 msgr2=0x7fdbd8076350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:51.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.554+0000 7fdbe0467700 1 -- 172.21.15.150:0/412868207 shutdown_connections 2024-05-10T03:57:51.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.554+0000 7fdbe0467700 1 -- 172.21.15.150:0/412868207 wait complete. 2024-05-10T03:57:51.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.555+0000 7fdbe0467700 1 Processor -- start 2024-05-10T03:57:51.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.555+0000 7fdbe0467700 1 -- start start 2024-05-10T03:57:51.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.555+0000 7fdbe0467700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd8102610 0x7fdbd8100300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:51.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.555+0000 7fdbe0467700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd8102170 con 0x7fdbd8102610 2024-05-10T03:57:51.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.555+0000 7fdbde203700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd8102610 0x7fdbd8100300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:51.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.555+0000 7fdbde203700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd8102610 0x7fdbd8100300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43936/0 (socket says 172.21.15.150:43936) 2024-05-10T03:57:51.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.555+0000 7fdbde203700 1 -- 172.21.15.150:0/4286047010 learned_addr learned my addr 172.21.15.150:0/4286047010 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:51.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.555+0000 7fdbde203700 1 -- 172.21.15.150:0/4286047010 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc80095f0 con 0x7fdbd8102610 2024-05-10T03:57:51.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.555+0000 7fdbde203700 1 --2- 172.21.15.150:0/4286047010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd8102610 0x7fdbd8100300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8006010 tx=0x7fdbc8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:51.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.556+0000 7fdbcf7fe700 1 -- 172.21.15.150:0/4286047010 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbc801d070 con 0x7fdbd8102610 2024-05-10T03:57:51.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.556+0000 7fdbcf7fe700 1 -- 172.21.15.150:0/4286047010 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbc800bc10 con 0x7fdbd8102610 2024-05-10T03:57:51.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.556+0000 7fdbe0467700 1 -- 172.21.15.150:0/4286047010 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd8100840 con 0x7fdbd8102610 2024-05-10T03:57:51.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.556+0000 7fdbcf7fe700 1 -- 172.21.15.150:0/4286047010 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbc80038f0 con 0x7fdbd8102610 2024-05-10T03:57:51.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.556+0000 7fdbe0467700 1 -- 172.21.15.150:0/4286047010 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd8100c60 con 0x7fdbd8102610 2024-05-10T03:57:51.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.556+0000 7fdbcf7fe700 1 -- 172.21.15.150:0/4286047010 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fdbc8003b60 con 0x7fdbd8102610 2024-05-10T03:57:51.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.557+0000 7fdbcf7fe700 1 --2- 172.21.15.150:0/4286047010 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdbc40387a0 0x7fdbc403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:51.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.557+0000 7fdbcf7fe700 1 -- 172.21.15.150:0/4286047010 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdbc804d3e0 con 0x7fdbd8102610 2024-05-10T03:57:51.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.557+0000 7fdbe0467700 1 -- 172.21.15.150:0/4286047010 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd8068dd0 con 0x7fdbd8102610 2024-05-10T03:57:51.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.557+0000 7fdbdda02700 1 --2- 172.21.15.150:0/4286047010 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdbc40387a0 0x7fdbc403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:51.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.557+0000 7fdbdda02700 1 --2- 172.21.15.150:0/4286047010 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdbc40387a0 0x7fdbc403acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4006fd0 tx=0x7fdbd4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:51.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.562+0000 7fdbcf7fe700 1 -- 172.21.15.150:0/4286047010 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdbc8026020 con 0x7fdbd8102610 2024-05-10T03:57:51.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.761+0000 7fdbe0467700 1 -- 172.21.15.150:0/4286047010 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdbd8101c60 con 0x7fdbc40387a0 2024-05-10T03:57:51.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.762+0000 7fdbcf7fe700 1 -- 172.21.15.150:0/4286047010 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7fdbd8101c60 con 0x7fdbc40387a0 2024-05-10T03:57:51.765 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T03:57:51.765 INFO:teuthology.orchestra.run.smithi150.stdout:[{"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}, {"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}] 2024-05-10T03:57:51.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.764+0000 7fdbe0467700 1 -- 172.21.15.150:0/4286047010 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdbc40387a0 msgr2=0x7fdbc403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:51.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.764+0000 7fdbe0467700 1 --2- 172.21.15.150:0/4286047010 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdbc40387a0 0x7fdbc403acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4006fd0 tx=0x7fdbd4006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:51.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.764+0000 7fdbe0467700 1 -- 172.21.15.150:0/4286047010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd8102610 msgr2=0x7fdbd8100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:51.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.764+0000 7fdbe0467700 1 --2- 172.21.15.150:0/4286047010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd8102610 0x7fdbd8100300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8006010 tx=0x7fdbc8004ff0 comp rx=0 tx=0).stop 2024-05-10T03:57:51.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.764+0000 7fdbe0467700 1 -- 172.21.15.150:0/4286047010 shutdown_connections 2024-05-10T03:57:51.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.764+0000 7fdbe0467700 1 --2- 172.21.15.150:0/4286047010 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdbc40387a0 0x7fdbc403acb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:51.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.764+0000 7fdbe0467700 1 --2- 172.21.15.150:0/4286047010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbd8102610 0x7fdbd8100300 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:51.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.764+0000 7fdbe0467700 1 -- 172.21.15.150:0/4286047010 >> 172.21.15.150:0/4286047010 conn(0x7fdbd8075f10 msgr2=0x7fdbd8103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:51.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.764+0000 7fdbe0467700 1 -- 172.21.15.150:0/4286047010 shutdown_connections 2024-05-10T03:57:51.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:51.764+0000 7fdbe0467700 1 -- 172.21.15.150:0/4286047010 wait complete. 2024-05-10T03:57:52.071 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:52 smithi150 ceph-mon[42477]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:57:52.071 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:52.142 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-10T03:57:52.142 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-10T03:57:52.358 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T03:57:53.153 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:53 smithi150 ceph-mon[42477]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T03:57:53.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.328+0000 7f0dea79f700 1 -- 172.21.15.150:0/2628364628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de4102610 msgr2=0x7f0de4102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:53.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.328+0000 7f0dea79f700 1 --2- 172.21.15.150:0/2628364628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de4102610 0x7f0de4102a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4009910 tx=0x7f0dd4009c20 comp rx=0 tx=0).stop 2024-05-10T03:57:53.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.328+0000 7f0dea79f700 1 -- 172.21.15.150:0/2628364628 shutdown_connections 2024-05-10T03:57:53.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.328+0000 7f0dea79f700 1 --2- 172.21.15.150:0/2628364628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de4102610 0x7f0de4102a20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:53.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.328+0000 7f0dea79f700 1 -- 172.21.15.150:0/2628364628 >> 172.21.15.150:0/2628364628 conn(0x7f0de4075f10 msgr2=0x7f0de4076350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:53.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.328+0000 7f0dea79f700 1 -- 172.21.15.150:0/2628364628 shutdown_connections 2024-05-10T03:57:53.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.328+0000 7f0dea79f700 1 -- 172.21.15.150:0/2628364628 wait complete. 2024-05-10T03:57:53.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.328+0000 7f0dea79f700 1 Processor -- start 2024-05-10T03:57:53.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.328+0000 7f0dea79f700 1 -- start start 2024-05-10T03:57:53.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.329+0000 7f0dea79f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de4102610 0x7f0de41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:53.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.329+0000 7f0dea79f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de4199810 con 0x7f0de4102610 2024-05-10T03:57:53.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.329+0000 7f0de3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de4102610 0x7f0de41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:53.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.329+0000 7f0de3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de4102610 0x7f0de41992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43954/0 (socket says 172.21.15.150:43954) 2024-05-10T03:57:53.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.329+0000 7f0de3fff700 1 -- 172.21.15.150:0/2688410777 learned_addr learned my addr 172.21.15.150:0/2688410777 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:57:53.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.329+0000 7f0de3fff700 1 -- 172.21.15.150:0/2688410777 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd40095f0 con 0x7f0de4102610 2024-05-10T03:57:53.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.329+0000 7f0de3fff700 1 --2- 172.21.15.150:0/2688410777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de4102610 0x7f0de41992d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4004f40 tx=0x7f0dd4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.329+0000 7f0de17fa700 1 -- 172.21.15.150:0/2688410777 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0dd401d070 con 0x7f0de4102610 2024-05-10T03:57:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.329+0000 7f0de17fa700 1 -- 172.21.15.150:0/2688410777 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dd4004370 con 0x7f0de4102610 2024-05-10T03:57:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.330+0000 7f0dea79f700 1 -- 172.21.15.150:0/2688410777 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de4199a10 con 0x7f0de4102610 2024-05-10T03:57:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.330+0000 7f0de17fa700 1 -- 172.21.15.150:0/2688410777 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0dd4003970 con 0x7f0de4102610 2024-05-10T03:57:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.330+0000 7f0dea79f700 1 -- 172.21.15.150:0/2688410777 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de4199e30 con 0x7f0de4102610 2024-05-10T03:57:53.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.330+0000 7f0de17fa700 1 -- 172.21.15.150:0/2688410777 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f0dd4003c30 con 0x7f0de4102610 2024-05-10T03:57:53.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.331+0000 7f0de17fa700 1 --2- 172.21.15.150:0/2688410777 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0dcc0387a0 0x7f0dcc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:53.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.331+0000 7f0de17fa700 1 -- 172.21.15.150:0/2688410777 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0dd404d560 con 0x7f0de4102610 2024-05-10T03:57:53.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.331+0000 7f0de37fe700 1 --2- 172.21.15.150:0/2688410777 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0dcc0387a0 0x7f0dcc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:53.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.331+0000 7f0dea79f700 1 -- 172.21.15.150:0/2688410777 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de40ff390 con 0x7f0de4102610 2024-05-10T03:57:53.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.331+0000 7f0de37fe700 1 --2- 172.21.15.150:0/2688410777 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0dcc0387a0 0x7f0dcc03acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8006fd0 tx=0x7f0dd8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:53.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.336+0000 7f0de17fa700 1 -- 172.21.15.150:0/2688410777 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0de40ff390 con 0x7f0de4102610 2024-05-10T03:57:53.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:53.533+0000 7f0dea79f700 1 -- 172.21.15.150:0/2688410777 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f0de4068dd0 con 0x7f0de4102610 2024-05-10T03:57:54.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:54.033+0000 7f0de17fa700 1 -- 172.21.15.150:0/2688410777 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0dd4026070 con 0x7f0de4102610 2024-05-10T03:57:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:54.035+0000 7f0dea79f700 1 -- 172.21.15.150:0/2688410777 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0dcc0387a0 msgr2=0x7f0dcc03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:54.035+0000 7f0dea79f700 1 --2- 172.21.15.150:0/2688410777 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0dcc0387a0 0x7f0dcc03acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8006fd0 tx=0x7f0dd8006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:54.035+0000 7f0dea79f700 1 -- 172.21.15.150:0/2688410777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de4102610 msgr2=0x7f0de41992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:54.035+0000 7f0dea79f700 1 --2- 172.21.15.150:0/2688410777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de4102610 0x7f0de41992d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4004f40 tx=0x7f0dd4005020 comp rx=0 tx=0).stop 2024-05-10T03:57:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:54.035+0000 7f0dea79f700 1 -- 172.21.15.150:0/2688410777 shutdown_connections 2024-05-10T03:57:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:54.035+0000 7f0dea79f700 1 --2- 172.21.15.150:0/2688410777 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0dcc0387a0 0x7f0dcc03acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:54.035+0000 7f0dea79f700 1 --2- 172.21.15.150:0/2688410777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de4102610 0x7f0de41992d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:54.035+0000 7f0dea79f700 1 -- 172.21.15.150:0/2688410777 >> 172.21.15.150:0/2688410777 conn(0x7f0de4075f10 msgr2=0x7f0de4107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:54.035+0000 7f0dea79f700 1 -- 172.21.15.150:0/2688410777 shutdown_connections 2024-05-10T03:57:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:57:54.036+0000 7f0dea79f700 1 -- 172.21.15.150:0/2688410777 wait complete. 2024-05-10T03:57:54.040 INFO:teuthology.orchestra.run.smithi150.stderr:adjusted tunables profile to default 2024-05-10T03:57:54.296 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:54 smithi150 ceph-mon[42477]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:57:54.296 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:54 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2688410777' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T03:57:54.296 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:54 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:54.296 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:54 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:54.296 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:54 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:54.296 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:54 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:54.296 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:54 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:57:54.296 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:54 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:57:54.296 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:54 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:57:54.407 INFO:tasks.cephadm:Adding mon.a on smithi150 2024-05-10T03:57:54.407 INFO:tasks.cephadm:Adding mon.b on smithi161 2024-05-10T03:57:54.408 INFO:tasks.cephadm:Adding mon.c on smithi181 2024-05-10T03:57:54.408 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c' 2024-05-10T03:57:54.655 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:57:55.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:55 smithi150 ceph-mon[42477]: Updating smithi181:/etc/ceph/ceph.conf 2024-05-10T03:57:55.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:55 smithi150 ceph-mon[42477]: Updating smithi181:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:57:55.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:55 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2688410777' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T03:57:55.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:55 smithi150 ceph-mon[42477]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T03:57:55.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:55 smithi150 ceph-mon[42477]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-05-10T03:57:55.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:55.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:55.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:55.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.625+0000 7f7b7f809700 1 -- 172.21.15.181:0/2061701942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b781064a0 msgr2=0x7f7b781068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:55.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.625+0000 7f7b7f809700 1 --2- 172.21.15.181:0/2061701942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b781064a0 0x7f7b781068b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009910 tx=0x7f7b74009c20 comp rx=0 tx=0).stop 2024-05-10T03:57:55.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.625+0000 7f7b7f809700 1 -- 172.21.15.181:0/2061701942 shutdown_connections 2024-05-10T03:57:55.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.625+0000 7f7b7f809700 1 --2- 172.21.15.181:0/2061701942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b781064a0 0x7f7b781068b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:55.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.625+0000 7f7b7f809700 1 -- 172.21.15.181:0/2061701942 >> 172.21.15.181:0/2061701942 conn(0x7f7b78075670 msgr2=0x7f7b78075ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:55.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.625+0000 7f7b7f809700 1 -- 172.21.15.181:0/2061701942 shutdown_connections 2024-05-10T03:57:55.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.625+0000 7f7b7f809700 1 -- 172.21.15.181:0/2061701942 wait complete. 2024-05-10T03:57:55.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.626+0000 7f7b7f809700 1 Processor -- start 2024-05-10T03:57:55.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.626+0000 7f7b7f809700 1 -- start start 2024-05-10T03:57:55.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.626+0000 7f7b7f809700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b781064a0 0x7f7b78197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:55.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.626+0000 7f7b7f809700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78197680 con 0x7f7b781064a0 2024-05-10T03:57:55.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.626+0000 7f7b7d5a5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b781064a0 0x7f7b78197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:55.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.626+0000 7f7b7d5a5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b781064a0 0x7f7b78197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.181:51602/0 (socket says 172.21.15.181:51602) 2024-05-10T03:57:55.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.626+0000 7f7b7d5a5700 1 -- 172.21.15.181:0/198771006 learned_addr learned my addr 172.21.15.181:0/198771006 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T03:57:55.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.627+0000 7f7b7d5a5700 1 -- 172.21.15.181:0/198771006 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b740095f0 con 0x7f7b781064a0 2024-05-10T03:57:55.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.627+0000 7f7b7d5a5700 1 --2- 172.21.15.181:0/198771006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b781064a0 0x7f7b78197140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b74004f40 tx=0x7f7b74005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:55.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.627+0000 7f7b6e7fc700 1 -- 172.21.15.181:0/198771006 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b7401d070 con 0x7f7b781064a0 2024-05-10T03:57:55.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.627+0000 7f7b7f809700 1 -- 172.21.15.181:0/198771006 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b78197880 con 0x7f7b781064a0 2024-05-10T03:57:55.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.627+0000 7f7b6e7fc700 1 -- 172.21.15.181:0/198771006 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b7400be20 con 0x7f7b781064a0 2024-05-10T03:57:55.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.627+0000 7f7b6e7fc700 1 -- 172.21.15.181:0/198771006 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b74022d30 con 0x7f7b781064a0 2024-05-10T03:57:55.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.627+0000 7f7b7f809700 1 -- 172.21.15.181:0/198771006 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b78197ca0 con 0x7f7b781064a0 2024-05-10T03:57:55.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.628+0000 7f7b7f809700 1 -- 172.21.15.181:0/198771006 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b78101ce0 con 0x7f7b781064a0 2024-05-10T03:57:55.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.628+0000 7f7b6e7fc700 1 -- 172.21.15.181:0/198771006 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f7b7400f460 con 0x7f7b781064a0 2024-05-10T03:57:55.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.628+0000 7f7b6e7fc700 1 --2- 172.21.15.181:0/198771006 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b640387f0 0x7f7b6403ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:55.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.628+0000 7f7b6e7fc700 1 -- 172.21.15.181:0/198771006 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7b7404cc10 con 0x7f7b781064a0 2024-05-10T03:57:55.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.629+0000 7f7b7cda4700 1 --2- 172.21.15.181:0/198771006 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b640387f0 0x7f7b6403ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:55.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.629+0000 7f7b7cda4700 1 --2- 172.21.15.181:0/198771006 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b640387f0 0x7f7b6403ad00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7b68006fd0 tx=0x7f7b68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:55.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.634+0000 7f7b6e7fc700 1 -- 172.21.15.181:0/198771006 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7b7402a950 con 0x7f7b781064a0 2024-05-10T03:57:55.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.836+0000 7f7b7f809700 1 -- 172.21.15.181:0/198771006 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b7810a870 con 0x7f7b640387f0 2024-05-10T03:57:55.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.844+0000 7f7b6e7fc700 1 -- 172.21.15.181:0/198771006 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7b7810a870 con 0x7f7b640387f0 2024-05-10T03:57:55.846 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mon update... 2024-05-10T03:57:55.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.846+0000 7f7b7f809700 1 -- 172.21.15.181:0/198771006 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b640387f0 msgr2=0x7f7b6403ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:55.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.846+0000 7f7b7f809700 1 --2- 172.21.15.181:0/198771006 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b640387f0 0x7f7b6403ad00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7b68006fd0 tx=0x7f7b68006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:55.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.846+0000 7f7b7f809700 1 -- 172.21.15.181:0/198771006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b781064a0 msgr2=0x7f7b78197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:55.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.846+0000 7f7b7f809700 1 --2- 172.21.15.181:0/198771006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b781064a0 0x7f7b78197140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b74004f40 tx=0x7f7b74005020 comp rx=0 tx=0).stop 2024-05-10T03:57:55.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.846+0000 7f7b7f809700 1 -- 172.21.15.181:0/198771006 shutdown_connections 2024-05-10T03:57:55.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.846+0000 7f7b7f809700 1 --2- 172.21.15.181:0/198771006 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b640387f0 0x7f7b6403ad00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:55.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.846+0000 7f7b7f809700 1 --2- 172.21.15.181:0/198771006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b781064a0 0x7f7b78197140 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:55.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.846+0000 7f7b7f809700 1 -- 172.21.15.181:0/198771006 >> 172.21.15.181:0/198771006 conn(0x7f7b78075670 msgr2=0x7f7b780fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:55.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.846+0000 7f7b7f809700 1 -- 172.21.15.181:0/198771006 shutdown_connections 2024-05-10T03:57:55.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:55.846+0000 7f7b7f809700 1 -- 172.21.15.181:0/198771006 wait complete. 2024-05-10T03:57:56.277 DEBUG:teuthology.orchestra.run.smithi161:mon.b> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.b.service 2024-05-10T03:57:56.280 DEBUG:teuthology.orchestra.run.smithi181:mon.c> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.c.service 2024-05-10T03:57:56.282 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T03:57:56.282 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T03:57:56.314 INFO:journalctl@ceph.mon.c.smithi181.stdout:-- Logs begin at Fri 2024-05-10 03:40:19 UTC. -- 2024-05-10T03:57:56.316 INFO:journalctl@ceph.mon.b.smithi161.stdout:-- Logs begin at Fri 2024-05-10 03:40:24 UTC. -- 2024-05-10T03:57:56.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:56 smithi150 ceph-mon[42477]: Updating smithi181:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T03:57:56.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:56 smithi150 ceph-mon[42477]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:57:56.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:56 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:56.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:56 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:57:56.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:56 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:57:56.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:56 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:57:56.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:56 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:56.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:56 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:57:56.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:56 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:57:56.658 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:57:57.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:57 smithi150 ceph-mon[42477]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:57.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:57 smithi150 ceph-mon[42477]: Saving service mon spec with placement smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c;count:3 2024-05-10T03:57:57.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:57 smithi150 ceph-mon[42477]: Deploying daemon mon.c on smithi181 2024-05-10T03:57:58.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.229+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2785573586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad94100650 msgr2=0x7fad94100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:58.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.229+0000 7fad9bcd6700 1 --2- 172.21.15.181:0/2785573586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad94100650 0x7fad94100a60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fad90009910 tx=0x7fad90009c20 comp rx=0 tx=0).stop 2024-05-10T03:57:58.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.229+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2785573586 shutdown_connections 2024-05-10T03:57:58.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.229+0000 7fad9bcd6700 1 --2- 172.21.15.181:0/2785573586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad94100650 0x7fad94100a60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:58.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.229+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2785573586 >> 172.21.15.181:0/2785573586 conn(0x7fad940fbee0 msgr2=0x7fad940fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:58.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.229+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2785573586 shutdown_connections 2024-05-10T03:57:58.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.229+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2785573586 wait complete. 2024-05-10T03:57:58.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.230+0000 7fad9bcd6700 1 Processor -- start 2024-05-10T03:57:58.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.230+0000 7fad9bcd6700 1 -- start start 2024-05-10T03:57:58.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.230+0000 7fad9bcd6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad94100650 0x7fad9419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:58.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.230+0000 7fad9bcd6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9419dc10 con 0x7fad94100650 2024-05-10T03:57:58.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.230+0000 7fad99a72700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad94100650 0x7fad9419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:58.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.230+0000 7fad99a72700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad94100650 0x7fad9419d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.181:42476/0 (socket says 172.21.15.181:42476) 2024-05-10T03:57:58.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.230+0000 7fad99a72700 1 -- 172.21.15.181:0/2211953288 learned_addr learned my addr 172.21.15.181:0/2211953288 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T03:57:58.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.231+0000 7fad99a72700 1 -- 172.21.15.181:0/2211953288 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad900095f0 con 0x7fad94100650 2024-05-10T03:57:58.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.231+0000 7fad99a72700 1 --2- 172.21.15.181:0/2211953288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad94100650 0x7fad9419d6d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad90004f40 tx=0x7fad90005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:58.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.231+0000 7fad8affd700 1 -- 172.21.15.181:0/2211953288 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad9001d070 con 0x7fad94100650 2024-05-10T03:57:58.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.231+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2211953288 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9419de70 con 0x7fad94100650 2024-05-10T03:57:58.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.231+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2211953288 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9419e2f0 con 0x7fad94100650 2024-05-10T03:57:58.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.231+0000 7fad8affd700 1 -- 172.21.15.181:0/2211953288 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad9000bbb0 con 0x7fad94100650 2024-05-10T03:57:58.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.231+0000 7fad8affd700 1 -- 172.21.15.181:0/2211953288 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad90003810 con 0x7fad94100650 2024-05-10T03:57:58.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.232+0000 7fad8affd700 1 -- 172.21.15.181:0/2211953288 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fad90003a60 con 0x7fad94100650 2024-05-10T03:57:58.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.232+0000 7fad8affd700 1 --2- 172.21.15.181:0/2211953288 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fad800387f0 0x7fad8003ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:57:58.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.232+0000 7fad8affd700 1 -- 172.21.15.181:0/2211953288 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fad9004d300 con 0x7fad94100650 2024-05-10T03:57:58.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.232+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2211953288 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad78005320 con 0x7fad94100650 2024-05-10T03:57:58.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.233+0000 7fad99271700 1 --2- 172.21.15.181:0/2211953288 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fad800387f0 0x7fad8003ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:57:58.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.233+0000 7fad99271700 1 --2- 172.21.15.181:0/2211953288 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fad800387f0 0x7fad8003ad00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad84006fd0 tx=0x7fad84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:57:58.240 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.238+0000 7fad8affd700 1 -- 172.21.15.181:0/2211953288 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fad90026080 con 0x7fad94100650 2024-05-10T03:57:58.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:58 smithi150 ceph-mon[42477]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:57:58.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.531+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2211953288 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fad78005190 con 0x7fad94100650 2024-05-10T03:57:58.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.531+0000 7fad8affd700 1 -- 172.21.15.181:0/2211953288 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7fad9004b090 con 0x7fad94100650 2024-05-10T03:57:58.534 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:57:58.534 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","modified":"2024-05-10T03:56:37.662524Z","created":"2024-05-10T03:56:37.662524Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-10T03:57:58.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.534+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2211953288 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fad800387f0 msgr2=0x7fad8003ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:58.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.534+0000 7fad9bcd6700 1 --2- 172.21.15.181:0/2211953288 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fad800387f0 0x7fad8003ad00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad84006fd0 tx=0x7fad84006e40 comp rx=0 tx=0).stop 2024-05-10T03:57:58.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.534+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2211953288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad94100650 msgr2=0x7fad9419d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:57:58.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.534+0000 7fad9bcd6700 1 --2- 172.21.15.181:0/2211953288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad94100650 0x7fad9419d6d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad90004f40 tx=0x7fad90005020 comp rx=0 tx=0).stop 2024-05-10T03:57:58.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.534+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2211953288 shutdown_connections 2024-05-10T03:57:58.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.534+0000 7fad9bcd6700 1 --2- 172.21.15.181:0/2211953288 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fad800387f0 0x7fad8003ad00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:58.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.534+0000 7fad9bcd6700 1 --2- 172.21.15.181:0/2211953288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad94100650 0x7fad9419d6d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:57:58.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.534+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2211953288 >> 172.21.15.181:0/2211953288 conn(0x7fad940fbee0 msgr2=0x7fad94106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:57:58.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.534+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2211953288 shutdown_connections 2024-05-10T03:57:58.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:57:58.534+0000 7fad9bcd6700 1 -- 172.21.15.181:0/2211953288 wait complete. 2024-05-10T03:57:58.537 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-05-10T03:57:58.781 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:58 smithi181 systemd[1]: Starting Ceph mon.c for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T03:57:59.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:57:59 smithi150 ceph-mon[42477]: from='client.? 172.21.15.181:0/2211953288' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T03:57:59.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 bash[46565]: 7cbe92669baf9fdba799534239bd67234f783d9a7809e3c7d9e1fa6d12cc672d 2024-05-10T03:57:59.740 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 systemd[1]: Started Ceph mon.c for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T03:57:59.740 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T03:57:59.740 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable), process ceph-mon, pid 7 2024-05-10T03:57:59.740 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: pidfile_write: ignore empty --pid-file 2024-05-10T03:57:59.740 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: load: jerasure load: lrc 2024-05-10T03:57:59.740 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: RocksDB version: 7.9.2 2024-05-10T03:57:59.740 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Git sha 0 2024-05-10T03:57:59.740 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Compile date 2024-05-10 00:51:33 2024-05-10T03:57:59.740 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: DB SUMMARY 2024-05-10T03:57:59.740 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: DB Session ID: DOHX0M35C1MOK4MOM2ID 2024-05-10T03:57:59.740 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: CURRENT file: CURRENT 2024-05-10T03:57:59.740 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T03:57:59.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-10T03:57:59.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-10T03:57:59.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-10T03:57:59.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.error_if_exists: 0 2024-05-10T03:57:59.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.create_if_missing: 0 2024-05-10T03:57:59.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.paranoid_checks: 1 2024-05-10T03:57:59.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T03:57:59.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T03:57:59.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T03:57:59.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.env: 0x5609749f4460 2024-05-10T03:57:59.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T03:57:59.741 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.info_log: 0x5609767cd760 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.statistics: (nil) 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.use_fsync: 0 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_log_file_size: 0 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.allow_fallocate: 1 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.use_direct_reads: 0 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T03:57:59.742 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T03:57:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.db_log_dir: 2024-05-10T03:57:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.wal_dir: 2024-05-10T03:57:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T03:57:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T03:57:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T03:57:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T03:57:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T03:57:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T03:57:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T03:57:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T03:57:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.write_buffer_manager: 0x5609767110e0 2024-05-10T03:57:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T03:57:59.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T03:57:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T03:57:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T03:57:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T03:57:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T03:57:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T03:57:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T03:57:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.unordered_write: 0 2024-05-10T03:57:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T03:57:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T03:57:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T03:57:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T03:57:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.row_cache: None 2024-05-10T03:57:59.744 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.wal_filter: None 2024-05-10T03:57:59.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T03:57:59.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T03:57:59.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.two_write_queues: 0 2024-05-10T03:57:59.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T03:57:59.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.wal_compression: 0 2024-05-10T03:57:59.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.atomic_flush: 0 2024-05-10T03:57:59.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T03:57:59.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T03:57:59.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T03:57:59.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.log_readahead_size: 0 2024-05-10T03:57:59.745 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T03:57:59.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T03:57:59.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T03:57:59.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T03:57:59.746 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T03:57:59.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T03:57:59.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T03:57:59.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_background_jobs: 2 2024-05-10T03:57:59.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_background_compactions: -1 2024-05-10T03:57:59.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_subcompactions: 1 2024-05-10T03:57:59.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T03:57:59.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T03:57:59.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T03:57:59.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T03:57:59.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T03:57:59.748 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_open_files: -1 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_background_flushes: -1 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Compression algorithms supported: 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: kZSTD supported: 0 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: kXpressCompression supported: 0 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T03:57:59.749 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: kZlibCompression supported: 1 2024-05-10T03:57:59.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: kSnappyCompression supported: 1 2024-05-10T03:57:59.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: kLZ4Compression supported: 1 2024-05-10T03:57:59.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: kBZip2Compression supported: 0 2024-05-10T03:57:59.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T03:57:59.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T03:57:59.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-10T03:57:59.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T03:57:59.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T03:57:59.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.merge_operator: 2024-05-10T03:57:59.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compaction_filter: None 2024-05-10T03:57:59.750 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5609767cd880) 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout: cache_index_and_filter_blocks: 1 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout: pin_top_level_index_and_filter: 1 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout: index_type: 0 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout: data_block_index_type: 0 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout: index_shortening: 1 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout: checksum: 4 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout: no_block_cache: 0 2024-05-10T03:57:59.751 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache: 0x5609767931f0 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache_name: BinnedLRUCache 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache_options: 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: capacity : 536870912 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: num_shard_bits : 4 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: strict_capacity_limit : 0 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: high_pri_pool_ratio: 0.000 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache_compressed: (nil) 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: persistent_cache: (nil) 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_size: 4096 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_size_deviation: 10 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_restart_interval: 16 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: index_block_restart_interval: 1 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: metadata_block_size: 4096 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: partition_filters: 0 2024-05-10T03:57:59.752 INFO:journalctl@ceph.mon.c.smithi181.stdout: use_delta_encoding: 1 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout: filter_policy: bloomfilter 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout: whole_key_filtering: 1 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout: verify_compression: 0 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout: read_amp_bytes_per_bit: 0 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout: format_version: 5 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout: enable_index_compression: 1 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_align: 0 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout: max_auto_readahead_size: 262144 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout: prepopulate_block_cache: 0 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout: initial_auto_readahead_size: 8192 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compression: NoCompression 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T03:57:59.753 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.num_levels: 7 2024-05-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T03:57:59.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T03:57:59.756 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.table_properties_collectors: 2024-05-10T03:57:59.757 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.inplace_update_support: 0 2024-05-10T03:57:59.758 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T03:57:59.758 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T03:57:59.758 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T03:57:59.758 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T03:57:59.758 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.bloom_locality: 0 2024-05-10T03:57:59.758 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.max_successive_merges: 0 2024-05-10T03:57:59.758 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T03:57:59.758 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T03:57:59.758 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T03:57:59.758 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T03:57:59.758 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.ttl: 2592000 2024-05-10T03:57:59.758 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T03:57:59.758 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T03:57:59.759 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T03:57:59.759 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.enable_blob_files: false 2024-05-10T03:57:59.759 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.min_blob_size: 0 2024-05-10T03:57:59.759 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T03:57:59.759 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T03:57:59.759 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T03:57:59.759 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T03:57:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T03:57:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T03:57:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T03:57:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T03:57:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-10T03:57:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-10T03:57:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b6f26ba9-991c-4857-b205-a702f0b429cf 2024-05-10T03:57:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715313479260519, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-10T03:57:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-10T03:57:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715313479260955, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715313479, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b6f26ba9-991c-4857-b205-a702f0b429cf", "db_session_id": "DOHX0M35C1MOK4MOM2ID", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-10T03:57:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715313479261024, "job": 1, "event": "recovery_finished"} 2024-05-10T03:57:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-10T03:57:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T03:57:59.761 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560976725c00 2024-05-10T03:57:59.761 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: DB pointer 0x56097b514000 2024-05-10T03:57:59.761 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T03:57:59.761 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T03:57:59.761 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** DB Stats ** 2024-05-10T03:57:59.761 INFO:journalctl@ceph.mon.c.smithi181.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T03:57:59.761 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T03:57:59.761 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T03:57:59.761 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T03:57:59.761 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T03:57:59.761 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** Compaction Stats [default] ** 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** Compaction Stats [default] ** 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T03:57:59.762 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-05-10T03:57:59.763 INFO:journalctl@ceph.mon.c.smithi181.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T03:57:59.763 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-05-10T03:57:59.763 INFO:journalctl@ceph.mon.c.smithi181.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T03:57:59.763 INFO:journalctl@ceph.mon.c.smithi181.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T03:57:59.763 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T03:57:59.763 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T03:57:59.763 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T03:57:59.763 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T03:57:59.763 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T03:57:59.763 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T03:57:59.763 INFO:journalctl@ceph.mon.c.smithi181.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout: Block cache BinnedLRUCache@0x5609767931f0#7 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: using public_addr v2:172.21.15.181:0/0 -> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: starting mon.c rank -1 at public addrs [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] at bind addrs [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(???) e0 preinit fsid 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(synchronizing).mds e1 new map 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout: e1 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T03:57:59.764 INFO:journalctl@ceph.mon.c.smithi181.stdout: legacy client fscid: -1 2024-05-10T03:57:59.765 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-05-10T03:57:59.765 INFO:journalctl@ceph.mon.c.smithi181.stdout: No filesystems configured 2024-05-10T03:57:59.765 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-10T03:57:59.765 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-10T03:57:59.765 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-10T03:57:59.765 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-10T03:57:59.765 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-10T03:57:59.765 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-10T03:57:59.766 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T03:57:59.766 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T03:57:59.766 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T03:57:59.766 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T03:57:59.766 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:57:59.766 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2688410777' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T03:57:59.766 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:59.766 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:59.766 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:59.766 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:59.766 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:57:59.766 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:57:59.767 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:57:59.767 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: Updating smithi181:/etc/ceph/ceph.conf 2024-05-10T03:57:59.767 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: Updating smithi181:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:57:59.767 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2688410777' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T03:57:59.767 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T03:57:59.767 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-05-10T03:57:59.767 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:59.767 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:59.768 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:59.768 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: Updating smithi181:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T03:57:59.768 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:57:59.768 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:59.768 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:57:59.768 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:57:59.768 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:57:59.768 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:57:59.768 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:57:59.769 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:57:59.769 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:57:59.769 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: Saving service mon spec with placement smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c;count:3 2024-05-10T03:57:59.769 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: Deploying daemon mon.c on smithi181 2024-05-10T03:57:59.769 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:57:59.769 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: from='client.? 172.21.15.181:0/2211953288' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T03:57:59.769 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:57:59 smithi181 ceph-mon[46602]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-10T03:57:59.939 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T03:57:59.940 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T03:58:00.163 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T03:58:04.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.382+0000 7f1f05a33700 1 -- 172.21.15.181:0/3809414689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee80035c0 msgr2=0x7f1ee8005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:04.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.382+0000 7f1f05a33700 1 --2- 172.21.15.181:0/3809414689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee80035c0 0x7f1ee8005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1f00068ca0 tx=0x7f1ef4013e90 comp rx=0 tx=0).stop 2024-05-10T03:58:04.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.382+0000 7f1f05a33700 1 -- 172.21.15.181:0/3809414689 shutdown_connections 2024-05-10T03:58:04.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.382+0000 7f1f05a33700 1 --2- 172.21.15.181:0/3809414689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee80035c0 0x7f1ee8005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:04.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.382+0000 7f1f05a33700 1 -- 172.21.15.181:0/3809414689 >> 172.21.15.181:0/3809414689 conn(0x7f1f000757f0 msgr2=0x7f1f00075c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:58:04.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.382+0000 7f1f05a33700 1 -- 172.21.15.181:0/3809414689 shutdown_connections 2024-05-10T03:58:04.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.382+0000 7f1f05a33700 1 -- 172.21.15.181:0/3809414689 wait complete. 2024-05-10T03:58:04.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.383+0000 7f1f05a33700 1 Processor -- start 2024-05-10T03:58:04.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.383+0000 7f1f05a33700 1 -- start start 2024-05-10T03:58:04.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1f05a33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ee80035c0 0x7f1f00197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:04.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1f05a33700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f00197750 0x7f1f0019bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:04.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1f05a33700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f00197cd0 con 0x7f1f00197750 2024-05-10T03:58:04.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1f05a33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f00197e10 con 0x7f1ee80035c0 2024-05-10T03:58:04.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1efeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ee80035c0 0x7f1f00197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:04.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1efeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ee80035c0 0x7f1f00197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42628/0 (socket says 172.21.15.181:42628) 2024-05-10T03:58:04.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1efeffd700 1 -- 172.21.15.181:0/1769141949 learned_addr learned my addr 172.21.15.181:0/1769141949 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T03:58:04.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1efeffd700 1 -- 172.21.15.181:0/1769141949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ee80035c0 msgr2=0x7f1f00197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-10T03:58:04.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1efeffd700 1 -- 172.21.15.181:0/1769141949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ee80035c0 msgr2=0x7f1f00197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T03:58:04.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1efeffd700 1 --2- 172.21.15.181:0/1769141949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ee80035c0 0x7f1f00197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:58:04.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1efe7fc700 1 --2- 172.21.15.181:0/1769141949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f00197750 0x7f1f0019bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:04.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1efeffd700 1 --2- 172.21.15.181:0/1769141949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ee80035c0 0x7f1f00197210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T03:58:04.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1efe7fc700 1 -- 172.21.15.181:0/1769141949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ee80035c0 msgr2=0x7f1f00197210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T03:58:04.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1efe7fc700 1 --2- 172.21.15.181:0/1769141949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ee80035c0 0x7f1f00197210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:04.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.384+0000 7f1efe7fc700 1 -- 172.21.15.181:0/1769141949 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef40138e0 con 0x7f1f00197750 2024-05-10T03:58:04.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.385+0000 7f1efe7fc700 1 --2- 172.21.15.181:0/1769141949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f00197750 0x7f1f0019bab0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000b4d0 tx=0x7f1ef000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:04.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.385+0000 7f1f04a31700 1 -- 172.21.15.181:0/1769141949 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f1ef0011630 con 0x7f1f00197750 2024-05-10T03:58:04.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.385+0000 7f1f04a31700 1 -- 172.21.15.181:0/1769141949 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ef0011c50 con 0x7f1f00197750 2024-05-10T03:58:04.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.385+0000 7f1f05a33700 1 -- 172.21.15.181:0/1769141949 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0019c0b0 con 0x7f1f00197750 2024-05-10T03:58:04.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.385+0000 7f1f04a31700 1 -- 172.21.15.181:0/1769141949 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f1ef0010d90 con 0x7f1f00197750 2024-05-10T03:58:04.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.385+0000 7f1f05a33700 1 -- 172.21.15.181:0/1769141949 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0019c600 con 0x7f1f00197750 2024-05-10T03:58:04.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.386+0000 7f1f04a31700 1 -- 172.21.15.181:0/1769141949 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f1ef001a400 con 0x7f1f00197750 2024-05-10T03:58:04.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.386+0000 7f1f05a33700 1 -- 172.21.15.181:0/1769141949 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f00105eb0 con 0x7f1f00197750 2024-05-10T03:58:04.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.386+0000 7f1f04a31700 1 --2- 172.21.15.181:0/1769141949 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1eec0385d0 0x7f1eec03aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:04.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.386+0000 7f1f04a31700 1 -- 172.21.15.181:0/1769141949 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1ef004bd00 con 0x7f1f00197750 2024-05-10T03:58:04.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.387+0000 7f1efeffd700 1 --2- 172.21.15.181:0/1769141949 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1eec0385d0 0x7f1eec03aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:04.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.387+0000 7f1efeffd700 1 --2- 172.21.15.181:0/1769141949 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1eec0385d0 0x7f1eec03aae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4001eb0 tx=0x7f1ef4013460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:04.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.392+0000 7f1f04a31700 1 -- 172.21.15.181:0/1769141949 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1ef00178b0 con 0x7f1f00197750 2024-05-10T03:58:04.670 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.667+0000 7f1f05a33700 1 -- 172.21.15.181:0/1769141949 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1f00198570 con 0x7f1f00197750 2024-05-10T03:58:04.670 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.668+0000 7f1f04a31700 1 -- 172.21.15.181:0/1769141949 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7f1ef00172c0 con 0x7f1f00197750 2024-05-10T03:58:04.670 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:58:04.671 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":2,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","modified":"2024-05-10T03:57:59.291768Z","created":"2024-05-10T03:56:37.662524Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-10T03:58:04.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.670+0000 7f1f05a33700 1 -- 172.21.15.181:0/1769141949 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1eec0385d0 msgr2=0x7f1eec03aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.670+0000 7f1f05a33700 1 --2- 172.21.15.181:0/1769141949 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1eec0385d0 0x7f1eec03aae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4001eb0 tx=0x7f1ef4013460 comp rx=0 tx=0).stop 2024-05-10T03:58:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.670+0000 7f1f05a33700 1 -- 172.21.15.181:0/1769141949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f00197750 msgr2=0x7f1f0019bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.670+0000 7f1f05a33700 1 --2- 172.21.15.181:0/1769141949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f00197750 0x7f1f0019bab0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000b4d0 tx=0x7f1ef000b890 comp rx=0 tx=0).stop 2024-05-10T03:58:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.670+0000 7f1f05a33700 1 -- 172.21.15.181:0/1769141949 shutdown_connections 2024-05-10T03:58:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.670+0000 7f1f05a33700 1 --2- 172.21.15.181:0/1769141949 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1eec0385d0 0x7f1eec03aae0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.670+0000 7f1f05a33700 1 --2- 172.21.15.181:0/1769141949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ee80035c0 0x7f1f00197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.670+0000 7f1f05a33700 1 --2- 172.21.15.181:0/1769141949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f00197750 0x7f1f0019bab0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.670+0000 7f1f05a33700 1 -- 172.21.15.181:0/1769141949 >> 172.21.15.181:0/1769141949 conn(0x7f1f000757f0 msgr2=0x7f1f000fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:58:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.670+0000 7f1f05a33700 1 -- 172.21.15.181:0/1769141949 shutdown_connections 2024-05-10T03:58:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:04.670+0000 7f1f05a33700 1 -- 172.21.15.181:0/1769141949 wait complete. 2024-05-10T03:58:04.674 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 2 2024-05-10T03:58:04.693 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:58:04.694 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:04.694 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: mon.a calling monitor election 2024-05-10T03:58:04.694 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:04.694 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:04.694 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:04.694 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: mon.c calling monitor election 2024-05-10T03:58:04.694 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:04.694 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:04.694 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:04.694 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:04.695 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T03:58:04.695 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: monmap e2: 2 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T03:58:04.695 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: fsmap 2024-05-10T03:58:04.695 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T03:58:04.695 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: mgrmap e13: x(active, since 35s) 2024-05-10T03:58:04.695 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: overall HEALTH_OK 2024-05-10T03:58:04.695 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:04.695 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:04.695 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:04.695 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:58:04.695 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:04.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: mon.a calling monitor election 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: mon.c calling monitor election 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: monmap e2: 2 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: fsmap 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T03:58:04.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: mgrmap e13: x(active, since 35s) 2024-05-10T03:58:04.716 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: overall HEALTH_OK 2024-05-10T03:58:04.716 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:04.716 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:04.716 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:04.716 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:58:04.716 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:05.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:05 smithi150 ceph-mon[42477]: Deploying daemon mon.b on smithi161 2024-05-10T03:58:05.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:05 smithi150 ceph-mon[42477]: from='client.? 172.21.15.181:0/1769141949' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T03:58:05.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:05 smithi150 ceph-mon[42477]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:05.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:05 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:05.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:05 smithi181 ceph-mon[46602]: Deploying daemon mon.b on smithi161 2024-05-10T03:58:05.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:05 smithi181 ceph-mon[46602]: from='client.? 172.21.15.181:0/1769141949' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T03:58:05.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:05 smithi181 ceph-mon[46602]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:05.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:05 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:06.092 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T03:58:06.092 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T03:58:06.323 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T03:58:07.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 systemd[1]: Started Ceph mon.b for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: RocksDB version: 7.9.2 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Git sha 0 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Compile date 2024-05-10 00:51:33 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: DB SUMMARY 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: DB Session ID: DN0C8VDIV5ENNVSWJ3AA 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: CURRENT file: CURRENT 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.error_if_exists: 0 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.create_if_missing: 0 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.paranoid_checks: 1 2024-05-10T03:58:07.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T03:58:07.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T03:58:07.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T03:58:07.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.env: 0x558c6bb0b460 2024-05-10T03:58:07.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T03:58:07.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.info_log: 0x558c6d06d760 2024-05-10T03:58:07.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T03:58:07.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.statistics: (nil) 2024-05-10T03:58:07.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.use_fsync: 0 2024-05-10T03:58:07.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_log_file_size: 0 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.allow_fallocate: 1 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.use_direct_reads: 0 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.db_log_dir: 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.wal_dir: 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T03:58:07.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T03:58:07.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T03:58:07.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T03:58:07.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T03:58:07.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T03:58:07.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T03:58:07.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T03:58:07.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.write_buffer_manager: 0x558c6cfb10e0 2024-05-10T03:58:07.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T03:58:07.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T03:58:07.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T03:58:07.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T03:58:07.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T03:58:07.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T03:58:07.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T03:58:07.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T03:58:07.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.unordered_write: 0 2024-05-10T03:58:07.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T03:58:07.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T03:58:07.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T03:58:07.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T03:58:07.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.row_cache: None 2024-05-10T03:58:07.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.wal_filter: None 2024-05-10T03:58:07.501 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T03:58:07.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T03:58:07.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.two_write_queues: 0 2024-05-10T03:58:07.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T03:58:07.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.wal_compression: 0 2024-05-10T03:58:07.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.atomic_flush: 0 2024-05-10T03:58:07.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T03:58:07.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T03:58:07.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T03:58:07.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.log_readahead_size: 0 2024-05-10T03:58:07.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T03:58:07.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T03:58:07.502 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T03:58:07.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T03:58:07.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T03:58:07.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T03:58:07.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T03:58:07.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_background_jobs: 2 2024-05-10T03:58:07.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_background_compactions: -1 2024-05-10T03:58:07.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_subcompactions: 1 2024-05-10T03:58:07.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T03:58:07.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T03:58:07.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T03:58:07.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T03:58:07.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T03:58:07.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T03:58:07.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T03:58:07.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T03:58:07.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_open_files: -1 2024-05-10T03:58:07.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T03:58:07.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T03:58:07.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T03:58:07.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T03:58:07.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_background_flushes: -1 2024-05-10T03:58:07.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Compression algorithms supported: 2024-05-10T03:58:07.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T03:58:07.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: kZSTD supported: 0 2024-05-10T03:58:07.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: kXpressCompression supported: 0 2024-05-10T03:58:07.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T03:58:07.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: kZlibCompression supported: 1 2024-05-10T03:58:07.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: kSnappyCompression supported: 1 2024-05-10T03:58:07.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: kLZ4Compression supported: 1 2024-05-10T03:58:07.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: kBZip2Compression supported: 0 2024-05-10T03:58:07.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.merge_operator: 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compaction_filter: None 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558c6d06d880) 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout: cache_index_and_filter_blocks: 1 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T03:58:07.508 INFO:journalctl@ceph.mon.b.smithi161.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: pin_top_level_index_and_filter: 1 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_type: 0 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: data_block_index_type: 0 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_shortening: 1 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: checksum: 4 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: no_block_cache: 0 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache: 0x558c6d0331f0 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_name: BinnedLRUCache 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_options: 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: capacity : 536870912 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: num_shard_bits : 4 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: strict_capacity_limit : 0 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: high_pri_pool_ratio: 0.000 2024-05-10T03:58:07.509 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_compressed: (nil) 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: persistent_cache: (nil) 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_size: 4096 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_size_deviation: 10 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_restart_interval: 16 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_block_restart_interval: 1 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: metadata_block_size: 4096 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: partition_filters: 0 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: use_delta_encoding: 1 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: filter_policy: bloomfilter 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: whole_key_filtering: 1 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: verify_compression: 0 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: read_amp_bytes_per_bit: 0 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: format_version: 5 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: enable_index_compression: 1 2024-05-10T03:58:07.510 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_align: 0 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout: max_auto_readahead_size: 262144 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout: prepopulate_block_cache: 0 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout: initial_auto_readahead_size: 8192 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compression: NoCompression 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.num_levels: 7 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T03:58:07.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T03:58:07.512 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T03:58:07.512 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T03:58:07.512 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T03:58:07.512 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T03:58:07.512 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T03:58:07.512 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T03:58:07.512 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T03:58:07.512 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T03:58:07.512 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T03:58:07.512 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T03:58:07.512 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T03:58:07.512 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T03:58:07.512 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T03:58:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T03:58:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T03:58:07.515 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T03:58:07.515 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T03:58:07.515 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T03:58:07.515 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T03:58:07.515 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T03:58:07.515 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T03:58:07.515 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.table_properties_collectors: 2024-05-10T03:58:07.515 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.inplace_update_support: 0 2024-05-10T03:58:07.515 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T03:58:07.516 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T03:58:07.516 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T03:58:07.516 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T03:58:07.516 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.bloom_locality: 0 2024-05-10T03:58:07.516 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.max_successive_merges: 0 2024-05-10T03:58:07.516 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T03:58:07.516 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T03:58:07.516 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T03:58:07.516 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T03:58:07.516 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.ttl: 2592000 2024-05-10T03:58:07.516 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T03:58:07.516 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T03:58:07.516 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.enable_blob_files: false 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.min_blob_size: 0 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5b10cad3-9b49-4c46-b5f6-806a0171e450 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715313487052230, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-10T03:58:07.517 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-10T03:58:07.518 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715313487052776, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715313487, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5b10cad3-9b49-4c46-b5f6-806a0171e450", "db_session_id": "DN0C8VDIV5ENNVSWJ3AA", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-10T03:58:07.518 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715313487052864, "job": 1, "event": "recovery_finished"} 2024-05-10T03:58:07.518 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-10T03:58:07.518 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T03:58:07.518 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558c6cfc5c00 2024-05-10T03:58:07.518 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: DB pointer 0x558c71db4000 2024-05-10T03:58:07.518 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T03:58:07.518 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T03:58:07.518 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** DB Stats ** 2024-05-10T03:58:07.518 INFO:journalctl@ceph.mon.b.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T03:58:07.518 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T03:58:07.518 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** Compaction Stats [default] ** 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** Compaction Stats [default] ** 2024-05-10T03:58:07.519 INFO:journalctl@ceph.mon.b.smithi161.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T03:58:07.520 INFO:journalctl@ceph.mon.b.smithi161.stdout: Block cache BinnedLRUCache@0x558c6d0331f0#7 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: using public_addr v2:172.21.15.161:0/0 -> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: starting mon.b rank -1 at public addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] at bind addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(???) e0 preinit fsid 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(synchronizing).mds e1 new map 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout: e1 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout: legacy client fscid: -1 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-10T03:58:07.521 INFO:journalctl@ceph.mon.b.smithi161.stdout: No filesystems configured 2024-05-10T03:58:07.522 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-10T03:58:07.522 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-10T03:58:07.522 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-10T03:58:07.522 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-10T03:58:07.522 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-10T03:58:07.522 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-10T03:58:07.522 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T03:58:07.522 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T03:58:07.522 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2688410777' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: Updating smithi181:/etc/ceph/ceph.conf 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: Updating smithi181:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2688410777' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T03:58:07.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-05-10T03:58:07.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:07.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:07.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:07.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: Updating smithi181:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T03:58:07.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:07.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:07.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:07.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:07.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:07.524 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: Saving service mon spec with placement smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c;count:3 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: Deploying daemon mon.c on smithi181 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='client.? 172.21.15.181:0/2211953288' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.a calling monitor election 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.c calling monitor election 2024-05-10T03:58:07.525 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: monmap e2: 2 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: fsmap 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mgrmap e13: x(active, since 35s) 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: overall HEALTH_OK 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:58:07.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:07.527 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: Deploying daemon mon.b on smithi161 2024-05-10T03:58:07.527 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='client.? 172.21.15.181:0/1769141949' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T03:58:07.527 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:07.527 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:07.527 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:07 smithi161 ceph-mon[46918]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-10T03:58:12.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:12.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:58:12.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: mon.a calling monitor election 2024-05-10T03:58:12.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: mon.c calling monitor election 2024-05-10T03:58:12.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:12.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:12.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: mon.b calling monitor election 2024-05-10T03:58:12.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:12.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: mon.a calling monitor election 2024-05-10T03:58:12.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T03:58:12.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: monmap e3: 3 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T03:58:12.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: fsmap 2024-05-10T03:58:12.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T03:58:12.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: mgrmap e13: x(active, since 43s) 2024-05-10T03:58:12.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: overall HEALTH_OK 2024-05-10T03:58:12.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:12.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:12.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:12.467 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:12.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:58:12.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: mon.a calling monitor election 2024-05-10T03:58:12.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: mon.c calling monitor election 2024-05-10T03:58:12.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:12.490 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.490 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:12.490 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.490 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: mon.b calling monitor election 2024-05-10T03:58:12.490 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.490 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:12.491 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.491 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.491 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: mon.a calling monitor election 2024-05-10T03:58:12.491 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T03:58:12.491 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: monmap e3: 3 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T03:58:12.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: fsmap 2024-05-10T03:58:12.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T03:58:12.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: mgrmap e13: x(active, since 43s) 2024-05-10T03:58:12.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: overall HEALTH_OK 2024-05-10T03:58:12.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:12.492 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:12.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:12.493 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:12.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:12.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:58:12.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: mon.a calling monitor election 2024-05-10T03:58:12.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: mon.c calling monitor election 2024-05-10T03:58:12.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:58:12.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:12.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: mon.b calling monitor election 2024-05-10T03:58:12.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:12.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:12.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: mon.a calling monitor election 2024-05-10T03:58:12.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T03:58:12.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: monmap e3: 3 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T03:58:12.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: fsmap 2024-05-10T03:58:12.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T03:58:12.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: mgrmap e13: x(active, since 43s) 2024-05-10T03:58:12.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: overall HEALTH_OK 2024-05-10T03:58:12.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:12.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:12.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:12.497 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:13.306 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.303+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2898172901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c00064e0 msgr2=0x7f26c0003ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:13.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.303+0000 7f26ddfa0700 1 --2- 172.21.15.181:0/2898172901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c00064e0 0x7f26c0003ab0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f26d8101580 tx=0x7f26cc00fe90 comp rx=0 tx=0).stop 2024-05-10T03:58:13.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.303+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2898172901 shutdown_connections 2024-05-10T03:58:13.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.303+0000 7f26ddfa0700 1 --2- 172.21.15.181:0/2898172901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c00064e0 0x7f26c0003ab0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:13.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.303+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2898172901 >> 172.21.15.181:0/2898172901 conn(0x7f26d8076030 msgr2=0x7f26d8076470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:58:13.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.303+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2898172901 shutdown_connections 2024-05-10T03:58:13.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.303+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2898172901 wait complete. 2024-05-10T03:58:13.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.304+0000 7f26ddfa0700 1 Processor -- start 2024-05-10T03:58:13.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.304+0000 7f26ddfa0700 1 -- start start 2024-05-10T03:58:13.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26ddfa0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c00064e0 0x7f26d8196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:13.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26ddfa0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d81966a0 0x7f26d819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:13.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26ddfa0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d819af40 0x7f26d819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:13.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26ddfa0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d8196da0 con 0x7f26c00064e0 2024-05-10T03:58:13.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26ddfa0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d81a2cb0 con 0x7f26d819af40 2024-05-10T03:58:13.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26ddfa0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d81a2df0 con 0x7f26d81966a0 2024-05-10T03:58:13.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c00064e0 0x7f26d8196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:13.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d81966a0 0x7f26d819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:13.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c00064e0 0x7f26d8196160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.181:49728/0 (socket says 172.21.15.181:49728) 2024-05-10T03:58:13.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d77fe700 1 -- 172.21.15.181:0/2798040853 learned_addr learned my addr 172.21.15.181:0/2798040853 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T03:58:13.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d7fff700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d819af40 0x7f26d819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d6ffd700 1 -- 172.21.15.181:0/2798040853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d81966a0 msgr2=0x7f26d819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d6ffd700 1 -- 172.21.15.181:0/2798040853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d81966a0 msgr2=0x7f26d819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d6ffd700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d81966a0 0x7f26d819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d6ffd700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d81966a0 0x7f26d819aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d7fff700 1 -- 172.21.15.181:0/2798040853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d819af40 msgr2=0x7f26d819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d7fff700 1 -- 172.21.15.181:0/2798040853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d819af40 msgr2=0x7f26d819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d7fff700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d819af40 0x7f26d819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d7fff700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d819af40 0x7f26d819b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d77fe700 1 -- 172.21.15.181:0/2798040853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d81966a0 msgr2=0x7f26d819aa00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d77fe700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d81966a0 0x7f26d819aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d77fe700 1 -- 172.21.15.181:0/2798040853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d819af40 msgr2=0x7f26d819b350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d77fe700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d819af40 0x7f26d819b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d77fe700 1 -- 172.21.15.181:0/2798040853 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc00f8e0 con 0x7f26c00064e0 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.305+0000 7f26d77fe700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c00064e0 0x7f26d8196160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f26cc014f10 tx=0x7f26cc014ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:13.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.306+0000 7f26d4ff9700 1 -- 172.21.15.181:0/2798040853 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc012d60 con 0x7f26c00064e0 2024-05-10T03:58:13.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.306+0000 7f26d4ff9700 1 -- 172.21.15.181:0/2798040853 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26cc011d90 con 0x7f26c00064e0 2024-05-10T03:58:13.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.306+0000 7f26d4ff9700 1 -- 172.21.15.181:0/2798040853 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc014350 con 0x7f26c00064e0 2024-05-10T03:58:13.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.306+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2798040853 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d81a3070 con 0x7f26c00064e0 2024-05-10T03:58:13.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.306+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2798040853 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d81a35a0 con 0x7f26c00064e0 2024-05-10T03:58:13.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.307+0000 7f26d4ff9700 1 -- 172.21.15.181:0/2798040853 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f26cc02e030 con 0x7f26c00064e0 2024-05-10T03:58:13.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.307+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2798040853 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d8104d60 con 0x7f26c00064e0 2024-05-10T03:58:13.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.307+0000 7f26d4ff9700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f26c4038c20 0x7f26c403b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:13.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.307+0000 7f26d4ff9700 1 -- 172.21.15.181:0/2798040853 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f26cc000e40 con 0x7f26c00064e0 2024-05-10T03:58:13.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.307+0000 7f26d6ffd700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f26c4038c20 0x7f26c403b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:13.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.308+0000 7f26d6ffd700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f26c4038c20 0x7f26c403b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f26d8105f20 tx=0x7f26c8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:13.316 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.313+0000 7f26d4ff9700 1 -- 172.21.15.181:0/2798040853 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f26cc02f970 con 0x7f26c00064e0 2024-05-10T03:58:13.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:13.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:13 smithi150 ceph-mon[42477]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:13.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:13.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:13.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:13.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:13.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:13.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:13.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:13.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:13 smithi150 ceph-mon[42477]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-10T03:58:13.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:13 smithi150 ceph-mon[42477]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-10T03:58:13.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:13 smithi150 ceph-mon[42477]: Updating smithi181:/etc/ceph/ceph.conf 2024-05-10T03:58:13.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:13.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:13 smithi181 ceph-mon[46602]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:13.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:13.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:13.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:13.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:13.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:13.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:13.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:13.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:13 smithi181 ceph-mon[46602]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-10T03:58:13.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:13 smithi181 ceph-mon[46602]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-10T03:58:13.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:13 smithi181 ceph-mon[46602]: Updating smithi181:/etc/ceph/ceph.conf 2024-05-10T03:58:13.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:13.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:13 smithi161 ceph-mon[46918]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:13.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:13.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:13.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:58:13.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:13.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:13.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:13.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:13.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:13 smithi161 ceph-mon[46918]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-10T03:58:13.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:13 smithi161 ceph-mon[46918]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-10T03:58:13.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:13 smithi161 ceph-mon[46918]: Updating smithi181:/etc/ceph/ceph.conf 2024-05-10T03:58:13.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.632+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2798040853 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f26d81974c0 con 0x7f26c00064e0 2024-05-10T03:58:13.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.632+0000 7f26d4ff9700 1 -- 172.21.15.181:0/2798040853 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1280 (secure 0 0 0) 0x7f26cc02f380 con 0x7f26c00064e0 2024-05-10T03:58:13.636 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T03:58:13.637 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":3,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","modified":"2024-05-10T03:58:07.087025Z","created":"2024-05-10T03:56:37.662524Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-10T03:58:13.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.634+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2798040853 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f26c4038c20 msgr2=0x7f26c403b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:13.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.634+0000 7f26ddfa0700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f26c4038c20 0x7f26c403b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f26d8105f20 tx=0x7f26c8008040 comp rx=0 tx=0).stop 2024-05-10T03:58:13.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.634+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2798040853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c00064e0 msgr2=0x7f26d8196160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:13.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.634+0000 7f26ddfa0700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c00064e0 0x7f26d8196160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f26cc014f10 tx=0x7f26cc014ff0 comp rx=0 tx=0).stop 2024-05-10T03:58:13.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.634+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2798040853 shutdown_connections 2024-05-10T03:58:13.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.635+0000 7f26ddfa0700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f26c4038c20 0x7f26c403b130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:13.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.635+0000 7f26ddfa0700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d819af40 0x7f26d819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:13.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.635+0000 7f26ddfa0700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c00064e0 0x7f26d8196160 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:13.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.635+0000 7f26ddfa0700 1 --2- 172.21.15.181:0/2798040853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d81966a0 0x7f26d819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:13.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.635+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2798040853 >> 172.21.15.181:0/2798040853 conn(0x7f26d8076030 msgr2=0x7f26d80ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:58:13.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.635+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2798040853 shutdown_connections 2024-05-10T03:58:13.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:13.635+0000 7f26ddfa0700 1 -- 172.21.15.181:0/2798040853 wait complete. 2024-05-10T03:58:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 3 2024-05-10T03:58:13.976 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-10T03:58:13.976 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-10T03:58:14.453 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T03:58:14.864 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: Updating smithi150:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: Updating smithi161:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: Updating smithi181:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='client.? 172.21.15.181:0/2798040853' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.865 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T03:58:14.866 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:58:14.866 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T03:58:14.866 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:14.866 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: Reconfiguring daemon mon.a on smithi150 2024-05-10T03:58:14.866 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: Updating smithi150:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:58:14.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: Updating smithi161:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:58:14.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: Updating smithi181:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:58:14.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='client.? 172.21.15.181:0/2798040853' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T03:58:14.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T03:58:14.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:58:14.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T03:58:14.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:14.991 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: Reconfiguring daemon mon.a on smithi150 2024-05-10T03:58:14.991 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: Updating smithi150:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: Updating smithi161:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: Updating smithi181:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='client.? 172.21.15.181:0/2798040853' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:14.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T03:58:14.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:58:14.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T03:58:14.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:14.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: Reconfiguring daemon mon.a on smithi150 2024-05-10T03:58:14.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:15.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.556+0000 7f4b078d5700 1 -- 172.21.15.150:0/3794401893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b00105980 msgr2=0x7f4b00105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:15.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.556+0000 7f4b078d5700 1 --2- 172.21.15.150:0/3794401893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b00105980 0x7f4b00105d90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009990 tx=0x7f4afc009ca0 comp rx=0 tx=0).stop 2024-05-10T03:58:15.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.556+0000 7f4b078d5700 1 -- 172.21.15.150:0/3794401893 shutdown_connections 2024-05-10T03:58:15.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.556+0000 7f4b078d5700 1 --2- 172.21.15.150:0/3794401893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b001062d0 0x7f4b0010dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:15.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.556+0000 7f4b078d5700 1 --2- 172.21.15.150:0/3794401893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b00101570 0x7f4b00101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:15.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.556+0000 7f4b078d5700 1 --2- 172.21.15.150:0/3794401893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b00105980 0x7f4b00105d90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:15.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.556+0000 7f4b078d5700 1 -- 172.21.15.150:0/3794401893 >> 172.21.15.150:0/3794401893 conn(0x7f4b000fbed0 msgr2=0x7f4b000fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:58:15.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.556+0000 7f4b078d5700 1 -- 172.21.15.150:0/3794401893 shutdown_connections 2024-05-10T03:58:15.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.556+0000 7f4b078d5700 1 -- 172.21.15.150:0/3794401893 wait complete. 2024-05-10T03:58:15.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.557+0000 7f4b078d5700 1 Processor -- start 2024-05-10T03:58:15.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.557+0000 7f4b078d5700 1 -- start start 2024-05-10T03:58:15.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.557+0000 7f4b078d5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b00101570 0x7f4b0019d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:15.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.557+0000 7f4b078d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b00105980 0x7f4b0019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:15.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.557+0000 7f4b078d5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b001062d0 0x7f4b00197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:15.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.557+0000 7f4b078d5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00197a40 con 0x7f4b001062d0 2024-05-10T03:58:15.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.557+0000 7f4b078d5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00197bb0 con 0x7f4b00101570 2024-05-10T03:58:15.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.557+0000 7f4b078d5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00197d70 con 0x7f4b00105980 2024-05-10T03:58:15.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4b05e72700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b001062d0 0x7f4b00197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:15.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4b05e72700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b001062d0 0x7f4b00197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57480/0 (socket says 172.21.15.150:57480) 2024-05-10T03:58:15.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4b05e72700 1 -- 172.21.15.150:0/73114007 learned_addr learned my addr 172.21.15.150:0/73114007 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:58:15.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4b04e70700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b00105980 0x7f4b0019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:15.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4b05671700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b00101570 0x7f4b0019d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:15.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4b05e72700 1 -- 172.21.15.150:0/73114007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b00105980 msgr2=0x7f4b0019da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:15.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4b05e72700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b00105980 0x7f4b0019da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:15.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4b05e72700 1 -- 172.21.15.150:0/73114007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b00101570 msgr2=0x7f4b0019d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:15.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4b05e72700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b00101570 0x7f4b0019d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:15.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4b05e72700 1 -- 172.21.15.150:0/73114007 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc0095f0 con 0x7f4b001062d0 2024-05-10T03:58:15.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4b05e72700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b001062d0 0x7f4b00197470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4af800b970 tx=0x7f4af800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:15.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4af67fc700 1 -- 172.21.15.150:0/73114007 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af800d860 con 0x7f4b001062d0 2024-05-10T03:58:15.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4af67fc700 1 -- 172.21.15.150:0/73114007 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4af800de80 con 0x7f4b001062d0 2024-05-10T03:58:15.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4b078d5700 1 -- 172.21.15.150:0/73114007 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b00198050 con 0x7f4b001062d0 2024-05-10T03:58:15.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4af67fc700 1 -- 172.21.15.150:0/73114007 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8017400 con 0x7f4b001062d0 2024-05-10T03:58:15.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.558+0000 7f4b078d5700 1 -- 172.21.15.150:0/73114007 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b001a44e0 con 0x7f4b001062d0 2024-05-10T03:58:15.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.559+0000 7f4af67fc700 1 -- 172.21.15.150:0/73114007 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f4af80203f0 con 0x7f4b001062d0 2024-05-10T03:58:15.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.559+0000 7f4af67fc700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4aec0389d0 0x7f4aec03aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:15.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.559+0000 7f4af67fc700 1 -- 172.21.15.150:0/73114007 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4af8006e40 con 0x7f4b001062d0 2024-05-10T03:58:15.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.559+0000 7f4b078d5700 1 -- 172.21.15.150:0/73114007 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b00109380 con 0x7f4b001062d0 2024-05-10T03:58:15.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.559+0000 7f4b05671700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4aec0389d0 0x7f4aec03aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:15.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.560+0000 7f4b05671700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4aec0389d0 0x7f4aec03aee0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4af0005950 tx=0x7f4af000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:15.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.565+0000 7f4af67fc700 1 -- 172.21.15.150:0/73114007 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4af80206a0 con 0x7f4b001062d0 2024-05-10T03:58:15.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.754+0000 7f4b078d5700 1 -- 172.21.15.150:0/73114007 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f4b00068b00 con 0x7f4b001062d0 2024-05-10T03:58:15.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.754+0000 7f4af67fc700 1 -- 172.21.15.150:0/73114007 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f4af801e2f0 con 0x7f4b001062d0 2024-05-10T03:58:15.758 INFO:teuthology.orchestra.run.smithi150.stdout:# minimal ceph.conf for 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:58:15.758 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-05-10T03:58:15.758 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T03:58:15.758 INFO:teuthology.orchestra.run.smithi150.stdout: mon_host = [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] 2024-05-10T03:58:15.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.757+0000 7f4b078d5700 1 -- 172.21.15.150:0/73114007 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4aec0389d0 msgr2=0x7f4aec03aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:15.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.757+0000 7f4b078d5700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4aec0389d0 0x7f4aec03aee0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4af0005950 tx=0x7f4af000a300 comp rx=0 tx=0).stop 2024-05-10T03:58:15.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.757+0000 7f4b078d5700 1 -- 172.21.15.150:0/73114007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b001062d0 msgr2=0x7f4b00197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:15.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.757+0000 7f4b078d5700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b001062d0 0x7f4b00197470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4af800b970 tx=0x7f4af800bc80 comp rx=0 tx=0).stop 2024-05-10T03:58:15.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.757+0000 7f4b078d5700 1 -- 172.21.15.150:0/73114007 shutdown_connections 2024-05-10T03:58:15.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.757+0000 7f4b078d5700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4aec0389d0 0x7f4aec03aee0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:15.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.757+0000 7f4b078d5700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b001062d0 0x7f4b00197470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:15.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.757+0000 7f4b078d5700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b00101570 0x7f4b0019d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:15.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.757+0000 7f4b078d5700 1 --2- 172.21.15.150:0/73114007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b00105980 0x7f4b0019da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:15.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.757+0000 7f4b078d5700 1 -- 172.21.15.150:0/73114007 >> 172.21.15.150:0/73114007 conn(0x7f4b000fbed0 msgr2=0x7f4b00107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:58:15.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.757+0000 7f4b078d5700 1 -- 172.21.15.150:0/73114007 shutdown_connections 2024-05-10T03:58:15.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:15.757+0000 7f4b078d5700 1 -- 172.21.15.150:0/73114007 wait complete. 2024-05-10T03:58:16.118 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:16.118 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:16.118 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: Reconfiguring mon.b (monmap changed)... 2024-05-10T03:58:16.118 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:58:16.118 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T03:58:16.118 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:16.118 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: Reconfiguring daemon mon.b on smithi161 2024-05-10T03:58:16.118 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:16.119 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/73114007' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:16.119 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:16.119 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:16.119 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:58:16.119 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T03:58:16.119 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:16.120 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-10T03:58:16.120 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:58:16.120 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T03:58:16.150 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:58:16.150 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T03:58:16.217 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T03:58:16.217 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T03:58:16.233 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:16.233 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:16.233 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: Reconfiguring mon.b (monmap changed)... 2024-05-10T03:58:16.233 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:58:16.233 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T03:58:16.233 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:16.234 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: Reconfiguring daemon mon.b on smithi161 2024-05-10T03:58:16.234 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:16.234 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/73114007' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:16.234 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:16.234 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:16.234 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:58:16.234 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T03:58:16.234 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:16.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: Reconfiguring mon.b (monmap changed)... 2024-05-10T03:58:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:58:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T03:58:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: Reconfiguring daemon mon.b on smithi161 2024-05-10T03:58:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/73114007' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T03:58:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T03:58:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:16.247 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T03:58:16.247 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T03:58:16.315 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T03:58:16.315 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T03:58:16.348 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T03:58:16.348 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T03:58:16.416 INFO:tasks.cephadm:Adding mgr.x on smithi150 2024-05-10T03:58:16.416 INFO:tasks.cephadm:Adding mgr.y on smithi161 2024-05-10T03:58:16.417 INFO:tasks.cephadm:Adding mgr.z on smithi181 2024-05-10T03:58:16.417 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi150=x;smithi161=y;smithi181=z' 2024-05-10T03:58:16.828 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T03:58:17.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:16 smithi150 ceph-mon[42477]: Reconfiguring mon.c (monmap changed)... 2024-05-10T03:58:17.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:16 smithi150 ceph-mon[42477]: Reconfiguring daemon mon.c on smithi181 2024-05-10T03:58:17.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:17.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:17.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:17.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:17.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:17.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:16 smithi181 ceph-mon[46602]: Reconfiguring mon.c (monmap changed)... 2024-05-10T03:58:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:16 smithi181 ceph-mon[46602]: Reconfiguring daemon mon.c on smithi181 2024-05-10T03:58:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:17.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:16 smithi161 ceph-mon[46918]: Reconfiguring mon.c (monmap changed)... 2024-05-10T03:58:17.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:16 smithi161 ceph-mon[46918]: Reconfiguring daemon mon.c on smithi181 2024-05-10T03:58:17.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:17.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:17.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:17.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:17.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:17.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:17.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.871+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2656075396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d28102130 msgr2=0x7f3d28102540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:17.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.871+0000 7f3d2d2ab700 1 --2- 172.21.15.181:0/2656075396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d28102130 0x7f3d28102540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009990 tx=0x7f3d18009ca0 comp rx=0 tx=0).stop 2024-05-10T03:58:17.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.871+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2656075396 shutdown_connections 2024-05-10T03:58:17.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.871+0000 7f3d2d2ab700 1 --2- 172.21.15.181:0/2656075396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d281017a0 0x7f3d281090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:17.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.871+0000 7f3d2d2ab700 1 --2- 172.21.15.181:0/2656075396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d28102130 0x7f3d28102540 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:17.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.871+0000 7f3d2d2ab700 1 --2- 172.21.15.181:0/2656075396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d28100e50 0x7f3d28101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:17.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.871+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2656075396 >> 172.21.15.181:0/2656075396 conn(0x7f3d280fb990 msgr2=0x7f3d280fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:58:17.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.871+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2656075396 shutdown_connections 2024-05-10T03:58:17.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.871+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2656075396 wait complete. 2024-05-10T03:58:17.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.872+0000 7f3d2d2ab700 1 Processor -- start 2024-05-10T03:58:17.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.872+0000 7f3d2d2ab700 1 -- start start 2024-05-10T03:58:17.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.872+0000 7f3d2d2ab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d28100e50 0x7f3d2819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:17.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.872+0000 7f3d2d2ab700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d281017a0 0x7f3d2819d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:17.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.872+0000 7f3d2d2ab700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d28102130 0x7f3d2819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:17.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.872+0000 7f3d2d2ab700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2819fd60 con 0x7f3d28102130 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.872+0000 7f3d2d2ab700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2819fea0 con 0x7f3d281017a0 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.872+0000 7f3d2d2ab700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2819ffe0 con 0x7f3d28100e50 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.872+0000 7f3d26ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d28100e50 0x7f3d2819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.872+0000 7f3d26ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d28100e50 0x7f3d2819ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47680/0 (socket says 172.21.15.181:47680) 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.872+0000 7f3d26ffd700 1 -- 172.21.15.181:0/2366665502 learned_addr learned my addr 172.21.15.181:0/2366665502 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.872+0000 7f3d267fc700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d281017a0 0x7f3d2819d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.873+0000 7f3d277fe700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d28102130 0x7f3d2819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.873+0000 7f3d26ffd700 1 -- 172.21.15.181:0/2366665502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d281017a0 msgr2=0x7f3d2819d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.873+0000 7f3d26ffd700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d281017a0 0x7f3d2819d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.873+0000 7f3d26ffd700 1 -- 172.21.15.181:0/2366665502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d28102130 msgr2=0x7f3d2819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.873+0000 7f3d26ffd700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d28102130 0x7f3d2819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.873+0000 7f3d26ffd700 1 -- 172.21.15.181:0/2366665502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d180095f0 con 0x7f3d28100e50 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.873+0000 7f3d26ffd700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d28100e50 0x7f3d2819ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3d18006010 tx=0x7f3d1800f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:17.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.873+0000 7f3d0ffff700 1 -- 172.21.15.181:0/2366665502 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1801d070 con 0x7f3d28100e50 2024-05-10T03:58:17.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.873+0000 7f3d0ffff700 1 -- 172.21.15.181:0/2366665502 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d18005510 con 0x7f3d28100e50 2024-05-10T03:58:17.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.873+0000 7f3d0ffff700 1 -- 172.21.15.181:0/2366665502 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18004bd0 con 0x7f3d28100e50 2024-05-10T03:58:17.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.873+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2366665502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d281a0210 con 0x7f3d28100e50 2024-05-10T03:58:17.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.873+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2366665502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d281a0570 con 0x7f3d28100e50 2024-05-10T03:58:17.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.874+0000 7f3d0ffff700 1 -- 172.21.15.181:0/2366665502 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f3d18004d30 con 0x7f3d28100e50 2024-05-10T03:58:17.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.874+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2366665502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d28107f50 con 0x7f3d28100e50 2024-05-10T03:58:17.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.874+0000 7f3d0ffff700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3d100389d0 0x7f3d1003aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:17.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.874+0000 7f3d0ffff700 1 -- 172.21.15.181:0/2366665502 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3d1804c460 con 0x7f3d28100e50 2024-05-10T03:58:17.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.875+0000 7f3d267fc700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3d100389d0 0x7f3d1003aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:17.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.875+0000 7f3d267fc700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3d100389d0 0x7f3d1003aee0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3d28100170 tx=0x7f3d1c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:17.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:17.880+0000 7f3d0ffff700 1 -- 172.21.15.181:0/2366665502 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3d18029750 con 0x7f3d28100e50 2024-05-10T03:58:17.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:17 smithi181 ceph-mon[46602]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:18.089 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.086+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2366665502 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi150=x;smithi161=y;smithi181=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d2810c6b0 con 0x7f3d100389d0 2024-05-10T03:58:18.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.093+0000 7f3d0ffff700 1 -- 172.21.15.181:0/2366665502 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3d2810c6b0 con 0x7f3d100389d0 2024-05-10T03:58:18.096 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mgr update... 2024-05-10T03:58:18.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.095+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2366665502 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3d100389d0 msgr2=0x7f3d1003aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:18.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.095+0000 7f3d2d2ab700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3d100389d0 0x7f3d1003aee0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3d28100170 tx=0x7f3d1c00b410 comp rx=0 tx=0).stop 2024-05-10T03:58:18.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.095+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2366665502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d28100e50 msgr2=0x7f3d2819ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:18.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.095+0000 7f3d2d2ab700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d28100e50 0x7f3d2819ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3d18006010 tx=0x7f3d1800f670 comp rx=0 tx=0).stop 2024-05-10T03:58:18.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.095+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2366665502 shutdown_connections 2024-05-10T03:58:18.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.095+0000 7f3d2d2ab700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3d100389d0 0x7f3d1003aee0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:18.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.095+0000 7f3d2d2ab700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d28102130 0x7f3d2819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:18.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.095+0000 7f3d2d2ab700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d28100e50 0x7f3d2819ad80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:18.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.095+0000 7f3d2d2ab700 1 --2- 172.21.15.181:0/2366665502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d281017a0 0x7f3d2819d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:18.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.095+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2366665502 >> 172.21.15.181:0/2366665502 conn(0x7f3d280fb990 msgr2=0x7f3d28105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:58:18.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.095+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2366665502 shutdown_connections 2024-05-10T03:58:18.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:18.095+0000 7f3d2d2ab700 1 -- 172.21.15.181:0/2366665502 wait complete. 2024-05-10T03:58:18.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:17 smithi150 ceph-mon[42477]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:18.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:17 smithi161 ceph-mon[46918]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:18.479 DEBUG:teuthology.orchestra.run.smithi161:mgr.y> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.y.service 2024-05-10T03:58:18.482 DEBUG:teuthology.orchestra.run.smithi181:mgr.z> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.z.service 2024-05-10T03:58:18.484 INFO:tasks.cephadm:Deploying OSDs... 2024-05-10T03:58:18.485 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T03:58:18.485 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T03:58:18.507 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T03:58:18.507 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-05-10T03:58:18.514 INFO:journalctl@ceph.mgr.y.smithi161.stdout:-- Logs begin at Fri 2024-05-10 03:40:24 UTC. -- 2024-05-10T03:58:18.515 INFO:journalctl@ceph.mgr.z.smithi181.stdout:-- Logs begin at Fri 2024-05-10 03:40:19 UTC. -- 2024-05-10T03:58:18.568 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T03:58:18.568 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:58:18.568 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 182624 Links: 1 2024-05-10T03:58:18.569 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:58:18.569 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:58:18.569 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-10 03:57:38.974673333 +0000 2024-05-10T03:58:18.569 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-10 03:50:50.113131576 +0000 2024-05-10T03:58:18.569 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-10 03:50:50.113131576 +0000 2024-05-10T03:58:18.569 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-05-10T03:58:18.569 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T03:58:18.640 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-10T03:58:18.640 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-10T03:58:18.640 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000276014 s, 1.9 MB/s 2024-05-10T03:58:18.641 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T03:58:18.703 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-05-10T03:58:18.762 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T03:58:18.762 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:58:18.762 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 184566 Links: 1 2024-05-10T03:58:18.762 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:58:18.762 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:58:18.763 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-10 03:57:39.017672521 +0000 2024-05-10T03:58:18.763 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-10 03:50:50.467124885 +0000 2024-05-10T03:58:18.763 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-10 03:50:50.467124885 +0000 2024-05-10T03:58:18.763 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-05-10T03:58:18.763 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T03:58:18.832 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-10T03:58:18.832 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-10T03:58:18.832 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000202368 s, 2.5 MB/s 2024-05-10T03:58:18.833 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T03:58:18.893 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-05-10T03:58:18.952 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T03:58:18.953 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:58:18.953 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 182725 Links: 1 2024-05-10T03:58:18.953 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:58:18.953 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:58:18.953 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-10 03:57:39.058671747 +0000 2024-05-10T03:58:18.953 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-10 03:50:50.842117798 +0000 2024-05-10T03:58:18.953 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-10 03:50:50.842117798 +0000 2024-05-10T03:58:18.953 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-05-10T03:58:18.953 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T03:58:19.020 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-10T03:58:19.020 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-10T03:58:19.021 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000197988 s, 2.6 MB/s 2024-05-10T03:58:19.022 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T03:58:19.081 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-05-10T03:58:19.139 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T03:58:19.139 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:58:19.139 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 178795 Links: 1 2024-05-10T03:58:19.139 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:58:19.140 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:58:19.140 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-10 03:57:39.102670916 +0000 2024-05-10T03:58:19.140 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-10 03:50:51.196111107 +0000 2024-05-10T03:58:19.140 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-10 03:50:51.196111107 +0000 2024-05-10T03:58:19.140 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-05-10T03:58:19.140 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T03:58:19.206 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-10T03:58:19.207 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-10T03:58:19.207 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000231439 s, 2.2 MB/s 2024-05-10T03:58:19.208 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T03:58:19.266 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T03:58:19.266 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T03:58:19.288 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T03:58:19.288 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-05-10T03:58:19.343 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:19 smithi181 ceph-mon[46602]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi150=x;smithi161=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:58:19.343 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:19 smithi181 ceph-mon[46602]: Saving service mgr spec with placement smithi150=x;smithi161=y;smithi181=z;count:3 2024-05-10T03:58:19.343 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:19.343 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:19.343 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:19.343 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:19.343 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:19.343 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T03:58:19.343 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T03:58:19.344 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T03:58:19.344 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:19.348 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T03:58:19.348 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:58:19.348 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 183307 Links: 1 2024-05-10T03:58:19.348 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:58:19.348 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:58:19.348 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-10 03:58:06.579467513 +0000 2024-05-10T03:58:19.348 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-10 03:51:10.875951760 +0000 2024-05-10T03:58:19.348 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-10 03:51:10.875951760 +0000 2024-05-10T03:58:19.348 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-05-10T03:58:19.349 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T03:58:19.415 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:19 smithi161 ceph-mon[46918]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi150=x;smithi161=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:58:19.415 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:19 smithi161 ceph-mon[46918]: Saving service mgr spec with placement smithi150=x;smithi161=y;smithi181=z;count:3 2024-05-10T03:58:19.415 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:19.415 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:19.415 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:19.415 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:19.416 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:19.416 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T03:58:19.416 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T03:58:19.416 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T03:58:19.416 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:19.417 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-10T03:58:19.417 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-10T03:58:19.417 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.00021421 s, 2.4 MB/s 2024-05-10T03:58:19.419 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T03:58:19.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:19 smithi150 ceph-mon[42477]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi150=x;smithi161=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:58:19.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:19 smithi150 ceph-mon[42477]: Saving service mgr spec with placement smithi150=x;smithi161=y;smithi181=z;count:3 2024-05-10T03:58:19.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:19.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:19.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:19.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:19.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:19.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T03:58:19.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T03:58:19.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T03:58:19.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:19.479 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-05-10T03:58:19.538 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T03:58:19.538 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:58:19.538 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 178927 Links: 1 2024-05-10T03:58:19.538 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:58:19.538 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:58:19.538 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-10 03:58:06.579467513 +0000 2024-05-10T03:58:19.538 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-10 03:51:11.242944155 +0000 2024-05-10T03:58:19.538 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-10 03:51:11.242944155 +0000 2024-05-10T03:58:19.538 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-05-10T03:58:19.539 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T03:58:19.607 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-10T03:58:19.607 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-10T03:58:19.607 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000227661 s, 2.2 MB/s 2024-05-10T03:58:19.609 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T03:58:19.668 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-05-10T03:58:19.726 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T03:58:19.726 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:58:19.727 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 180844 Links: 1 2024-05-10T03:58:19.727 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:58:19.727 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:58:19.727 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-10 03:58:06.579467513 +0000 2024-05-10T03:58:19.727 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-10 03:51:11.584937069 +0000 2024-05-10T03:58:19.727 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-10 03:51:11.584937069 +0000 2024-05-10T03:58:19.727 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-05-10T03:58:19.727 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T03:58:19.738 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:19 smithi181 systemd[1]: Starting Ceph mgr.z for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T03:58:19.794 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-10T03:58:19.794 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-10T03:58:19.794 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000191186 s, 2.7 MB/s 2024-05-10T03:58:19.795 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T03:58:19.853 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-05-10T03:58:19.911 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T03:58:19.912 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:58:19.912 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 178996 Links: 1 2024-05-10T03:58:19.912 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:58:19.912 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:58:19.912 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-10 03:58:06.580467493 +0000 2024-05-10T03:58:19.912 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-10 03:51:11.922930065 +0000 2024-05-10T03:58:19.912 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-10 03:51:11.922930065 +0000 2024-05-10T03:58:19.912 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-05-10T03:58:19.912 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T03:58:19.978 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-10T03:58:19.978 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-10T03:58:19.978 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000258652 s, 2.0 MB/s 2024-05-10T03:58:19.979 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T03:58:20.037 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T03:58:20.037 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T03:58:20.056 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T03:58:20.056 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2024-05-10T03:58:20.114 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T03:58:20.115 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:58:20.115 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 179708 Links: 1 2024-05-10T03:58:20.115 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:58:20.115 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:58:20.115 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-05-10 03:58:19.513817181 +0000 2024-05-10T03:58:20.115 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-05-10 03:51:10.090389502 +0000 2024-05-10T03:58:20.115 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-05-10 03:51:10.090389502 +0000 2024-05-10T03:58:20.115 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2024-05-10T03:58:20.115 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T03:58:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-05-10T03:58:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-05-10T03:58:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000226451 s, 2.3 MB/s 2024-05-10T03:58:20.186 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T03:58:20.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:20 smithi181 ceph-mon[46602]: Deploying daemon mgr.z on smithi181 2024-05-10T03:58:20.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:20 smithi181 ceph-mon[46602]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:20.239 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:19 smithi181 ceph-mgr[48481]: -- 172.21.15.181:0/3359895881 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557e2d1821a0 con 0x557e2d1df600 2024-05-10T03:58:20.239 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:20 smithi181 bash[48443]: c64f7be61e40f758badb3608f89aecf4a9d202df3a3b3fd9e94a983066b3d063 2024-05-10T03:58:20.239 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:20 smithi181 systemd[1]: Started Ceph mgr.z for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T03:58:20.244 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2024-05-10T03:58:20.301 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T03:58:20.302 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:58:20.302 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 183431 Links: 1 2024-05-10T03:58:20.302 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:58:20.302 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:58:20.302 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-05-10 03:58:19.513817181 +0000 2024-05-10T03:58:20.302 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-05-10 03:51:10.424383100 +0000 2024-05-10T03:58:20.302 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-05-10 03:51:10.424383100 +0000 2024-05-10T03:58:20.302 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2024-05-10T03:58:20.302 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T03:58:20.369 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-05-10T03:58:20.369 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-05-10T03:58:20.369 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000221885 s, 2.3 MB/s 2024-05-10T03:58:20.370 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T03:58:20.429 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2024-05-10T03:58:20.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:20 smithi150 ceph-mon[42477]: Deploying daemon mgr.z on smithi181 2024-05-10T03:58:20.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:20 smithi150 ceph-mon[42477]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:20.488 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T03:58:20.488 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:58:20.488 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 179823 Links: 1 2024-05-10T03:58:20.488 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:58:20.488 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:58:20.488 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-05-10 03:58:19.513817181 +0000 2024-05-10T03:58:20.489 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-05-10 03:51:10.753376793 +0000 2024-05-10T03:58:20.489 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-05-10 03:51:10.753376793 +0000 2024-05-10T03:58:20.489 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2024-05-10T03:58:20.489 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T03:58:20.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:20 smithi161 ceph-mon[46918]: Deploying daemon mgr.z on smithi181 2024-05-10T03:58:20.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:20 smithi161 ceph-mon[46918]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:20.494 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:20 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:20.394+0000 7fbbaa1a9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T03:58:20.523 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-05-10T03:58:20.523 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-05-10T03:58:20.523 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000192958 s, 2.7 MB/s 2024-05-10T03:58:20.524 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T03:58:20.584 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2024-05-10T03:58:20.643 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T03:58:20.644 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:58:20.644 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 179875 Links: 1 2024-05-10T03:58:20.644 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:58:20.644 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:58:20.644 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-05-10 03:58:19.513817181 +0000 2024-05-10T03:58:20.644 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-05-10 03:51:11.098370180 +0000 2024-05-10T03:58:20.644 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-05-10 03:51:11.098370180 +0000 2024-05-10T03:58:20.644 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2024-05-10T03:58:20.644 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T03:58:20.682 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:20 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:20.491+0000 7fbbaa1a9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T03:58:20.683 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:20 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:20.589+0000 7fbbaa1a9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T03:58:20.708 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-05-10T03:58:20.708 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-05-10T03:58:20.708 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000217301 s, 2.4 MB/s 2024-05-10T03:58:20.709 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T03:58:20.769 INFO:tasks.cephadm:Deploying osd.0 on smithi150 with /dev/vg_nvme/lv_4... 2024-05-10T03:58:20.769 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T03:58:20.988 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:20 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:20.678+0000 7fbbaa1a9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T03:58:20.997 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T03:58:21.395 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:21 smithi161 systemd[1]: Starting Ceph mgr.y for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T03:58:21.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:21.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:21.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:21.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T03:58:21.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T03:58:21.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T03:58:21.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:21.396 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:21 smithi161 ceph-mon[46918]: Deploying daemon mgr.y on smithi161 2024-05-10T03:58:21.424 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:21.424 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:21.424 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:21.424 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T03:58:21.424 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T03:58:21.424 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T03:58:21.424 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:21.424 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:21 smithi150 ceph-mon[42477]: Deploying daemon mgr.y on smithi161 2024-05-10T03:58:21.488 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:21 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:21.326+0000 7fbbaa1a9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:58:21.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:21.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:21.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:21.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T03:58:21.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T03:58:21.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T03:58:21.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:21.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:21 smithi181 ceph-mon[46602]: Deploying daemon mgr.y on smithi161 2024-05-10T03:58:21.938 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:21 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:21.581+0000 7fbbaa1a9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T03:58:21.938 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:21 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:21.667+0000 7fbbaa1a9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T03:58:21.938 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:21 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:21.776+0000 7fbbaa1a9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T03:58:21.938 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:21 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:21.934+0000 7fbbaa1a9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T03:58:22.136 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:21 smithi161 ceph-mgr[48417]: -- 172.21.15.161:0/2089440656 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b5c557e1a0 con 0x55b5ca304000 2024-05-10T03:58:22.137 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:21 smithi161 bash[48379]: 668c0c0c30f0899159bc1393c5838c2e555eca2978220d8fe9b4e7cb9a449eec 2024-05-10T03:58:22.137 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:21 smithi161 systemd[1]: Started Ceph mgr.y for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T03:58:22.238 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:22 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:22.062+0000 7fbbaa1a9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T03:58:22.238 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:22 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:22.156+0000 7fbbaa1a9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T03:58:22.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:22 smithi181 ceph-mon[46602]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:22.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:22 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:22.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:22 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:22.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:22 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:22.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:22 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:22.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:22 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:22.296 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T03:58:22.296 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T03:58:22.297 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-10T03:58:22.297 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-10T03:58:22.297 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113662 s, 923 MB/s 2024-05-10T03:58:22.297 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-10T03:58:22.359 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:22 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:22.252+0000 7f4bcf4b5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T03:58:22.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:22 smithi161 ceph-mon[46918]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:22.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:22 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:22.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:22 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:22.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:22 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:22.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:22 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:22.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:22 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:22.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:22 smithi150 ceph-mon[42477]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:22.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:22 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:22.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:22 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:22.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:22 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:22.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:22 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:22.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:22 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:22.468 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:22 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:22.356+0000 7f4bcf4b5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T03:58:22.634 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-05-10T03:58:22.743 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:22 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:22.465+0000 7f4bcf4b5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T03:58:22.743 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:22 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:22.553+0000 7f4bcf4b5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T03:58:22.852 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T03:58:22.988 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:22 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:22.571+0000 7fbbaa1a9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T03:58:23.280 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:22 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:22.986+0000 7fbbaa1a9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T03:58:23.280 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:23 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:23.183+0000 7fbbaa1a9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T03:58:23.280 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:23 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:23.276+0000 7fbbaa1a9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T03:58:23.491 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:23 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:23.235+0000 7f4bcf4b5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:58:23.743 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:23 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:23.488+0000 7f4bcf4b5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T03:58:23.743 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:23 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:23.574+0000 7f4bcf4b5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T03:58:23.743 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:23 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:23.683+0000 7f4bcf4b5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T03:58:23.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.895+0000 7fbcad7ac700 1 -- 172.21.15.150:0/403339503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca80fffd0 msgr2=0x7fbca81003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:23.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.895+0000 7fbcad7ac700 1 --2- 172.21.15.150:0/403339503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca80fffd0 0x7fbca81003e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009990 tx=0x7fbc9c009ca0 comp rx=0 tx=0).stop 2024-05-10T03:58:23.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.895+0000 7fbcad7ac700 1 -- 172.21.15.150:0/403339503 shutdown_connections 2024-05-10T03:58:23.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.895+0000 7fbcad7ac700 1 --2- 172.21.15.150:0/403339503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca8107ab0 0x7fbca8109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:23.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.895+0000 7fbcad7ac700 1 --2- 172.21.15.150:0/403339503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca80fffd0 0x7fbca81003e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:23.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.895+0000 7fbcad7ac700 1 --2- 172.21.15.150:0/403339503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca8105150 0x7fbca8107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:23.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.895+0000 7fbcad7ac700 1 -- 172.21.15.150:0/403339503 >> 172.21.15.150:0/403339503 conn(0x7fbca80fb950 msgr2=0x7fbca80fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:58:23.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.895+0000 7fbcad7ac700 1 -- 172.21.15.150:0/403339503 shutdown_connections 2024-05-10T03:58:23.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.895+0000 7fbcad7ac700 1 -- 172.21.15.150:0/403339503 wait complete. 2024-05-10T03:58:23.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbcad7ac700 1 Processor -- start 2024-05-10T03:58:23.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbcad7ac700 1 -- start start 2024-05-10T03:58:23.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbcad7ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca80fffd0 0x7fbca8198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:23.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbcad7ac700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca8105150 0x7fbca8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:23.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbcad7ac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca8107ab0 0x7fbca81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:23.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbcad7ac700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8199a50 con 0x7fbca8107ab0 2024-05-10T03:58:23.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbcad7ac700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8199b90 con 0x7fbca8105150 2024-05-10T03:58:23.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbcad7ac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8199d50 con 0x7fbca80fffd0 2024-05-10T03:58:23.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbca77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca8107ab0 0x7fbca81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:23.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbca6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca80fffd0 0x7fbca8198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:23.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbca67fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca8105150 0x7fbca8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:23.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbca77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca8107ab0 0x7fbca81993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39824/0 (socket says 172.21.15.150:39824) 2024-05-10T03:58:23.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbca6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca80fffd0 0x7fbca8198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:48260/0 (socket says 172.21.15.150:48260) 2024-05-10T03:58:23.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbca67fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca8105150 0x7fbca8198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:34642/0 (socket says 172.21.15.150:34642) 2024-05-10T03:58:23.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.896+0000 7fbca77fe700 1 -- 172.21.15.150:0/3922450300 learned_addr learned my addr 172.21.15.150:0/3922450300 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:58:23.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.897+0000 7fbca6ffd700 1 -- 172.21.15.150:0/3922450300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca8105150 msgr2=0x7fbca8198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:23.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.897+0000 7fbca6ffd700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca8105150 0x7fbca8198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:23.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.897+0000 7fbca6ffd700 1 -- 172.21.15.150:0/3922450300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca8107ab0 msgr2=0x7fbca81993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:23.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.897+0000 7fbca6ffd700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca8107ab0 0x7fbca81993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.897+0000 7fbca6ffd700 1 -- 172.21.15.150:0/3922450300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c0095f0 con 0x7fbca80fffd0 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.897+0000 7fbca77fe700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca8107ab0 0x7fbca81993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.897+0000 7fbca67fc700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca8105150 0x7fbca8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.897+0000 7fbca6ffd700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca80fffd0 0x7fbca8198100 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c000c00 tx=0x7fbc9c005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.897+0000 7fbc8ffff700 1 -- 172.21.15.150:0/3922450300 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c01d070 con 0x7fbca80fffd0 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.897+0000 7fbc8ffff700 1 -- 172.21.15.150:0/3922450300 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc9c021410 con 0x7fbca80fffd0 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.897+0000 7fbcad7ac700 1 -- 172.21.15.150:0/3922450300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca8199fd0 con 0x7fbca80fffd0 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.897+0000 7fbc8ffff700 1 -- 172.21.15.150:0/3922450300 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c017650 con 0x7fbca80fffd0 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.897+0000 7fbcad7ac700 1 -- 172.21.15.150:0/3922450300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca81a0810 con 0x7fbca80fffd0 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.898+0000 7fbc8ffff700 1 -- 172.21.15.150:0/3922450300 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fbc9c0177b0 con 0x7fbca80fffd0 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.898+0000 7fbcad7ac700 1 -- 172.21.15.150:0/3922450300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca80fd500 con 0x7fbca80fffd0 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.898+0000 7fbc8ffff700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc900389d0 0x7fbc9003aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.898+0000 7fbc8ffff700 1 -- 172.21.15.150:0/3922450300 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbc9c04c650 con 0x7fbca80fffd0 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.898+0000 7fbca67fc700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc900389d0 0x7fbc9003aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:23.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.899+0000 7fbca67fc700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc900389d0 0x7fbc9003aee0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbca81990d0 tx=0x7fbc98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:23.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:23.903+0000 7fbc8ffff700 1 -- 172.21.15.150:0/3922450300 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbc9c01f310 con 0x7fbca80fffd0 2024-05-10T03:58:24.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:24.114+0000 7fbcad7ac700 1 -- 172.21.15.150:0/3922450300 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbca810d510 con 0x7fbc900389d0 2024-05-10T03:58:24.146 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:24 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:24.045+0000 7fbbaa1a9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T03:58:24.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:24.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:24.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:23 smithi181 ceph-mon[46602]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:24.147 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:23 smithi181 ceph-mon[46602]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T03:58:24.147 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T03:58:24.147 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T03:58:24.147 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:24.147 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:23 smithi181 ceph-mon[46602]: Reconfiguring daemon mgr.x on smithi150 2024-05-10T03:58:24.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:24.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:24.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:23 smithi150 ceph-mon[42477]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:24.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:23 smithi150 ceph-mon[42477]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T03:58:24.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T03:58:24.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T03:58:24.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:24.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:23 smithi150 ceph-mon[42477]: Reconfiguring daemon mgr.x on smithi150 2024-05-10T03:58:24.243 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:23 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:23.842+0000 7f4bcf4b5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T03:58:24.243 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:23 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:23.971+0000 7f4bcf4b5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T03:58:24.243 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:24 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:24.057+0000 7f4bcf4b5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T03:58:24.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:24.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:24.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:24.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:23 smithi161 ceph-mon[46918]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:24.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:23 smithi161 ceph-mon[46918]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T03:58:24.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T03:58:24.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T03:58:24.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:24.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:23 smithi161 ceph-mon[46918]: Reconfiguring daemon mgr.x on smithi150 2024-05-10T03:58:24.488 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:24 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:24.142+0000 7fbbaa1a9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T03:58:24.488 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:24 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:24.298+0000 7fbbaa1a9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T03:58:24.743 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:24 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:24.436+0000 7f4bcf4b5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T03:58:25.080 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:24 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:24.824+0000 7f4bcf4b5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T03:58:25.080 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:25 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:25.017+0000 7f4bcf4b5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T03:58:25.239 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:24 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:24.933+0000 7fbbaa1a9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T03:58:25.239 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:25 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:25.116+0000 7fbbaa1a9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T03:58:25.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:25.240 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:25 smithi181 ceph-mon[46602]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:58:25.240 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:58:25.240 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:25.240 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:25.241 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:58:25.241 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:25.241 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:25.241 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:25.241 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:25.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:25.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:25.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:25 smithi150 ceph-mon[42477]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:58:25.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:58:25.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:25.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:25.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:58:25.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:25.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:25.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:25.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:25.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:25.493 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:25 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:25.110+0000 7f4bcf4b5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T03:58:25.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:25.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:25 smithi161 ceph-mon[46918]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:58:25.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:58:25.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:25.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:25.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:58:25.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:25.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:25.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:25.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:25.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:25.738 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:25 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:25.314+0000 7fbbaa1a9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T03:58:26.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:26 smithi161 ceph-mon[46918]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:26.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:26 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1089484526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6088d05-bf1a-4fd0-be86-fde95b461d3a"}]: dispatch 2024-05-10T03:58:26.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:26 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1089484526' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6088d05-bf1a-4fd0-be86-fde95b461d3a"}]': finished 2024-05-10T03:58:26.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:26 smithi161 ceph-mon[46918]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T03:58:26.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:26.244 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:25 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:25.874+0000 7f4bcf4b5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T03:58:26.244 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:25 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:25.971+0000 7f4bcf4b5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T03:58:26.244 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:26 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:26.121+0000 7f4bcf4b5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T03:58:26.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:26 smithi150 ceph-mon[42477]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:26.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:26 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1089484526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6088d05-bf1a-4fd0-be86-fde95b461d3a"}]: dispatch 2024-05-10T03:58:26.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:26 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1089484526' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6088d05-bf1a-4fd0-be86-fde95b461d3a"}]': finished 2024-05-10T03:58:26.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:26 smithi150 ceph-mon[42477]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T03:58:26.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:26.488 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:26 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:26.247+0000 7fbbaa1a9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T03:58:26.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:26 smithi181 ceph-mon[46602]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:26.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:26 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1089484526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6088d05-bf1a-4fd0-be86-fde95b461d3a"}]: dispatch 2024-05-10T03:58:26.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:26 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1089484526' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6088d05-bf1a-4fd0-be86-fde95b461d3a"}]': finished 2024-05-10T03:58:26.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:26 smithi181 ceph-mon[46602]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T03:58:26.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:26.817 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:26 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:26.532+0000 7fbbaa1a9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:58:26.817 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:26 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:26.620+0000 7fbbaa1a9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T03:58:26.817 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 03:58:26 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z[48458]: 2024-05-10T03:58:26.813+0000 7fbbaa1a9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T03:58:27.088 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:26 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:26.752+0000 7f4bcf4b5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T03:58:27.089 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:26 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:26.933+0000 7f4bcf4b5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T03:58:27.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:27.088+0000 7fbc8ffff700 1 -- 172.21.15.150:0/3922450300 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 14) v1 ==== 90708+0+0 (secure 0 0 0) 0x7fbc9c021580 con 0x7fbca80fffd0 2024-05-10T03:58:27.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:27 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1197050220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:58:27.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:27 smithi150 ceph-mon[42477]: Standby manager daemon z started 2024-05-10T03:58:27.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:27 smithi150 ceph-mon[42477]: from='mgr.? 172.21.15.181:0/940499167' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T03:58:27.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:27 smithi150 ceph-mon[42477]: from='mgr.? 172.21.15.181:0/940499167' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T03:58:27.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:27 smithi150 ceph-mon[42477]: from='mgr.? 172.21.15.181:0/940499167' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T03:58:27.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:27 smithi150 ceph-mon[42477]: from='mgr.? 172.21.15.181:0/940499167' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T03:58:27.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:27 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1197050220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:58:27.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:27 smithi181 ceph-mon[46602]: Standby manager daemon z started 2024-05-10T03:58:27.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:27 smithi181 ceph-mon[46602]: from='mgr.? 172.21.15.181:0/940499167' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T03:58:27.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:27 smithi181 ceph-mon[46602]: from='mgr.? 172.21.15.181:0/940499167' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T03:58:27.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:27 smithi181 ceph-mon[46602]: from='mgr.? 172.21.15.181:0/940499167' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T03:58:27.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:27 smithi181 ceph-mon[46602]: from='mgr.? 172.21.15.181:0/940499167' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T03:58:27.492 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:27 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:27.129+0000 7f4bcf4b5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T03:58:27.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:27 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1197050220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:58:27.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:27 smithi161 ceph-mon[46918]: Standby manager daemon z started 2024-05-10T03:58:27.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:27 smithi161 ceph-mon[46918]: from='mgr.? 172.21.15.181:0/940499167' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T03:58:27.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:27 smithi161 ceph-mon[46918]: from='mgr.? 172.21.15.181:0/940499167' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T03:58:27.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:27 smithi161 ceph-mon[46918]: from='mgr.? 172.21.15.181:0/940499167' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T03:58:27.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:27 smithi161 ceph-mon[46918]: from='mgr.? 172.21.15.181:0/940499167' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T03:58:28.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:28 smithi161 ceph-mon[46918]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:28.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:28 smithi161 ceph-mon[46918]: mgrmap e14: x(active, since 58s), standbys: z 2024-05-10T03:58:28.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T03:58:28.350 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:28 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:28.059+0000 7f4bcf4b5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T03:58:28.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:28 smithi150 ceph-mon[42477]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:28.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:28 smithi150 ceph-mon[42477]: mgrmap e14: x(active, since 58s), standbys: z 2024-05-10T03:58:28.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T03:58:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:28 smithi181 ceph-mon[46602]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:28 smithi181 ceph-mon[46602]: mgrmap e14: x(active, since 58s), standbys: z 2024-05-10T03:58:28.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T03:58:28.626 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:28 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:28.344+0000 7f4bcf4b5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:58:28.626 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:28 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:28.433+0000 7f4bcf4b5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T03:58:28.993 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 03:58:28 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y[48394]: 2024-05-10T03:58:28.623+0000 7f4bcf4b5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T03:58:29.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:29.098+0000 7fbc8ffff700 1 -- 172.21.15.150:0/3922450300 <== mon.1 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fbc9c029360 con 0x7fbca80fffd0 2024-05-10T03:58:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:29 smithi150 ceph-mon[42477]: Standby manager daemon y started 2024-05-10T03:58:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:29 smithi150 ceph-mon[42477]: from='mgr.? 172.21.15.161:0/1797845768' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T03:58:29.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:29 smithi150 ceph-mon[42477]: from='mgr.? 172.21.15.161:0/1797845768' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T03:58:29.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:29 smithi150 ceph-mon[42477]: from='mgr.? 172.21.15.161:0/1797845768' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T03:58:29.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:29 smithi150 ceph-mon[42477]: from='mgr.? 172.21.15.161:0/1797845768' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T03:58:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:29 smithi181 ceph-mon[46602]: Standby manager daemon y started 2024-05-10T03:58:29.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:29 smithi181 ceph-mon[46602]: from='mgr.? 172.21.15.161:0/1797845768' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T03:58:29.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:29 smithi181 ceph-mon[46602]: from='mgr.? 172.21.15.161:0/1797845768' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T03:58:29.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:29 smithi181 ceph-mon[46602]: from='mgr.? 172.21.15.161:0/1797845768' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T03:58:29.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:29 smithi181 ceph-mon[46602]: from='mgr.? 172.21.15.161:0/1797845768' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T03:58:29.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:29 smithi161 ceph-mon[46918]: Standby manager daemon y started 2024-05-10T03:58:29.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:29 smithi161 ceph-mon[46918]: from='mgr.? 172.21.15.161:0/1797845768' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T03:58:29.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:29 smithi161 ceph-mon[46918]: from='mgr.? 172.21.15.161:0/1797845768' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T03:58:29.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:29 smithi161 ceph-mon[46918]: from='mgr.? 172.21.15.161:0/1797845768' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T03:58:29.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:29 smithi161 ceph-mon[46918]: from='mgr.? 172.21.15.161:0/1797845768' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T03:58:30.352 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:30 smithi150 ceph-mon[42477]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:30.352 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:30 smithi150 ceph-mon[42477]: mgrmap e15: x(active, since 60s), standbys: z, y 2024-05-10T03:58:30.352 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T03:58:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:30 smithi181 ceph-mon[46602]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:30 smithi181 ceph-mon[46602]: mgrmap e15: x(active, since 60s), standbys: z, y 2024-05-10T03:58:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T03:58:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:30 smithi161 ceph-mon[46918]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:30 smithi161 ceph-mon[46918]: mgrmap e15: x(active, since 60s), standbys: z, y 2024-05-10T03:58:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T03:58:32.396 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:32 smithi150 ceph-mon[42477]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:32.396 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T03:58:32.396 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:32 smithi181 ceph-mon[46602]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:32 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T03:58:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:32 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:32.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:32 smithi161 ceph-mon[46918]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:32.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:32 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T03:58:32.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:32 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:33.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:33 smithi150 ceph-mon[42477]: Deploying daemon osd.0 on smithi150 2024-05-10T03:58:33.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:33 smithi181 ceph-mon[46602]: Deploying daemon osd.0 on smithi150 2024-05-10T03:58:33.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:33 smithi161 ceph-mon[46918]: Deploying daemon osd.0 on smithi150 2024-05-10T03:58:34.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:34 smithi150 ceph-mon[42477]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:34 smithi181 ceph-mon[46602]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:34.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:34 smithi161 ceph-mon[46918]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:36.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:36 smithi150 ceph-mon[42477]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:36.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:36 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:36.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:36 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:36.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:36 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:36.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:36 smithi181 ceph-mon[46602]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:36.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:36 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:36.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:36 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:36.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:36 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:36.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:36 smithi161 ceph-mon[46918]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:36.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:36 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:36.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:36 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:36.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:36 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:37.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:37.459+0000 7fbc8ffff700 1 -- 172.21.15.150:0/3922450300 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbca810d510 con 0x7fbc900389d0 2024-05-10T03:58:37.463 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 0 on host 'smithi150' 2024-05-10T03:58:37.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:37.461+0000 7fbcad7ac700 1 -- 172.21.15.150:0/3922450300 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc900389d0 msgr2=0x7fbc9003aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:37.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:37.461+0000 7fbcad7ac700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc900389d0 0x7fbc9003aee0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbca81990d0 tx=0x7fbc98009260 comp rx=0 tx=0).stop 2024-05-10T03:58:37.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:37.461+0000 7fbcad7ac700 1 -- 172.21.15.150:0/3922450300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca80fffd0 msgr2=0x7fbca8198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:37.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:37.461+0000 7fbcad7ac700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca80fffd0 0x7fbca8198100 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c000c00 tx=0x7fbc9c005b00 comp rx=0 tx=0).stop 2024-05-10T03:58:37.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:37.462+0000 7fbcad7ac700 1 -- 172.21.15.150:0/3922450300 shutdown_connections 2024-05-10T03:58:37.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:37.462+0000 7fbcad7ac700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc900389d0 0x7fbc9003aee0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:37.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:37.462+0000 7fbcad7ac700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca8107ab0 0x7fbca81993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:37.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:37.462+0000 7fbcad7ac700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca80fffd0 0x7fbca8198100 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:37.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:37.462+0000 7fbcad7ac700 1 --2- 172.21.15.150:0/3922450300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca8105150 0x7fbca8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:37.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:37.462+0000 7fbcad7ac700 1 -- 172.21.15.150:0/3922450300 >> 172.21.15.150:0/3922450300 conn(0x7fbca80fb950 msgr2=0x7fbca8106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:58:37.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:37.462+0000 7fbcad7ac700 1 -- 172.21.15.150:0/3922450300 shutdown_connections 2024-05-10T03:58:37.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:58:37.462+0000 7fbcad7ac700 1 -- 172.21.15.150:0/3922450300 wait complete. 2024-05-10T03:58:37.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:37 smithi150 ceph-mon[42477]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:37.857 DEBUG:teuthology.orchestra.run.smithi150:osd.0> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.0.service 2024-05-10T03:58:37.859 INFO:tasks.cephadm:Deploying osd.1 on smithi161 with /dev/vg_nvme/lv_4... 2024-05-10T03:58:37.859 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T03:58:37.891 INFO:journalctl@ceph.osd.0.smithi150.stdout:-- Logs begin at Fri 2024-05-10 03:40:30 UTC. -- 2024-05-10T03:58:37.892 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.892 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.892 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:37.892 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:37.892 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.892 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:37 smithi161 ceph-mon[46918]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:37.892 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.892 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.893 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:37.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:37.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:37.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:37 smithi181 ceph-mon[46602]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:37.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:37.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:38.088 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T03:58:39.491 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T03:58:39.491 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T03:58:39.492 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-05-10T03:58:39.492 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-05-10T03:58:39.492 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113759 s, 922 MB/s 2024-05-10T03:58:39.492 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 2024-05-10T03:58:39.492 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-05-10T03:58:39.678 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 03:58:39 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-0[60933]: 2024-05-10T03:58:39.368+0000 7f8db8b857c0 -1 osd.0 0 log_to_monitors true 2024-05-10T03:58:39.867 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-05-10T03:58:40.130 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T03:58:40.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:40 smithi161 ceph-mon[46918]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:40.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:40 smithi161 ceph-mon[46918]: from='osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T03:58:40.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:40 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:40.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:40 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:40.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:40 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:58:40.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:40 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:40.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:40 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:40.436 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:40 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:40.437 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:40 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:40.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:40 smithi150 ceph-mon[42477]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:40.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:40 smithi150 ceph-mon[42477]: from='osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T03:58:40.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:40.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:40.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:58:40.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:40.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:40.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:40.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:40.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:40 smithi181 ceph-mon[46602]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:40.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:40 smithi181 ceph-mon[46602]: from='osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T03:58:40.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:40 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:40.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:40 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:40.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:40 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:58:40.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:40 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:40.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:40 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:40.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:40 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:40.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:40 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:41.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.182+0000 7f2104da9700 1 -- 172.21.15.161:0/967126377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2100100bb0 msgr2=0x7f2100100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.182+0000 7f2104da9700 1 --2- 172.21.15.161:0/967126377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2100100bb0 0x7f2100100fc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009990 tx=0x7f20f0009ca0 comp rx=0 tx=0).stop 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.182+0000 7f2104da9700 1 -- 172.21.15.161:0/967126377 shutdown_connections 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.182+0000 7f2104da9700 1 --2- 172.21.15.161:0/967126377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2100102680 0x7f2100108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.182+0000 7f2104da9700 1 --2- 172.21.15.161:0/967126377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2100100bb0 0x7f2100100fc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.182+0000 7f2104da9700 1 --2- 172.21.15.161:0/967126377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2100101ce0 0x7f21001020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.182+0000 7f2104da9700 1 -- 172.21.15.161:0/967126377 >> 172.21.15.161:0/967126377 conn(0x7f21000fc210 msgr2=0x7f21000fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.182+0000 7f2104da9700 1 -- 172.21.15.161:0/967126377 shutdown_connections 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.182+0000 7f2104da9700 1 -- 172.21.15.161:0/967126377 wait complete. 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.183+0000 7f2104da9700 1 Processor -- start 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.183+0000 7f2104da9700 1 -- start start 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.183+0000 7f2104da9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2100100bb0 0x7f2100197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.183+0000 7f2104da9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2100101ce0 0x7f2100197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.183+0000 7f2104da9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2100102680 0x7f2100198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.183+0000 7f2104da9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21001989a0 con 0x7f2100100bb0 2024-05-10T03:58:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.183+0000 7f2104da9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2100198ae0 con 0x7f2100101ce0 2024-05-10T03:58:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.183+0000 7f2104da9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2100198ca0 con 0x7f2100102680 2024-05-10T03:58:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.183+0000 7f20fdd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2100101ce0 0x7f2100197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.183+0000 7f20fdd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2100101ce0 0x7f2100197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59850/0 (socket says 172.21.15.161:59850) 2024-05-10T03:58:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.183+0000 7f20fdd9b700 1 -- 172.21.15.161:0/397691497 learned_addr learned my addr 172.21.15.161:0/397691497 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-10T03:58:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.183+0000 7f20fe59c700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2100100bb0 0x7f2100197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.183+0000 7f20fed9d700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2100102680 0x7f2100198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.184+0000 7f20fdd9b700 1 -- 172.21.15.161:0/397691497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2100102680 msgr2=0x7f2100198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.184+0000 7f20fdd9b700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2100102680 0x7f2100198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.184+0000 7f20fdd9b700 1 -- 172.21.15.161:0/397691497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2100100bb0 msgr2=0x7f2100197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.184+0000 7f20fdd9b700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2100100bb0 0x7f2100197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.184+0000 7f20fdd9b700 1 -- 172.21.15.161:0/397691497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00095f0 con 0x7f2100101ce0 2024-05-10T03:58:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.184+0000 7f20fdd9b700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2100101ce0 0x7f2100197590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f20f400d950 tx=0x7f20f400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:41.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.184+0000 7f20ef7fe700 1 -- 172.21.15.161:0/397691497 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f400f890 con 0x7f2100101ce0 2024-05-10T03:58:41.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.184+0000 7f20ef7fe700 1 -- 172.21.15.161:0/397691497 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20f4010460 con 0x7f2100101ce0 2024-05-10T03:58:41.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.184+0000 7f2104da9700 1 -- 172.21.15.161:0/397691497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2100198f80 con 0x7f2100101ce0 2024-05-10T03:58:41.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.184+0000 7f20ef7fe700 1 -- 172.21.15.161:0/397691497 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f400e610 con 0x7f2100101ce0 2024-05-10T03:58:41.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.184+0000 7f2104da9700 1 -- 172.21.15.161:0/397691497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2100073610 con 0x7f2100101ce0 2024-05-10T03:58:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.185+0000 7f2104da9700 1 -- 172.21.15.161:0/397691497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21000ffeb0 con 0x7f2100101ce0 2024-05-10T03:58:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.186+0000 7f20ef7fe700 1 -- 172.21.15.161:0/397691497 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7f20f40108d0 con 0x7f2100101ce0 2024-05-10T03:58:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.187+0000 7f20ef7fe700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f20e80a1110 0x7f20e80a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.187+0000 7f20ef7fe700 1 -- 172.21.15.161:0/397691497 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7f20f40caab0 con 0x7f2100101ce0 2024-05-10T03:58:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.188+0000 7f20fe59c700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f20e80a1110 0x7f20e80a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.188+0000 7f20fe59c700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f20e80a1110 0x7f20e80a3620 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20f000b5c0 tx=0x7f20f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:41.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.191+0000 7f20ef7fe700 1 -- 172.21.15.161:0/397691497 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f20f4095db0 con 0x7f2100101ce0 2024-05-10T03:58:41.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:41.391+0000 7f2104da9700 1 -- 172.21.15.161:0/397691497 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2100073830 con 0x7f20e80a1110 2024-05-10T03:58:41.464 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 03:58:41 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-0[60933]: 2024-05-10T03:58:41.092+0000 7f8dae1e9700 -1 osd.0 0 waiting for initial osdmap 2024-05-10T03:58:41.464 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 03:58:41 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-0[60933]: 2024-05-10T03:58:41.099+0000 7f8da9fa0700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T03:58:41.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:41 smithi150 ceph-mon[42477]: Detected new or changed devices on smithi150 2024-05-10T03:58:41.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:41 smithi150 ceph-mon[42477]: Adjusting osd_memory_target on smithi150 to 17114M 2024-05-10T03:58:41.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:41 smithi150 ceph-mon[42477]: from='osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T03:58:41.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:41 smithi150 ceph-mon[42477]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T03:58:41.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:41 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:41.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:41 smithi150 ceph-mon[42477]: from='osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T03:58:41.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:41 smithi181 ceph-mon[46602]: Detected new or changed devices on smithi150 2024-05-10T03:58:41.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:41 smithi181 ceph-mon[46602]: Adjusting osd_memory_target on smithi150 to 17114M 2024-05-10T03:58:41.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:41 smithi181 ceph-mon[46602]: from='osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T03:58:41.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:41 smithi181 ceph-mon[46602]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T03:58:41.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:41 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:41.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:41 smithi181 ceph-mon[46602]: from='osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T03:58:41.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:41 smithi161 ceph-mon[46918]: Detected new or changed devices on smithi150 2024-05-10T03:58:41.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:41 smithi161 ceph-mon[46918]: Adjusting osd_memory_target on smithi150 to 17114M 2024-05-10T03:58:41.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:41 smithi161 ceph-mon[46918]: from='osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T03:58:41.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:41 smithi161 ceph-mon[46918]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T03:58:41.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:41 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:41.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:41 smithi161 ceph-mon[46918]: from='osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T03:58:42.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:42 smithi150 ceph-mon[42477]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:42.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:42 smithi150 ceph-mon[42477]: from='osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T03:58:42.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:42 smithi150 ceph-mon[42477]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T03:58:42.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:42 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:42.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:42 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:42.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:42 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:58:42.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:42 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:58:42.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:42 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:42.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:42 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:42.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:42 smithi181 ceph-mon[46602]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:42.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:42 smithi181 ceph-mon[46602]: from='osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T03:58:42.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:42 smithi181 ceph-mon[46602]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T03:58:42.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:42 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:42.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:42 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:42.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:42 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:58:42.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:42 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:58:42.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:42 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:42.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:42 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:42.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:42 smithi161 ceph-mon[46918]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:42.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:42 smithi161 ceph-mon[46918]: from='osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T03:58:42.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:42 smithi161 ceph-mon[46918]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T03:58:42.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:42 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:42.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:42 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:42.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:42 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:58:42.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:42 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:58:42.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:42 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:42.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:42 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:43.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:43 smithi150 ceph-mon[42477]: purged_snaps scrub starts 2024-05-10T03:58:43.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:43 smithi150 ceph-mon[42477]: purged_snaps scrub ok 2024-05-10T03:58:43.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:43 smithi150 ceph-mon[42477]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:58:43.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:43 smithi150 ceph-mon[42477]: osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215] boot 2024-05-10T03:58:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:43 smithi150 ceph-mon[42477]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T03:58:43.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:43 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:43.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:43 smithi181 ceph-mon[46602]: purged_snaps scrub starts 2024-05-10T03:58:43.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:43 smithi181 ceph-mon[46602]: purged_snaps scrub ok 2024-05-10T03:58:43.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:43 smithi181 ceph-mon[46602]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:58:43.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:43 smithi181 ceph-mon[46602]: osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215] boot 2024-05-10T03:58:43.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:43 smithi181 ceph-mon[46602]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T03:58:43.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:43 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:43.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:43 smithi161 ceph-mon[46918]: purged_snaps scrub starts 2024-05-10T03:58:43.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:43 smithi161 ceph-mon[46918]: purged_snaps scrub ok 2024-05-10T03:58:43.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:43 smithi161 ceph-mon[46918]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:58:43.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:43 smithi161 ceph-mon[46918]: osd.0 [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215] boot 2024-05-10T03:58:43.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:43 smithi161 ceph-mon[46918]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T03:58:43.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:43 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T03:58:44.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:44 smithi150 ceph-mon[42477]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:44.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:44 smithi150 ceph-mon[42477]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T03:58:44.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:44 smithi150 ceph-mon[42477]: from='client.? 172.21.15.161:0/2291600061' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cef6c663-3fc6-4ea4-9dd8-e95845b8bac8"}]: dispatch 2024-05-10T03:58:44.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:44 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cef6c663-3fc6-4ea4-9dd8-e95845b8bac8"}]: dispatch 2024-05-10T03:58:44.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:44 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cef6c663-3fc6-4ea4-9dd8-e95845b8bac8"}]': finished 2024-05-10T03:58:44.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:44 smithi150 ceph-mon[42477]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T03:58:44.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:44 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:58:44.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:44 smithi150 ceph-mon[42477]: from='client.? 172.21.15.161:0/366652582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:58:44.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:44 smithi181 ceph-mon[46602]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:44.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:44 smithi181 ceph-mon[46602]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T03:58:44.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:44 smithi181 ceph-mon[46602]: from='client.? 172.21.15.161:0/2291600061' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cef6c663-3fc6-4ea4-9dd8-e95845b8bac8"}]: dispatch 2024-05-10T03:58:44.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:44 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cef6c663-3fc6-4ea4-9dd8-e95845b8bac8"}]: dispatch 2024-05-10T03:58:44.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:44 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cef6c663-3fc6-4ea4-9dd8-e95845b8bac8"}]': finished 2024-05-10T03:58:44.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:44 smithi181 ceph-mon[46602]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T03:58:44.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:44 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:58:44.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:44 smithi181 ceph-mon[46602]: from='client.? 172.21.15.161:0/366652582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:58:44.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:44 smithi161 ceph-mon[46918]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T03:58:44.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:44 smithi161 ceph-mon[46918]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T03:58:44.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:44 smithi161 ceph-mon[46918]: from='client.? 172.21.15.161:0/2291600061' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cef6c663-3fc6-4ea4-9dd8-e95845b8bac8"}]: dispatch 2024-05-10T03:58:44.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:44 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cef6c663-3fc6-4ea4-9dd8-e95845b8bac8"}]: dispatch 2024-05-10T03:58:44.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:44 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cef6c663-3fc6-4ea4-9dd8-e95845b8bac8"}]': finished 2024-05-10T03:58:44.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:44 smithi161 ceph-mon[46918]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T03:58:44.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:44 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:58:44.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:44 smithi161 ceph-mon[46918]: from='client.? 172.21.15.161:0/366652582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:58:46.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:46 smithi150 ceph-mon[42477]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:46.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:46 smithi181 ceph-mon[46602]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:46.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:46 smithi161 ceph-mon[46918]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:48.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:48 smithi150 ceph-mon[42477]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:48.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:48 smithi181 ceph-mon[46602]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:48.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:48 smithi161 ceph-mon[46918]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:50.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:50 smithi161 ceph-mon[46918]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:50.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:50 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T03:58:50.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:50 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:50.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:50 smithi150 ceph-mon[42477]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:50.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:50 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T03:58:50.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:50 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:50.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:50 smithi181 ceph-mon[46602]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:50.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:50 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T03:58:50.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:50 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:51.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:51 smithi161 ceph-mon[46918]: Deploying daemon osd.1 on smithi161 2024-05-10T03:58:51.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:51 smithi150 ceph-mon[42477]: Deploying daemon osd.1 on smithi161 2024-05-10T03:58:51.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:51 smithi181 ceph-mon[46602]: Deploying daemon osd.1 on smithi161 2024-05-10T03:58:52.461 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:52 smithi161 ceph-mon[46918]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:52.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:52 smithi150 ceph-mon[42477]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:52.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:52 smithi181 ceph-mon[46602]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:54.404 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:54 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:54.405 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:54 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:54.405 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:54 smithi161 ceph-mon[46918]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:54.405 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:54 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:54.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:54 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:54.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:54 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:54.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:54 smithi150 ceph-mon[42477]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:54.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:54 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:54.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:54 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:54.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:54 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:54.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:54 smithi181 ceph-mon[46602]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:54.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:54 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:55.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:55.443+0000 7f20ef7fe700 1 -- 172.21.15.161:0/397691497 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2100073830 con 0x7f20e80a1110 2024-05-10T03:58:55.451 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 1 on host 'smithi161' 2024-05-10T03:58:55.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:55.446+0000 7f2104da9700 1 -- 172.21.15.161:0/397691497 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f20e80a1110 msgr2=0x7f20e80a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:55.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:55.446+0000 7f2104da9700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f20e80a1110 0x7f20e80a3620 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20f000b5c0 tx=0x7f20f0005fb0 comp rx=0 tx=0).stop 2024-05-10T03:58:55.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:55.446+0000 7f2104da9700 1 -- 172.21.15.161:0/397691497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2100101ce0 msgr2=0x7f2100197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:55.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:55.446+0000 7f2104da9700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2100101ce0 0x7f2100197590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f20f400d950 tx=0x7f20f400dc60 comp rx=0 tx=0).stop 2024-05-10T03:58:55.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:55.446+0000 7f2104da9700 1 -- 172.21.15.161:0/397691497 shutdown_connections 2024-05-10T03:58:55.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:55.446+0000 7f2104da9700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f20e80a1110 0x7f20e80a3620 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:55.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:55.446+0000 7f2104da9700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2100102680 0x7f2100198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:55.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:55.446+0000 7f2104da9700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2100100bb0 0x7f2100197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:55.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:55.446+0000 7f2104da9700 1 --2- 172.21.15.161:0/397691497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2100101ce0 0x7f2100197590 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:55.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:55.446+0000 7f2104da9700 1 -- 172.21.15.161:0/397691497 >> 172.21.15.161:0/397691497 conn(0x7f21000fc210 msgr2=0x7f2100104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:58:55.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:55.446+0000 7f2104da9700 1 -- 172.21.15.161:0/397691497 shutdown_connections 2024-05-10T03:58:55.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:58:55.446+0000 7f2104da9700 1 -- 172.21.15.161:0/397691497 wait complete. 2024-05-10T03:58:55.710 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:55.711 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:55.711 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:55.711 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:55.711 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:55.711 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:55 smithi161 ceph-mon[46918]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:55.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:55.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:55.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:55.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:55.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:55.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:55 smithi150 ceph-mon[42477]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:55.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:55.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:55.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:55.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:55.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:55.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:55 smithi181 ceph-mon[46602]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:55.908 DEBUG:teuthology.orchestra.run.smithi161:osd.1> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.1.service 2024-05-10T03:58:55.915 INFO:tasks.cephadm:Deploying osd.2 on smithi181 with /dev/vg_nvme/lv_4... 2024-05-10T03:58:55.915 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T03:58:55.945 INFO:journalctl@ceph.osd.1.smithi161.stdout:-- Logs begin at Fri 2024-05-10 03:40:24 UTC. -- 2024-05-10T03:58:56.150 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T03:58:56.612 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:56 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:56.612 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:56 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:56.613 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:56 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:56.685 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:56 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:56.685 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:56 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:56.685 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:56 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:56.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:56 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:56.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:56 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:56.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:56 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:58:57.431 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 03:58:57 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-1[51890]: 2024-05-10T03:58:57.011+0000 7f262efb77c0 -1 osd.1 0 log_to_monitors true 2024-05-10T03:58:57.529 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T03:58:57.529 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T03:58:57.529 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-05-10T03:58:57.529 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-05-10T03:58:57.529 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0157323 s, 667 MB/s 2024-05-10T03:58:57.530 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-05-10T03:58:57.704 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:57 smithi181 ceph-mon[46602]: from='osd.1 [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T03:58:57.704 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:57 smithi181 ceph-mon[46602]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T03:58:57.704 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:57 smithi181 ceph-mon[46602]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:57.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:57 smithi150 ceph-mon[42477]: from='osd.1 [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T03:58:57.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:57 smithi150 ceph-mon[42477]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T03:58:57.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:57 smithi150 ceph-mon[42477]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:57.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:57 smithi161 ceph-mon[46918]: from='osd.1 [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T03:58:57.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:57 smithi161 ceph-mon[46918]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T03:58:57.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:57 smithi161 ceph-mon[46918]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:57.908 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_4 2024-05-10T03:58:58.132 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T03:58:58.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T03:58:58.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T03:58:58.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:58:58.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: from='osd.1 [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T03:58:58.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T03:58:58.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: Detected new or changed devices on smithi161 2024-05-10T03:58:58.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:58.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:58.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:58:58.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: Adjusting osd_memory_target on smithi161 to 17114M 2024-05-10T03:58:58.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:58.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:58.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:58.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:58 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:58.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T03:58:58.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T03:58:58.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:58:58.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: from='osd.1 [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T03:58:58.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T03:58:58.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: Detected new or changed devices on smithi161 2024-05-10T03:58:58.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:58.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:58.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:58:58.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: Adjusting osd_memory_target on smithi161 to 17114M 2024-05-10T03:58:58.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:58.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:58.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:58.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:58 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:58.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T03:58:58.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T03:58:58.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:58:58.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: from='osd.1 [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T03:58:58.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T03:58:58.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: Detected new or changed devices on smithi161 2024-05-10T03:58:58.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:58.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:58.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:58:58.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: Adjusting osd_memory_target on smithi161 to 17114M 2024-05-10T03:58:58.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:58.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:58.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:58:58.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:58 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:58:59.160 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.156+0000 7ff343cde700 1 -- 172.21.15.181:0/948400396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c100b80 msgr2=0x7ff33c100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:59.160 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.156+0000 7ff343cde700 1 --2- 172.21.15.181:0/948400396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c100b80 0x7ff33c100f90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff338009990 tx=0x7ff338009ca0 comp rx=0 tx=0).stop 2024-05-10T03:58:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.156+0000 7ff343cde700 1 -- 172.21.15.181:0/948400396 shutdown_connections 2024-05-10T03:58:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.156+0000 7ff343cde700 1 --2- 172.21.15.181:0/948400396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c102650 0x7ff33c108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.156+0000 7ff343cde700 1 --2- 172.21.15.181:0/948400396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c100b80 0x7ff33c100f90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.156+0000 7ff343cde700 1 --2- 172.21.15.181:0/948400396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c101cb0 0x7ff33c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.156+0000 7ff343cde700 1 -- 172.21.15.181:0/948400396 >> 172.21.15.181:0/948400396 conn(0x7ff33c0fc1c0 msgr2=0x7ff33c0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:58:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.156+0000 7ff343cde700 1 -- 172.21.15.181:0/948400396 shutdown_connections 2024-05-10T03:58:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.157+0000 7ff343cde700 1 -- 172.21.15.181:0/948400396 wait complete. 2024-05-10T03:58:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.157+0000 7ff343cde700 1 Processor -- start 2024-05-10T03:58:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.157+0000 7ff343cde700 1 -- start start 2024-05-10T03:58:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.157+0000 7ff343cde700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c100b80 0x7ff33c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.157+0000 7ff343cde700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c101cb0 0x7ff33c19d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.157+0000 7ff343cde700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c102650 0x7ff33c19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.157+0000 7ff343cde700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c069c70 con 0x7ff33c102650 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.157+0000 7ff343cde700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c19fdf0 con 0x7ff33c101cb0 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.157+0000 7ff343cde700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c19ff60 con 0x7ff33c100b80 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff341a7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c100b80 0x7ff33c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff341279700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c101cb0 0x7ff33c19d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff341a7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c100b80 0x7ff33c19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53330/0 (socket says 172.21.15.181:53330) 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff341a7a700 1 -- 172.21.15.181:0/2600608272 learned_addr learned my addr 172.21.15.181:0/2600608272 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff34227b700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c102650 0x7ff33c19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff341a7a700 1 -- 172.21.15.181:0/2600608272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c101cb0 msgr2=0x7ff33c19d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff341a7a700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c101cb0 0x7ff33c19d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff341a7a700 1 -- 172.21.15.181:0/2600608272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c102650 msgr2=0x7ff33c19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff341a7a700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c102650 0x7ff33c19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff341a7a700 1 -- 172.21.15.181:0/2600608272 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3380095f0 con 0x7ff33c100b80 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff341279700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c101cb0 0x7ff33c19d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:58:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff341a7a700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c100b80 0x7ff33c19ae10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff3380098e0 tx=0x7ff338009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:59.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff32effd700 1 -- 172.21.15.181:0/2600608272 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33801d070 con 0x7ff33c100b80 2024-05-10T03:58:59.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff32effd700 1 -- 172.21.15.181:0/2600608272 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff338021410 con 0x7ff33c100b80 2024-05-10T03:58:59.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff343cde700 1 -- 172.21.15.181:0/2600608272 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c1a01e0 con 0x7ff33c100b80 2024-05-10T03:58:59.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff32effd700 1 -- 172.21.15.181:0/2600608272 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff338017650 con 0x7ff33c100b80 2024-05-10T03:58:59.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.158+0000 7ff343cde700 1 -- 172.21.15.181:0/2600608272 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c1a0650 con 0x7ff33c100b80 2024-05-10T03:58:59.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.159+0000 7ff343cde700 1 -- 172.21.15.181:0/2600608272 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33c107e10 con 0x7ff33c100b80 2024-05-10T03:58:59.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.160+0000 7ff32effd700 1 -- 172.21.15.181:0/2600608272 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7ff338021580 con 0x7ff33c100b80 2024-05-10T03:58:59.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.161+0000 7ff32effd700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff3280a1110 0x7ff3280a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:58:59.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.161+0000 7ff32effd700 1 -- 172.21.15.181:0/2600608272 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7ff3380cc900 con 0x7ff33c100b80 2024-05-10T03:58:59.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.161+0000 7ff341279700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff3280a1110 0x7ff3280a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:58:59.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.162+0000 7ff341279700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff3280a1110 0x7ff3280a3620 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff33c069130 tx=0x7ff33000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:58:59.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.165+0000 7ff32effd700 1 -- 172.21.15.181:0/2600608272 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff338097a90 con 0x7ff33c100b80 2024-05-10T03:58:59.243 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 03:58:58 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-1[51890]: 2024-05-10T03:58:58.877+0000 7f262461b700 -1 osd.1 0 waiting for initial osdmap 2024-05-10T03:58:59.243 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 03:58:58 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-1[51890]: 2024-05-10T03:58:58.881+0000 7f261c3ca700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T03:58:59.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:58:59.369+0000 7ff343cde700 1 -- 172.21.15.181:0/2600608272 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff33c10c550 con 0x7ff3280a1110 2024-05-10T03:58:59.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:59 smithi150 ceph-mon[42477]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T03:58:59.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:59 smithi150 ceph-mon[42477]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T03:58:59.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:59 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:58:59.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:59 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:58:59.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:59 smithi150 ceph-mon[42477]: from='osd.1 ' entity='osd.1' 2024-05-10T03:58:59.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:59 smithi150 ceph-mon[42477]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:59.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:59 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:58:59.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:59 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:58:59.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:58:59 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:59.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:59 smithi181 ceph-mon[46602]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T03:58:59.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:59 smithi181 ceph-mon[46602]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T03:58:59.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:58:59.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:58:59.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:59 smithi181 ceph-mon[46602]: from='osd.1 ' entity='osd.1' 2024-05-10T03:58:59.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:59 smithi181 ceph-mon[46602]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:59.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:58:59.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:58:59.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:58:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:58:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:59 smithi161 ceph-mon[46918]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T03:58:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:59 smithi161 ceph-mon[46918]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T03:58:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:59 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:58:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:59 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:58:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:59 smithi161 ceph-mon[46918]: from='osd.1 ' entity='osd.1' 2024-05-10T03:58:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:59 smithi161 ceph-mon[46918]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T03:58:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:59 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:58:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:59 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:58:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:58:59 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:00.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:00 smithi150 ceph-mon[42477]: purged_snaps scrub starts 2024-05-10T03:59:00.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:00 smithi150 ceph-mon[42477]: purged_snaps scrub ok 2024-05-10T03:59:00.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:00 smithi150 ceph-mon[42477]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:59:00.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:00 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:59:00.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:00 smithi150 ceph-mon[42477]: osd.1 [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735] boot 2024-05-10T03:59:00.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:00 smithi150 ceph-mon[42477]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T03:59:00.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:00 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:59:00.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:00 smithi181 ceph-mon[46602]: purged_snaps scrub starts 2024-05-10T03:59:00.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:00 smithi181 ceph-mon[46602]: purged_snaps scrub ok 2024-05-10T03:59:00.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:00 smithi181 ceph-mon[46602]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:59:00.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:00 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:59:00.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:00 smithi181 ceph-mon[46602]: osd.1 [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735] boot 2024-05-10T03:59:00.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:00 smithi181 ceph-mon[46602]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T03:59:00.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:00 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:59:00.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:00 smithi161 ceph-mon[46918]: purged_snaps scrub starts 2024-05-10T03:59:00.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:00 smithi161 ceph-mon[46918]: purged_snaps scrub ok 2024-05-10T03:59:00.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:00 smithi161 ceph-mon[46918]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:59:00.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:00 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:59:00.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:00 smithi161 ceph-mon[46918]: osd.1 [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735] boot 2024-05-10T03:59:00.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:00 smithi161 ceph-mon[46918]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T03:59:00.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:00 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T03:59:02.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:01 smithi150 ceph-mon[42477]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T03:59:02.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:01 smithi150 ceph-mon[42477]: pgmap v50: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:02.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:01 smithi150 ceph-mon[42477]: from='client.? 172.21.15.181:0/3951163155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27256076-4b91-40ac-a69b-9e20e0f37b4c"}]: dispatch 2024-05-10T03:59:02.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:01 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27256076-4b91-40ac-a69b-9e20e0f37b4c"}]: dispatch 2024-05-10T03:59:02.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:01 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27256076-4b91-40ac-a69b-9e20e0f37b4c"}]': finished 2024-05-10T03:59:02.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:01 smithi150 ceph-mon[42477]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T03:59:02.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:02.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:01 smithi181 ceph-mon[46602]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T03:59:02.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:01 smithi181 ceph-mon[46602]: pgmap v50: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:02.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:01 smithi181 ceph-mon[46602]: from='client.? 172.21.15.181:0/3951163155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27256076-4b91-40ac-a69b-9e20e0f37b4c"}]: dispatch 2024-05-10T03:59:02.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:01 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27256076-4b91-40ac-a69b-9e20e0f37b4c"}]: dispatch 2024-05-10T03:59:02.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:01 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27256076-4b91-40ac-a69b-9e20e0f37b4c"}]': finished 2024-05-10T03:59:02.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:01 smithi181 ceph-mon[46602]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T03:59:02.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:01 smithi161 ceph-mon[46918]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T03:59:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:01 smithi161 ceph-mon[46918]: pgmap v50: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:01 smithi161 ceph-mon[46918]: from='client.? 172.21.15.181:0/3951163155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27256076-4b91-40ac-a69b-9e20e0f37b4c"}]: dispatch 2024-05-10T03:59:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:01 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27256076-4b91-40ac-a69b-9e20e0f37b4c"}]: dispatch 2024-05-10T03:59:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:01 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27256076-4b91-40ac-a69b-9e20e0f37b4c"}]': finished 2024-05-10T03:59:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:01 smithi161 ceph-mon[46918]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T03:59:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:03.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:02 smithi150 ceph-mon[42477]: from='client.? 172.21.15.181:0/76237830' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:59:03.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:02 smithi181 ceph-mon[46602]: from='client.? 172.21.15.181:0/76237830' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:59:03.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:02 smithi161 ceph-mon[46918]: from='client.? 172.21.15.181:0/76237830' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:59:04.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:03 smithi150 ceph-mon[42477]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:04.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:03 smithi181 ceph-mon[46602]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:04.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:03 smithi161 ceph-mon[46918]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:06.445 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:06 smithi181 ceph-mon[46602]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:06.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:06 smithi150 ceph-mon[42477]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:06.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:06 smithi161 ceph-mon[46918]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:08.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:08 smithi181 ceph-mon[46602]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:08.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:08 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T03:59:08.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:08 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:08.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:08 smithi150 ceph-mon[42477]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:08.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:08 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T03:59:08.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:08 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:08.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:08 smithi161 ceph-mon[46918]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:08.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:08 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T03:59:08.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:08 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:09.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:09 smithi181 ceph-mon[46602]: Deploying daemon osd.2 on smithi181 2024-05-10T03:59:09.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:09 smithi150 ceph-mon[42477]: Deploying daemon osd.2 on smithi181 2024-05-10T03:59:09.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:09 smithi161 ceph-mon[46918]: Deploying daemon osd.2 on smithi181 2024-05-10T03:59:10.365 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:10 smithi181 ceph-mon[46602]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:10.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:10 smithi150 ceph-mon[42477]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:10.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:10 smithi161 ceph-mon[46918]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:12.461 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:12 smithi181 ceph-mon[46602]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:12.461 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:12.461 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:12.461 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:12 smithi150 ceph-mon[42477]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:12.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:12 smithi161 ceph-mon[46918]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:12.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:12.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:12.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:13.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:13.422+0000 7ff32effd700 1 -- 172.21.15.181:0/2600608272 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff33c10c550 con 0x7ff3280a1110 2024-05-10T03:59:13.426 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 2 on host 'smithi181' 2024-05-10T03:59:13.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:13.425+0000 7ff343cde700 1 -- 172.21.15.181:0/2600608272 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff3280a1110 msgr2=0x7ff3280a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:13.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:13.425+0000 7ff343cde700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff3280a1110 0x7ff3280a3620 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff33c069130 tx=0x7ff33000a380 comp rx=0 tx=0).stop 2024-05-10T03:59:13.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:13.425+0000 7ff343cde700 1 -- 172.21.15.181:0/2600608272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c100b80 msgr2=0x7ff33c19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:13.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:13.425+0000 7ff343cde700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c100b80 0x7ff33c19ae10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff3380098e0 tx=0x7ff338009580 comp rx=0 tx=0).stop 2024-05-10T03:59:13.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:13.425+0000 7ff343cde700 1 -- 172.21.15.181:0/2600608272 shutdown_connections 2024-05-10T03:59:13.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:13.425+0000 7ff343cde700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff3280a1110 0x7ff3280a3620 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:13.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:13.425+0000 7ff343cde700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c102650 0x7ff33c19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:13.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:13.425+0000 7ff343cde700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c100b80 0x7ff33c19ae10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:13.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:13.425+0000 7ff343cde700 1 --2- 172.21.15.181:0/2600608272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff33c101cb0 0x7ff33c19d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:13.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:13.425+0000 7ff343cde700 1 -- 172.21.15.181:0/2600608272 >> 172.21.15.181:0/2600608272 conn(0x7ff33c0fc1c0 msgr2=0x7ff33c104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:59:13.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:13.425+0000 7ff343cde700 1 -- 172.21.15.181:0/2600608272 shutdown_connections 2024-05-10T03:59:13.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:13.425+0000 7ff343cde700 1 -- 172.21.15.181:0/2600608272 wait complete. 2024-05-10T03:59:13.593 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:13.593 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:13.593 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:13.593 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:13.593 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:13.593 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:13 smithi181 ceph-mon[46602]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:13.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:13.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:13.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:13.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:13.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:13.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:13 smithi150 ceph-mon[42477]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:13.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:13.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:13.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:13.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:13.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:13.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:13 smithi161 ceph-mon[46918]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:13.812 DEBUG:teuthology.orchestra.run.smithi181:osd.2> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.2.service 2024-05-10T03:59:13.814 INFO:tasks.cephadm:Deploying osd.3 on smithi150 with /dev/vg_nvme/lv_3... 2024-05-10T03:59:13.814 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T03:59:13.847 INFO:journalctl@ceph.osd.2.smithi181.stdout:-- Logs begin at Fri 2024-05-10 03:40:19 UTC. -- 2024-05-10T03:59:14.049 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T03:59:14.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:14.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:14.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:14.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:14.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:14.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:14.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:14.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:14.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:15.362 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T03:59:15.362 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T03:59:15.362 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-10T03:59:15.362 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-10T03:59:15.362 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0150135 s, 698 MB/s 2024-05-10T03:59:15.362 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-10T03:59:15.414 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 03:59:15 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-2[52428]: 2024-05-10T03:59:15.078+0000 7fedd087f7c0 -1 osd.2 0 log_to_monitors true 2024-05-10T03:59:15.652 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:15 smithi150 ceph-mon[42477]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:15.652 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:15 smithi150 ceph-mon[42477]: from='osd.2 [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T03:59:15.652 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:15 smithi150 ceph-mon[42477]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T03:59:15.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:15 smithi181 ceph-mon[46602]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:15.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:15 smithi181 ceph-mon[46602]: from='osd.2 [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T03:59:15.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:15 smithi181 ceph-mon[46602]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T03:59:15.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:15 smithi161 ceph-mon[46918]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:15.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:15 smithi161 ceph-mon[46918]: from='osd.2 [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T03:59:15.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:15 smithi161 ceph-mon[46918]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T03:59:15.753 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-05-10T03:59:15.972 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T03:59:16.698 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T03:59:16.699 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T03:59:16.699 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:16.699 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: from='osd.2 [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T03:59:16.699 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T03:59:16.699 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: Detected new or changed devices on smithi181 2024-05-10T03:59:16.699 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:16.699 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:16.699 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:16.699 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: Adjusting osd_memory_target on smithi181 to 17113M 2024-05-10T03:59:16.699 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:16.699 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:16.699 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:16.699 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:16.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T03:59:16.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T03:59:16.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:16.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: from='osd.2 [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T03:59:16.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T03:59:16.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: Detected new or changed devices on smithi181 2024-05-10T03:59:16.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:16.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:16.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:16.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: Adjusting osd_memory_target on smithi181 to 17113M 2024-05-10T03:59:16.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:16.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:16.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:16.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:16.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T03:59:16.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T03:59:16.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:16.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: from='osd.2 [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T03:59:16.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T03:59:16.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: Detected new or changed devices on smithi181 2024-05-10T03:59:16.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:16.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:16.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:16.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: Adjusting osd_memory_target on smithi181 to 17113M 2024-05-10T03:59:16.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:16.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:16.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:16.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:16.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.877+0000 7f32750d4700 1 -- 172.21.15.150:0/1554685142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3270101720 msgr2=0x7f3270109030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:16.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.877+0000 7f32750d4700 1 --2- 172.21.15.150:0/1554685142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3270101720 0x7f3270109030 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f326800b0a0 tx=0x7f326800b3b0 comp rx=0 tx=0).stop 2024-05-10T03:59:16.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.877+0000 7f32750d4700 1 -- 172.21.15.150:0/1554685142 shutdown_connections 2024-05-10T03:59:16.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.877+0000 7f32750d4700 1 --2- 172.21.15.150:0/1554685142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3270101720 0x7f3270109030 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:16.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.877+0000 7f32750d4700 1 --2- 172.21.15.150:0/1554685142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32701020c0 0x7f32701024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:16.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.877+0000 7f32750d4700 1 --2- 172.21.15.150:0/1554685142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3270100dd0 0x7f32701011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:16.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.877+0000 7f32750d4700 1 -- 172.21.15.150:0/1554685142 >> 172.21.15.150:0/1554685142 conn(0x7f32700fb990 msgr2=0x7f32700fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:59:16.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.878+0000 7f32750d4700 1 -- 172.21.15.150:0/1554685142 shutdown_connections 2024-05-10T03:59:16.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.878+0000 7f32750d4700 1 -- 172.21.15.150:0/1554685142 wait complete. 2024-05-10T03:59:16.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.879+0000 7f32750d4700 1 Processor -- start 2024-05-10T03:59:16.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.879+0000 7f32750d4700 1 -- start start 2024-05-10T03:59:16.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.879+0000 7f32750d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3270100dd0 0x7f3270197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:59:16.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.879+0000 7f32750d4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3270101720 0x7f3270197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:59:16.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.879+0000 7f32750d4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32701020c0 0x7f3270198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:59:16.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.879+0000 7f32750d4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32701989d0 con 0x7f32701020c0 2024-05-10T03:59:16.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.879+0000 7f32750d4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3270198b10 con 0x7f3270101720 2024-05-10T03:59:16.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.879+0000 7f32750d4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3270198cd0 con 0x7f3270100dd0 2024-05-10T03:59:16.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.879+0000 7f326f59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32701020c0 0x7f3270198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:59:16.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.879+0000 7f326f59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32701020c0 0x7f3270198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60128/0 (socket says 172.21.15.150:60128) 2024-05-10T03:59:16.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.879+0000 7f326f59e700 1 -- 172.21.15.150:0/2767347098 learned_addr learned my addr 172.21.15.150:0/2767347098 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T03:59:16.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.879+0000 7f326ed9d700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3270100dd0 0x7f3270197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:59:16.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.879+0000 7f326e59c700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3270101720 0x7f3270197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:59:16.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.880+0000 7f326f59e700 1 -- 172.21.15.150:0/2767347098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3270100dd0 msgr2=0x7f3270197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:16.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.880+0000 7f326f59e700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3270100dd0 0x7f3270197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:16.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.880+0000 7f326f59e700 1 -- 172.21.15.150:0/2767347098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3270101720 msgr2=0x7f3270197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:16.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.880+0000 7f326f59e700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3270101720 0x7f3270197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:16.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.880+0000 7f326f59e700 1 -- 172.21.15.150:0/2767347098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3268009d00 con 0x7f32701020c0 2024-05-10T03:59:16.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.880+0000 7f326e59c700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3270101720 0x7f3270197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:59:16.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.880+0000 7f326f59e700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32701020c0 0x7f3270198400 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3268006130 tx=0x7f3268007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:59:16.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.880+0000 7f3257fff700 1 -- 172.21.15.150:0/2767347098 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326800e040 con 0x7f32701020c0 2024-05-10T03:59:16.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.880+0000 7f3257fff700 1 -- 172.21.15.150:0/2767347098 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3268004c50 con 0x7f32701020c0 2024-05-10T03:59:16.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.880+0000 7f32750d4700 1 -- 172.21.15.150:0/2767347098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3270198f50 con 0x7f32701020c0 2024-05-10T03:59:16.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.880+0000 7f3257fff700 1 -- 172.21.15.150:0/2767347098 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32680052a0 con 0x7f32701020c0 2024-05-10T03:59:16.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.880+0000 7f32750d4700 1 -- 172.21.15.150:0/2767347098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3270199280 con 0x7f32701020c0 2024-05-10T03:59:16.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.881+0000 7f32750d4700 1 -- 172.21.15.150:0/2767347098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32700ff650 con 0x7f32701020c0 2024-05-10T03:59:16.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.882+0000 7f3257fff700 1 -- 172.21.15.150:0/2767347098 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7f3268005400 con 0x7f32701020c0 2024-05-10T03:59:16.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.883+0000 7f3257fff700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f32580a1090 0x7f32580a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:59:16.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.883+0000 7f326ed9d700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f32580a1090 0x7f32580a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:59:16.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.883+0000 7f3257fff700 1 -- 172.21.15.150:0/2767347098 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2571+0+0 (secure 0 0 0) 0x7f32680cc6a0 con 0x7f32701020c0 2024-05-10T03:59:16.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.883+0000 7f326ed9d700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f32580a1090 0x7f32580a35a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3270069af0 tx=0x7f326000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:59:16.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:16.887+0000 7f3257fff700 1 -- 172.21.15.150:0/2767347098 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3268097570 con 0x7f32701020c0 2024-05-10T03:59:17.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:17.091+0000 7f32750d4700 1 -- 172.21.15.150:0/2767347098 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f32700ffcc0 con 0x7f32580a1090 2024-05-10T03:59:17.239 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 03:59:16 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-2[52428]: 2024-05-10T03:59:16.870+0000 7fedc66e4700 -1 osd.2 0 waiting for initial osdmap 2024-05-10T03:59:17.239 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 03:59:16 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-2[52428]: 2024-05-10T03:59:16.876+0000 7fedbdc92700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T03:59:17.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:17 smithi150 ceph-mon[42477]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T03:59:17.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:17 smithi150 ceph-mon[42477]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T03:59:17.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:17 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:17.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:17 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:17.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:17 smithi150 ceph-mon[42477]: from='osd.2 ' entity='osd.2' 2024-05-10T03:59:17.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:17 smithi150 ceph-mon[42477]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:17.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:17 smithi150 ceph-mon[42477]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:59:17.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:17 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:59:17.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:17 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:59:17.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:17 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:17.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:17 smithi181 ceph-mon[46602]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T03:59:17.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:17 smithi181 ceph-mon[46602]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T03:59:17.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:17 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:17.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:17 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:17.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:17 smithi181 ceph-mon[46602]: from='osd.2 ' entity='osd.2' 2024-05-10T03:59:17.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:17 smithi181 ceph-mon[46602]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:17.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:17 smithi181 ceph-mon[46602]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:59:17.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:17 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:59:17.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:17 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:59:17.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:17 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:17.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:17 smithi161 ceph-mon[46918]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T03:59:17.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:17 smithi161 ceph-mon[46918]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T03:59:17.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:17 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:17.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:17 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:17.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:17 smithi161 ceph-mon[46918]: from='osd.2 ' entity='osd.2' 2024-05-10T03:59:17.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:17 smithi161 ceph-mon[46918]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T03:59:17.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:17 smithi161 ceph-mon[46918]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:59:17.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:17 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:59:17.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:17 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:59:17.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:17 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:18.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:18 smithi150 ceph-mon[42477]: purged_snaps scrub starts 2024-05-10T03:59:18.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:18 smithi150 ceph-mon[42477]: purged_snaps scrub ok 2024-05-10T03:59:18.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:18 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:18.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:18 smithi150 ceph-mon[42477]: osd.2 [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145] boot 2024-05-10T03:59:18.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:18 smithi150 ceph-mon[42477]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T03:59:18.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:18 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:18.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:18 smithi181 ceph-mon[46602]: purged_snaps scrub starts 2024-05-10T03:59:18.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:18 smithi181 ceph-mon[46602]: purged_snaps scrub ok 2024-05-10T03:59:18.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:18 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:18.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:18 smithi181 ceph-mon[46602]: osd.2 [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145] boot 2024-05-10T03:59:18.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:18 smithi181 ceph-mon[46602]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T03:59:18.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:18 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:18.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:18 smithi161 ceph-mon[46918]: purged_snaps scrub starts 2024-05-10T03:59:18.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:18 smithi161 ceph-mon[46918]: purged_snaps scrub ok 2024-05-10T03:59:18.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:18 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:18.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:18 smithi161 ceph-mon[46918]: osd.2 [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145] boot 2024-05-10T03:59:18.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:18 smithi161 ceph-mon[46918]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T03:59:18.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:18 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T03:59:19.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:19 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4091228650' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "116fe668-1fb0-478b-a7ea-fd1542885ec9"}]: dispatch 2024-05-10T03:59:19.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:19 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4091228650' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "116fe668-1fb0-478b-a7ea-fd1542885ec9"}]': finished 2024-05-10T03:59:19.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:19 smithi150 ceph-mon[42477]: osdmap e19: 4 total, 3 up, 4 in 2024-05-10T03:59:19.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:19.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:19 smithi150 ceph-mon[42477]: pgmap v64: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:19.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T03:59:19.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:19 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4091228650' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "116fe668-1fb0-478b-a7ea-fd1542885ec9"}]: dispatch 2024-05-10T03:59:19.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:19 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4091228650' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "116fe668-1fb0-478b-a7ea-fd1542885ec9"}]': finished 2024-05-10T03:59:19.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:19 smithi181 ceph-mon[46602]: osdmap e19: 4 total, 3 up, 4 in 2024-05-10T03:59:19.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:19.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:19 smithi181 ceph-mon[46602]: pgmap v64: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:19.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T03:59:19.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:19 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4091228650' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "116fe668-1fb0-478b-a7ea-fd1542885ec9"}]: dispatch 2024-05-10T03:59:19.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:19 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4091228650' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "116fe668-1fb0-478b-a7ea-fd1542885ec9"}]': finished 2024-05-10T03:59:19.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:19 smithi161 ceph-mon[46918]: osdmap e19: 4 total, 3 up, 4 in 2024-05-10T03:59:19.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:19.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:19 smithi161 ceph-mon[46918]: pgmap v64: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:19.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T03:59:20.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:20 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1545349780' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:59:20.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T03:59:20.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:20 smithi150 ceph-mon[42477]: osdmap e20: 4 total, 3 up, 4 in 2024-05-10T03:59:20.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:20.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T03:59:20.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:20 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1545349780' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:59:20.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T03:59:20.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:20 smithi181 ceph-mon[46602]: osdmap e20: 4 total, 3 up, 4 in 2024-05-10T03:59:20.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:20.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T03:59:20.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:20 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1545349780' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:59:20.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T03:59:20.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:20 smithi161 ceph-mon[46918]: osdmap e20: 4 total, 3 up, 4 in 2024-05-10T03:59:20.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:20.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T03:59:21.787 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 03:59:21 smithi150 sudo[67428]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T03:59:21.787 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 03:59:21 smithi150 sudo[67428]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T03:59:21.787 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 03:59:21 smithi150 sudo[67428]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T03:59:21.787 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 03:59:21 smithi150 sudo[67428]: pam_unix(sudo:session): session closed for user root 2024-05-10T03:59:21.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:21.873+0000 7f3257fff700 1 -- 172.21.15.150:0/2767347098 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3268097940 con 0x7f32701020c0 2024-05-10T03:59:22.214 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 03:59:21 smithi150 sudo[67615]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T03:59:22.214 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 03:59:21 smithi150 sudo[67615]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T03:59:22.214 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 03:59:21 smithi150 sudo[67615]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T03:59:22.214 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 03:59:21 smithi150 sudo[67615]: pam_unix(sudo:session): session closed for user root 2024-05-10T03:59:22.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T03:59:22.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:21 smithi150 ceph-mon[42477]: osdmap e21: 4 total, 3 up, 4 in 2024-05-10T03:59:22.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:22.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:21 smithi150 ceph-mon[42477]: pgmap v67: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:22.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T03:59:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:21 smithi181 ceph-mon[46602]: osdmap e21: 4 total, 3 up, 4 in 2024-05-10T03:59:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:21 smithi181 ceph-mon[46602]: pgmap v67: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:22.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:22.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T03:59:22.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:21 smithi161 ceph-mon[46918]: osdmap e21: 4 total, 3 up, 4 in 2024-05-10T03:59:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:21 smithi161 ceph-mon[46918]: pgmap v67: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:22.243 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 03:59:22 smithi161 sudo[56700]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T03:59:22.243 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 03:59:22 smithi161 sudo[56700]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T03:59:22.243 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 03:59:22 smithi161 sudo[56700]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T03:59:22.244 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 03:59:22 smithi161 sudo[56700]: pam_unix(sudo:session): session closed for user root 2024-05-10T03:59:22.742 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 03:59:22 smithi161 sudo[56703]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T03:59:22.743 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 03:59:22 smithi161 sudo[56703]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T03:59:22.743 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 03:59:22 smithi161 sudo[56703]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T03:59:22.743 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 03:59:22 smithi161 sudo[56703]: pam_unix(sudo:session): session closed for user root 2024-05-10T03:59:22.988 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 03:59:22 smithi181 sudo[57254]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T03:59:22.988 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 03:59:22 smithi181 sudo[57254]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T03:59:22.988 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 03:59:22 smithi181 sudo[57254]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T03:59:22.988 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 03:59:22 smithi181 sudo[57254]: pam_unix(sudo:session): session closed for user root 2024-05-10T03:59:22.989 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 03:59:22 smithi181 sudo[57257]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T03:59:22.989 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 03:59:22 smithi181 sudo[57257]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T03:59:22.989 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 03:59:22 smithi181 sudo[57257]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T03:59:22.989 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 03:59:22 smithi181 sudo[57257]: pam_unix(sudo:session): session closed for user root 2024-05-10T03:59:23.350 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 sudo[68008]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T03:59:23.350 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 sudo[68008]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T03:59:23.350 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 sudo[68008]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T03:59:23.350 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 sudo[68008]: pam_unix(sudo:session): session closed for user root 2024-05-10T03:59:23.627 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 ceph-mon[46602]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T03:59:23.627 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 ceph-mon[46602]: mgrmap e16: x(active, since 112s), standbys: z, y 2024-05-10T03:59:23.627 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:23.627 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:23.627 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:23.627 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 ceph-mon[46602]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T03:59:23.627 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 sudo[57260]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T03:59:23.628 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 sudo[57260]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T03:59:23.628 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 sudo[57260]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T03:59:23.628 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 sudo[57260]: pam_unix(sudo:session): session closed for user root 2024-05-10T03:59:23.664 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 sudo[68084]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T03:59:23.664 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 sudo[68084]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T03:59:23.664 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 sudo[68084]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T03:59:23.664 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 sudo[68084]: pam_unix(sudo:session): session closed for user root 2024-05-10T03:59:23.665 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 ceph-mon[42477]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T03:59:23.665 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 ceph-mon[42477]: mgrmap e16: x(active, since 112s), standbys: z, y 2024-05-10T03:59:23.665 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:23.665 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:23.665 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:23.665 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:23 smithi150 ceph-mon[42477]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T03:59:23.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:23 smithi161 ceph-mon[46918]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T03:59:23.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:23 smithi161 ceph-mon[46918]: mgrmap e16: x(active, since 112s), standbys: z, y 2024-05-10T03:59:23.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:23.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:23.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:23.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:23 smithi161 ceph-mon[46918]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T03:59:23.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 sudo[57263]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T03:59:23.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 sudo[57263]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T03:59:23.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 sudo[57263]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T03:59:23.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:23 smithi181 sudo[57263]: pam_unix(sudo:session): session closed for user root 2024-05-10T03:59:24.380 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 sudo[56714]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T03:59:24.380 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 sudo[56714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T03:59:24.380 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 sudo[56714]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T03:59:24.380 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 sudo[56714]: pam_unix(sudo:session): session closed for user root 2024-05-10T03:59:24.380 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 sudo[56717]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T03:59:24.380 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 sudo[56717]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T03:59:24.380 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 sudo[56717]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T03:59:24.380 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 sudo[56717]: pam_unix(sudo:session): session closed for user root 2024-05-10T03:59:24.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:24.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T03:59:24.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:59:24.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:59:24.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:59:24.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T03:59:24.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:59:24.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:59:24.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:59:24.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T03:59:24.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T03:59:24.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:59:24.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:59:24.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:59:24.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:24 smithi150 ceph-mon[42477]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T03:59:24.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:24.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T03:59:24.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:59:24.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:59:24.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:59:24.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T03:59:24.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:59:24.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:59:24.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:59:24.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T03:59:24.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T03:59:24.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:59:24.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:59:24.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:59:24.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:24 smithi181 ceph-mon[46602]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T03:59:24.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:24.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T03:59:24.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:59:24.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:59:24.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:59:24.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T03:59:24.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:59:24.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:59:24.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:59:24.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T03:59:24.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T03:59:24.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T03:59:24.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T03:59:24.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T03:59:24.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:24 smithi161 ceph-mon[46918]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T03:59:25.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T03:59:25.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:25.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:25 smithi150 ceph-mon[42477]: Deploying daemon osd.3 on smithi150 2024-05-10T03:59:25.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:25 smithi150 ceph-mon[42477]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T03:59:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:25 smithi181 ceph-mon[46602]: Deploying daemon osd.3 on smithi150 2024-05-10T03:59:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:25 smithi181 ceph-mon[46602]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:25.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T03:59:25.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:25.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:25 smithi161 ceph-mon[46918]: Deploying daemon osd.3 on smithi150 2024-05-10T03:59:25.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:25 smithi161 ceph-mon[46918]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:28.377 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:28 smithi150 ceph-mon[42477]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:28 smithi181 ceph-mon[46602]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:28.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:28 smithi161 ceph-mon[46918]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:29.622 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:29.622 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:29.622 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:29.622 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:29 smithi150 ceph-mon[42477]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:29.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:29.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:29.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:29.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:29 smithi181 ceph-mon[46602]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:29.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:29.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:29.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:29.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:29 smithi161 ceph-mon[46918]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 3 on host 'smithi150' 2024-05-10T03:59:30.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:30.934+0000 7f3257fff700 1 -- 172.21.15.150:0/2767347098 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f32700ffcc0 con 0x7f32580a1090 2024-05-10T03:59:30.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:30.937+0000 7f32750d4700 1 -- 172.21.15.150:0/2767347098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f32580a1090 msgr2=0x7f32580a35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:30.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:30.937+0000 7f32750d4700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f32580a1090 0x7f32580a35a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3270069af0 tx=0x7f326000a380 comp rx=0 tx=0).stop 2024-05-10T03:59:30.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:30.937+0000 7f32750d4700 1 -- 172.21.15.150:0/2767347098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32701020c0 msgr2=0x7f3270198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:30.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:30.937+0000 7f32750d4700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32701020c0 0x7f3270198400 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3268006130 tx=0x7f3268007920 comp rx=0 tx=0).stop 2024-05-10T03:59:30.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:30.937+0000 7f32750d4700 1 -- 172.21.15.150:0/2767347098 shutdown_connections 2024-05-10T03:59:30.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:30.937+0000 7f32750d4700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f32580a1090 0x7f32580a35a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:30.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:30.937+0000 7f32750d4700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32701020c0 0x7f3270198400 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:30.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:30.937+0000 7f32750d4700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3270100dd0 0x7f3270197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:30.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:30.937+0000 7f32750d4700 1 --2- 172.21.15.150:0/2767347098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3270101720 0x7f3270197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:30.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:30.937+0000 7f32750d4700 1 -- 172.21.15.150:0/2767347098 >> 172.21.15.150:0/2767347098 conn(0x7f32700fb990 msgr2=0x7f3270105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:59:30.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:30.937+0000 7f32750d4700 1 -- 172.21.15.150:0/2767347098 shutdown_connections 2024-05-10T03:59:30.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T03:59:30.937+0000 7f32750d4700 1 -- 172.21.15.150:0/2767347098 wait complete. 2024-05-10T03:59:31.096 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:31.096 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:31.096 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:31.096 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:31.096 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:31.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:31.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:31.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:31.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:31.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:31.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:31.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:31.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:31.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:31.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:31.311 DEBUG:teuthology.orchestra.run.smithi150:osd.3> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.3.service 2024-05-10T03:59:31.313 INFO:tasks.cephadm:Deploying osd.4 on smithi161 with /dev/vg_nvme/lv_3... 2024-05-10T03:59:31.314 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T03:59:31.343 INFO:journalctl@ceph.osd.3.smithi150.stdout:-- Logs begin at Fri 2024-05-10 03:40:30 UTC. -- 2024-05-10T03:59:31.550 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T03:59:31.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:31 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:31.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:31 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:31.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:31 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:31.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:31 smithi150 ceph-mon[42477]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:32.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:31 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:32.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:31 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:32.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:31 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:32.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:31 smithi181 ceph-mon[46602]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:32.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:31 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:32.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:31 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:32.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:31 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:32.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:31 smithi161 ceph-mon[46918]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:32.822 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T03:59:32.822 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T03:59:32.822 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-05-10T03:59:32.822 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-05-10T03:59:32.822 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115414 s, 909 MB/s 2024-05-10T03:59:32.822 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-05-10T03:59:32.928 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 03:59:32 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-3[68825]: 2024-05-10T03:59:32.481+0000 7f3930f427c0 -1 osd.3 0 log_to_monitors true 2024-05-10T03:59:33.206 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:32 smithi161 ceph-mon[46918]: from='osd.3 [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T03:59:33.206 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:32 smithi161 ceph-mon[46918]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T03:59:33.208 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-05-10T03:59:33.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:32 smithi150 ceph-mon[42477]: from='osd.3 [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T03:59:33.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:32 smithi150 ceph-mon[42477]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T03:59:33.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:32 smithi181 ceph-mon[46602]: from='osd.3 [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T03:59:33.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:32 smithi181 ceph-mon[46602]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T03:59:33.429 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T03:59:34.215 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 03:59:33 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-3[68825]: 2024-05-10T03:59:33.940+0000 7f39265a6700 -1 osd.3 0 waiting for initial osdmap 2024-05-10T03:59:34.215 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 03:59:33 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-3[68825]: 2024-05-10T03:59:33.947+0000 7f392235d700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T03:59:34.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T03:59:34.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T03:59:34.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:34.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: from='osd.3 [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T03:59:34.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T03:59:34.216 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:34.216 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:34.216 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:34.216 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:34.216 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:34.216 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:34.216 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:34.216 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:34.216 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:34.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T03:59:34.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T03:59:34.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:34.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: from='osd.3 [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T03:59:34.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T03:59:34.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:34.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:34.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:34.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:34.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:34.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:34.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:34.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:34.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:34.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T03:59:34.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T03:59:34.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:34.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: from='osd.3 [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T03:59:34.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T03:59:34.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:34.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:34.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:34.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:34.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:34.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:34.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:34.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:34.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:34.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.456+0000 7f82e44c1700 1 -- 172.21.15.161:0/4256018773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc102e80 msgr2=0x7f82dc103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:34.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.456+0000 7f82e44c1700 1 --2- 172.21.15.161:0/4256018773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc102e80 0x7f82dc103290 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009990 tx=0x7f82d0009ca0 comp rx=0 tx=0).stop 2024-05-10T03:59:34.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 -- 172.21.15.161:0/4256018773 shutdown_connections 2024-05-10T03:59:34.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 --2- 172.21.15.161:0/4256018773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82dc103820 0x7f82dc10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 --2- 172.21.15.161:0/4256018773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82dc101d50 0x7f82dc102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 --2- 172.21.15.161:0/4256018773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc102e80 0x7f82dc103290 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 -- 172.21.15.161:0/4256018773 >> 172.21.15.161:0/4256018773 conn(0x7f82dc0fd390 msgr2=0x7f82dc0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:59:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 -- 172.21.15.161:0/4256018773 shutdown_connections 2024-05-10T03:59:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 -- 172.21.15.161:0/4256018773 wait complete. 2024-05-10T03:59:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 Processor -- start 2024-05-10T03:59:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 -- start start 2024-05-10T03:59:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82dc101d50 0x7f82dc198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:59:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82dc102e80 0x7f82dc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:59:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc103820 0x7f82dc199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:59:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc199ae0 con 0x7f82dc102e80 2024-05-10T03:59:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc199c20 con 0x7f82dc103820 2024-05-10T03:59:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.457+0000 7f82e44c1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc199de0 con 0x7f82dc101d50 2024-05-10T03:59:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e2a5e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc103820 0x7f82dc199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:59:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e2a5e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc103820 0x7f82dc199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48050/0 (socket says 172.21.15.161:48050) 2024-05-10T03:59:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e2a5e700 1 -- 172.21.15.161:0/2200567927 learned_addr learned my addr 172.21.15.161:0/2200567927 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-10T03:59:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e1a5c700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82dc102e80 0x7f82dc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:59:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e225d700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82dc101d50 0x7f82dc198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:59:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e2a5e700 1 -- 172.21.15.161:0/2200567927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82dc101d50 msgr2=0x7f82dc198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e2a5e700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82dc101d50 0x7f82dc198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e2a5e700 1 -- 172.21.15.161:0/2200567927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82dc102e80 msgr2=0x7f82dc1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e2a5e700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82dc102e80 0x7f82dc1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e2a5e700 1 -- 172.21.15.161:0/2200567927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d00095f0 con 0x7f82dc103820 2024-05-10T03:59:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e225d700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82dc101d50 0x7f82dc198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:59:34.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e2a5e700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc103820 0x7f82dc199480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f82d4007ee0 tx=0x7f82d400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:59:34.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82cf7fe700 1 -- 172.21.15.161:0/2200567927 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4017070 con 0x7f82dc103820 2024-05-10T03:59:34.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e44c1700 1 -- 172.21.15.161:0/2200567927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc19a0c0 con 0x7f82dc103820 2024-05-10T03:59:34.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82cf7fe700 1 -- 172.21.15.161:0/2200567927 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82d400f040 con 0x7f82dc103820 2024-05-10T03:59:34.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82cf7fe700 1 -- 172.21.15.161:0/2200567927 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4005740 con 0x7f82dc103820 2024-05-10T03:59:34.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.458+0000 7f82e44c1700 1 -- 172.21.15.161:0/2200567927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc1a08a0 con 0x7f82dc103820 2024-05-10T03:59:34.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.459+0000 7f82e44c1700 1 -- 172.21.15.161:0/2200567927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc108fe0 con 0x7f82dc103820 2024-05-10T03:59:34.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.461+0000 7f82cf7fe700 1 -- 172.21.15.161:0/2200567927 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f82d400dc60 con 0x7f82dc103820 2024-05-10T03:59:34.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.461+0000 7f82cf7fe700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82c80a1120 0x7f82c80a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:59:34.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.461+0000 7f82cf7fe700 1 -- 172.21.15.161:0/2200567927 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3389+0+0 (secure 0 0 0) 0x7f82d40cb3e0 con 0x7f82dc103820 2024-05-10T03:59:34.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.461+0000 7f82e225d700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82c80a1120 0x7f82c80a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:59:34.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.462+0000 7f82e225d700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82c80a1120 0x7f82c80a3630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f82d8005950 tx=0x7f82d80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:59:34.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.465+0000 7f82cf7fe700 1 -- 172.21.15.161:0/2200567927 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f82d4095eb0 con 0x7f82dc103820 2024-05-10T03:59:34.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:34.664+0000 7f82e44c1700 1 -- 172.21.15.161:0/2200567927 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f82dc10d680 con 0x7f82c80a1120 2024-05-10T03:59:35.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:34 smithi150 ceph-mon[42477]: Detected new or changed devices on smithi150 2024-05-10T03:59:35.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:34 smithi150 ceph-mon[42477]: Adjusting osd_memory_target on smithi150 to 8557M 2024-05-10T03:59:35.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:34 smithi150 ceph-mon[42477]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T03:59:35.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:34 smithi150 ceph-mon[42477]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T03:59:35.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:34 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:35.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:34 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:35.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:34 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:59:35.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:34 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:59:35.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:34 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:35.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:34 smithi181 ceph-mon[46602]: Detected new or changed devices on smithi150 2024-05-10T03:59:35.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:34 smithi181 ceph-mon[46602]: Adjusting osd_memory_target on smithi150 to 8557M 2024-05-10T03:59:35.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:34 smithi181 ceph-mon[46602]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T03:59:35.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:34 smithi181 ceph-mon[46602]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T03:59:35.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:34 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:35.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:34 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:35.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:34 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:59:35.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:34 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:59:35.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:34 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:35.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:34 smithi161 ceph-mon[46918]: Detected new or changed devices on smithi150 2024-05-10T03:59:35.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:34 smithi161 ceph-mon[46918]: Adjusting osd_memory_target on smithi150 to 8557M 2024-05-10T03:59:35.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:34 smithi161 ceph-mon[46918]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T03:59:35.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:34 smithi161 ceph-mon[46918]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T03:59:35.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:34 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:35.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:34 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:35.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:34 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:59:35.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:34 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:59:35.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:34 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:35.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:35 smithi161 ceph-mon[46918]: purged_snaps scrub starts 2024-05-10T03:59:35.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:35 smithi161 ceph-mon[46918]: purged_snaps scrub ok 2024-05-10T03:59:35.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:35 smithi161 ceph-mon[46918]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:59:35.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:35 smithi161 ceph-mon[46918]: osd.3 [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004] boot 2024-05-10T03:59:35.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:35 smithi161 ceph-mon[46918]: osdmap e25: 4 total, 4 up, 4 in 2024-05-10T03:59:35.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:35 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:35.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:35 smithi161 ceph-mon[46918]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:36.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:35 smithi150 ceph-mon[42477]: purged_snaps scrub starts 2024-05-10T03:59:36.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:35 smithi150 ceph-mon[42477]: purged_snaps scrub ok 2024-05-10T03:59:36.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:35 smithi150 ceph-mon[42477]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:59:36.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:35 smithi150 ceph-mon[42477]: osd.3 [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004] boot 2024-05-10T03:59:36.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:35 smithi150 ceph-mon[42477]: osdmap e25: 4 total, 4 up, 4 in 2024-05-10T03:59:36.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:35 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:36.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:35 smithi150 ceph-mon[42477]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:36.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:35 smithi181 ceph-mon[46602]: purged_snaps scrub starts 2024-05-10T03:59:36.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:35 smithi181 ceph-mon[46602]: purged_snaps scrub ok 2024-05-10T03:59:36.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:35 smithi181 ceph-mon[46602]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:59:36.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:35 smithi181 ceph-mon[46602]: osd.3 [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004] boot 2024-05-10T03:59:36.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:35 smithi181 ceph-mon[46602]: osdmap e25: 4 total, 4 up, 4 in 2024-05-10T03:59:36.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:35 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T03:59:36.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:35 smithi181 ceph-mon[46602]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T03:59:37.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:36 smithi150 ceph-mon[42477]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T03:59:37.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:36 smithi150 ceph-mon[42477]: from='client.? 172.21.15.161:0/1269584304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "827ce60d-b43f-45e0-a1e3-d40de3a1b5e3"}]: dispatch 2024-05-10T03:59:37.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:36 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "827ce60d-b43f-45e0-a1e3-d40de3a1b5e3"}]: dispatch 2024-05-10T03:59:37.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:36 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "827ce60d-b43f-45e0-a1e3-d40de3a1b5e3"}]': finished 2024-05-10T03:59:37.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:36 smithi150 ceph-mon[42477]: osdmap e27: 5 total, 4 up, 5 in 2024-05-10T03:59:37.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:36 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:37.241 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:36 smithi181 ceph-mon[46602]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T03:59:37.241 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:36 smithi181 ceph-mon[46602]: from='client.? 172.21.15.161:0/1269584304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "827ce60d-b43f-45e0-a1e3-d40de3a1b5e3"}]: dispatch 2024-05-10T03:59:37.241 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:36 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "827ce60d-b43f-45e0-a1e3-d40de3a1b5e3"}]: dispatch 2024-05-10T03:59:37.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:36 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "827ce60d-b43f-45e0-a1e3-d40de3a1b5e3"}]': finished 2024-05-10T03:59:37.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:36 smithi181 ceph-mon[46602]: osdmap e27: 5 total, 4 up, 5 in 2024-05-10T03:59:37.242 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:36 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:37.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:36 smithi161 ceph-mon[46918]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T03:59:37.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:36 smithi161 ceph-mon[46918]: from='client.? 172.21.15.161:0/1269584304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "827ce60d-b43f-45e0-a1e3-d40de3a1b5e3"}]: dispatch 2024-05-10T03:59:37.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:36 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "827ce60d-b43f-45e0-a1e3-d40de3a1b5e3"}]: dispatch 2024-05-10T03:59:37.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:36 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "827ce60d-b43f-45e0-a1e3-d40de3a1b5e3"}]': finished 2024-05-10T03:59:37.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:36 smithi161 ceph-mon[46918]: osdmap e27: 5 total, 4 up, 5 in 2024-05-10T03:59:37.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:36 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:38.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:37 smithi150 ceph-mon[42477]: pgmap v81: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-10T03:59:38.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:37 smithi150 ceph-mon[42477]: from='client.? 172.21.15.161:0/3573002402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:59:38.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:37 smithi150 ceph-mon[42477]: osdmap e28: 5 total, 4 up, 5 in 2024-05-10T03:59:38.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:38.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:37 smithi181 ceph-mon[46602]: pgmap v81: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-10T03:59:38.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:37 smithi181 ceph-mon[46602]: from='client.? 172.21.15.161:0/3573002402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:59:38.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:37 smithi181 ceph-mon[46602]: osdmap e28: 5 total, 4 up, 5 in 2024-05-10T03:59:38.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:38.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:37 smithi161 ceph-mon[46918]: pgmap v81: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-10T03:59:38.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:37 smithi161 ceph-mon[46918]: from='client.? 172.21.15.161:0/3573002402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:59:38.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:37 smithi161 ceph-mon[46918]: osdmap e28: 5 total, 4 up, 5 in 2024-05-10T03:59:38.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:39.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:39 smithi150 ceph-mon[42477]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T03:59:39.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:39 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:39.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:39 smithi150 ceph-mon[42477]: pgmap v84: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-10T03:59:39.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:39 smithi181 ceph-mon[46602]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T03:59:39.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:39 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:39.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:39 smithi181 ceph-mon[46602]: pgmap v84: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-10T03:59:39.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:39 smithi161 ceph-mon[46918]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T03:59:39.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:39 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:39.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:39 smithi161 ceph-mon[46918]: pgmap v84: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-10T03:59:42.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:42 smithi161 ceph-mon[46918]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-05-10T03:59:42.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:42 smithi150 ceph-mon[42477]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-05-10T03:59:42.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:42 smithi181 ceph-mon[46602]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-05-10T03:59:43.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:43 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T03:59:43.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:43 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:43.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:43 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T03:59:43.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:43 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:43.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:43 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T03:59:43.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:43 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:44.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:44 smithi161 ceph-mon[46918]: Deploying daemon osd.4 on smithi161 2024-05-10T03:59:44.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:44 smithi161 ceph-mon[46918]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-10T03:59:44.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:44 smithi150 ceph-mon[42477]: Deploying daemon osd.4 on smithi161 2024-05-10T03:59:44.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:44 smithi150 ceph-mon[42477]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-10T03:59:44.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:44 smithi181 ceph-mon[46602]: Deploying daemon osd.4 on smithi161 2024-05-10T03:59:44.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:44 smithi181 ceph-mon[46602]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-10T03:59:46.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:46 smithi150 ceph-mon[42477]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T03:59:46.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:46 smithi181 ceph-mon[46602]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T03:59:46.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:46 smithi161 ceph-mon[46918]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T03:59:47.454 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:47 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:47.454 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:47 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:47.454 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:47 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:47.454 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:47 smithi161 ceph-mon[46918]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-10T03:59:47.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:47 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:47.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:47 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:47.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:47 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:47.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:47 smithi150 ceph-mon[42477]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-10T03:59:47.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:47 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:47.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:47 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:47.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:47 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:47.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:47 smithi181 ceph-mon[46602]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-10T03:59:48.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:48.469+0000 7f82cf7fe700 1 -- 172.21.15.161:0/2200567927 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f82dc10d680 con 0x7f82c80a1120 2024-05-10T03:59:48.472 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 4 on host 'smithi161' 2024-05-10T03:59:48.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:48.471+0000 7f82e44c1700 1 -- 172.21.15.161:0/2200567927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82c80a1120 msgr2=0x7f82c80a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:48.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:48.471+0000 7f82e44c1700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82c80a1120 0x7f82c80a3630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f82d8005950 tx=0x7f82d80058e0 comp rx=0 tx=0).stop 2024-05-10T03:59:48.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:48.471+0000 7f82e44c1700 1 -- 172.21.15.161:0/2200567927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc103820 msgr2=0x7f82dc199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:48.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:48.471+0000 7f82e44c1700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc103820 0x7f82dc199480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f82d4007ee0 tx=0x7f82d400d3b0 comp rx=0 tx=0).stop 2024-05-10T03:59:48.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:48.471+0000 7f82e44c1700 1 -- 172.21.15.161:0/2200567927 shutdown_connections 2024-05-10T03:59:48.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:48.471+0000 7f82e44c1700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82c80a1120 0x7f82c80a3630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:48.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:48.471+0000 7f82e44c1700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc103820 0x7f82dc199480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:48.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:48.471+0000 7f82e44c1700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82dc101d50 0x7f82dc198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:48.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:48.471+0000 7f82e44c1700 1 --2- 172.21.15.161:0/2200567927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82dc102e80 0x7f82dc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:48.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:48.471+0000 7f82e44c1700 1 -- 172.21.15.161:0/2200567927 >> 172.21.15.161:0/2200567927 conn(0x7f82dc0fd390 msgr2=0x7f82dc106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:59:48.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:48.471+0000 7f82e44c1700 1 -- 172.21.15.161:0/2200567927 shutdown_connections 2024-05-10T03:59:48.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T03:59:48.471+0000 7f82e44c1700 1 -- 172.21.15.161:0/2200567927 wait complete. 2024-05-10T03:59:48.867 DEBUG:teuthology.orchestra.run.smithi161:osd.4> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.4.service 2024-05-10T03:59:48.869 INFO:tasks.cephadm:Deploying osd.5 on smithi181 with /dev/vg_nvme/lv_3... 2024-05-10T03:59:48.869 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T03:59:48.898 INFO:journalctl@ceph.osd.4.smithi161.stdout:-- Logs begin at Fri 2024-05-10 03:40:24 UTC. -- 2024-05-10T03:59:48.942 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:48 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:48.943 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:48 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:48.943 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:48 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:48.943 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:48 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:48.943 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:48 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:48.943 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:48 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:48.943 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:48 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:48.943 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:48 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:49.104 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T03:59:49.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:49.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:49.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:49.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:49.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:49.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:49.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:49.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:49.229 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:48 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:49.230 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:48 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:49.230 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:48 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:49.230 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:48 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:49.230 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:48 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:49.230 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:48 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:49.230 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:48 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T03:59:49.230 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:48 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:49.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:49 smithi161 ceph-mon[46918]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-10T03:59:50.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:49 smithi150 ceph-mon[42477]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-10T03:59:50.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:49 smithi181 ceph-mon[46602]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-10T03:59:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T03:59:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T03:59:50.422 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-05-10T03:59:50.422 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-05-10T03:59:50.422 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111568 s, 940 MB/s 2024-05-10T03:59:50.422 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-05-10T03:59:50.559 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 03:59:50 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-4[59786]: 2024-05-10T03:59:50.259+0000 7f4107b457c0 -1 osd.4 0 log_to_monitors true 2024-05-10T03:59:50.758 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_3 2024-05-10T03:59:50.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:50 smithi161 ceph-mon[46918]: from='osd.4 [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T03:59:50.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:50 smithi161 ceph-mon[46918]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T03:59:50.982 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T03:59:51.095 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:50 smithi181 ceph-mon[46602]: from='osd.4 [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T03:59:51.096 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:50 smithi181 ceph-mon[46602]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T03:59:51.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:50 smithi150 ceph-mon[42477]: from='osd.4 [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T03:59:51.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:50 smithi150 ceph-mon[42477]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T03:59:51.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.948+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/4077774403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd01009e0 msgr2=0x7f5dd0100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:51.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.948+0000 7f5dd4d9f700 1 --2- 172.21.15.181:0/4077774403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd01009e0 0x7f5dd0100df0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009990 tx=0x7f5dc0009ca0 comp rx=0 tx=0).stop 2024-05-10T03:59:51.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.948+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/4077774403 shutdown_connections 2024-05-10T03:59:51.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.948+0000 7f5dd4d9f700 1 --2- 172.21.15.181:0/4077774403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dd01024b0 0x7f5dd0108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:51.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.948+0000 7f5dd4d9f700 1 --2- 172.21.15.181:0/4077774403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd01009e0 0x7f5dd0100df0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:51.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.948+0000 7f5dd4d9f700 1 --2- 172.21.15.181:0/4077774403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dd0101b10 0x7f5dd0101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:51.952 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.948+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/4077774403 >> 172.21.15.181:0/4077774403 conn(0x7f5dd00fc020 msgr2=0x7f5dd00fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:59:51.952 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.949+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/4077774403 shutdown_connections 2024-05-10T03:59:51.952 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.949+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/4077774403 wait complete. 2024-05-10T03:59:51.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.949+0000 7f5dd4d9f700 1 Processor -- start 2024-05-10T03:59:51.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.949+0000 7f5dd4d9f700 1 -- start start 2024-05-10T03:59:51.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dd4d9f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dd01009e0 0x7f5dd0069780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:59:51.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dd4d9f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd0101b10 0x7f5dd019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:59:51.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dd4d9f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dd01024b0 0x7f5dd019f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:59:51.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dd4d9f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd019f950 con 0x7f5dd01024b0 2024-05-10T03:59:51.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dd4d9f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd019fac0 con 0x7f5dd01009e0 2024-05-10T03:59:51.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dd4d9f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd019fc80 con 0x7f5dd0101b10 2024-05-10T03:59:51.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dceffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd0101b10 0x7f5dd019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:59:51.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dceffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd0101b10 0x7f5dd019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40786/0 (socket says 172.21.15.181:40786) 2024-05-10T03:59:51.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dceffd700 1 -- 172.21.15.181:0/1229717200 learned_addr learned my addr 172.21.15.181:0/1229717200 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T03:59:51.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dcf7fe700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dd01009e0 0x7f5dd0069780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:59:51.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dcffff700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dd01024b0 0x7f5dd019f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:59:51.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dceffd700 1 -- 172.21.15.181:0/1229717200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dd01009e0 msgr2=0x7f5dd0069780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:51.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dceffd700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dd01009e0 0x7f5dd0069780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:51.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dceffd700 1 -- 172.21.15.181:0/1229717200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dd01024b0 msgr2=0x7f5dd019f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:59:51.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dceffd700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dd01024b0 0x7f5dd019f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:59:51.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dceffd700 1 -- 172.21.15.181:0/1229717200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc00095f0 con 0x7f5dd0101b10 2024-05-10T03:59:51.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dcffff700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dd01024b0 0x7f5dd019f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:59:51.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dceffd700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd0101b10 0x7f5dd019cce0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5db800d950 tx=0x7f5db800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:59:51.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.950+0000 7f5dccff9700 1 -- 172.21.15.181:0/1229717200 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8009940 con 0x7f5dd0101b10 2024-05-10T03:59:51.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.951+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/1229717200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd019ff60 con 0x7f5dd0101b10 2024-05-10T03:59:51.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.951+0000 7f5dccff9700 1 -- 172.21.15.181:0/1229717200 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db8010460 con 0x7f5dd0101b10 2024-05-10T03:59:51.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.951+0000 7f5dccff9700 1 -- 172.21.15.181:0/1229717200 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db800f620 con 0x7f5dd0101b10 2024-05-10T03:59:51.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.951+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/1229717200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd01a0380 con 0x7f5dd0101b10 2024-05-10T03:59:51.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.952+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/1229717200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd0107c00 con 0x7f5dd0101b10 2024-05-10T03:59:51.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.953+0000 7f5dccff9700 1 -- 172.21.15.181:0/1229717200 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5db800f780 con 0x7f5dd0101b10 2024-05-10T03:59:51.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.953+0000 7f5dccff9700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5dbc0a1120 0x7f5dbc0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:59:51.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.953+0000 7f5dccff9700 1 -- 172.21.15.181:0/1229717200 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3821+0+0 (secure 0 0 0) 0x7f5db80cb5b0 con 0x7f5dd0101b10 2024-05-10T03:59:51.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.953+0000 7f5dcf7fe700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5dbc0a1120 0x7f5dbc0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:59:51.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.954+0000 7f5dcf7fe700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5dbc0a1120 0x7f5dbc0a3630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000b5c0 tx=0x7f5dc001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:59:51.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:51.958+0000 7f5dccff9700 1 -- 172.21.15.181:0/1229717200 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5db8095ed0 con 0x7f5dd0101b10 2024-05-10T03:59:52.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T03:59:52.159+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/1229717200 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5dd0061650 con 0x7f5dbc0a1120 2024-05-10T03:59:52.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T03:59:52.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T03:59:52.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:52.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: from='osd.4 [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T03:59:52.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T03:59:52.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: Detected new or changed devices on smithi161 2024-05-10T03:59:52.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:52.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:52.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:52.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:52.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: Adjusting osd_memory_target on smithi161 to 8557M 2024-05-10T03:59:52.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:52.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:52.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:52.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:52.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:51 smithi150 ceph-mon[42477]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-10T03:59:52.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T03:59:52.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: from='osd.4 [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: Detected new or changed devices on smithi161 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: Adjusting osd_memory_target on smithi161 to 8557M 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:52.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:51 smithi181 ceph-mon[46602]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-10T03:59:52.242 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 03:59:51 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-4[59786]: 2024-05-10T03:59:51.808+0000 7f40fd1a9700 -1 osd.4 0 waiting for initial osdmap 2024-05-10T03:59:52.242 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 03:59:51 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-4[59786]: 2024-05-10T03:59:51.817+0000 7f40f8f60700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: from='osd.4 [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: Detected new or changed devices on smithi161 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: Adjusting osd_memory_target on smithi161 to 8557M 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T03:59:52.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T03:59:52.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:51 smithi161 ceph-mon[46918]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-10T03:59:53.203 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:52 smithi181 ceph-mon[46602]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T03:59:53.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:52 smithi181 ceph-mon[46602]: osdmap e31: 5 total, 4 up, 5 in 2024-05-10T03:59:53.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:52 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:53.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:52 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:53.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:52 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:59:53.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:52 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:59:53.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:52 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:53.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:52 smithi150 ceph-mon[42477]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T03:59:53.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:52 smithi150 ceph-mon[42477]: osdmap e31: 5 total, 4 up, 5 in 2024-05-10T03:59:53.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:53.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:53.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:59:53.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:59:53.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:53.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:52 smithi161 ceph-mon[46918]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T03:59:53.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:52 smithi161 ceph-mon[46918]: osdmap e31: 5 total, 4 up, 5 in 2024-05-10T03:59:53.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:52 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:53.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:52 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:53.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:52 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T03:59:53.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:52 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T03:59:53.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:52 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T03:59:54.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:53 smithi150 ceph-mon[42477]: purged_snaps scrub starts 2024-05-10T03:59:54.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:53 smithi150 ceph-mon[42477]: purged_snaps scrub ok 2024-05-10T03:59:54.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:53 smithi150 ceph-mon[42477]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:59:54.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:53 smithi150 ceph-mon[42477]: osd.4 [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503] boot 2024-05-10T03:59:54.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:53 smithi150 ceph-mon[42477]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T03:59:54.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:53 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:54.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:53 smithi150 ceph-mon[42477]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-10T03:59:54.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:53 smithi181 ceph-mon[46602]: purged_snaps scrub starts 2024-05-10T03:59:54.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:53 smithi181 ceph-mon[46602]: purged_snaps scrub ok 2024-05-10T03:59:54.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:53 smithi181 ceph-mon[46602]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:59:54.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:53 smithi181 ceph-mon[46602]: osd.4 [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503] boot 2024-05-10T03:59:54.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:53 smithi181 ceph-mon[46602]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T03:59:54.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:53 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:54.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:53 smithi181 ceph-mon[46602]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-10T03:59:54.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:53 smithi161 ceph-mon[46918]: purged_snaps scrub starts 2024-05-10T03:59:54.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:53 smithi161 ceph-mon[46918]: purged_snaps scrub ok 2024-05-10T03:59:54.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:53 smithi161 ceph-mon[46918]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T03:59:54.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:53 smithi161 ceph-mon[46918]: osd.4 [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503] boot 2024-05-10T03:59:54.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:53 smithi161 ceph-mon[46918]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T03:59:54.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:53 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T03:59:54.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:53 smithi161 ceph-mon[46918]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-10T03:59:55.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:54 smithi150 ceph-mon[42477]: osdmap e33: 5 total, 5 up, 5 in 2024-05-10T03:59:55.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:54 smithi150 ceph-mon[42477]: from='client.? 172.21.15.181:0/278000863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb8856d9-daa0-4119-9da6-84cc2c9c5dbc"}]: dispatch 2024-05-10T03:59:55.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:54 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb8856d9-daa0-4119-9da6-84cc2c9c5dbc"}]: dispatch 2024-05-10T03:59:55.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:54 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb8856d9-daa0-4119-9da6-84cc2c9c5dbc"}]': finished 2024-05-10T03:59:55.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:54 smithi150 ceph-mon[42477]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T03:59:55.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:54 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T03:59:55.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:54 smithi150 ceph-mon[42477]: from='client.? 172.21.15.181:0/2345792334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:59:55.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:54 smithi181 ceph-mon[46602]: osdmap e33: 5 total, 5 up, 5 in 2024-05-10T03:59:55.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:54 smithi181 ceph-mon[46602]: from='client.? 172.21.15.181:0/278000863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb8856d9-daa0-4119-9da6-84cc2c9c5dbc"}]: dispatch 2024-05-10T03:59:55.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:54 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb8856d9-daa0-4119-9da6-84cc2c9c5dbc"}]: dispatch 2024-05-10T03:59:55.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:54 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb8856d9-daa0-4119-9da6-84cc2c9c5dbc"}]': finished 2024-05-10T03:59:55.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:54 smithi181 ceph-mon[46602]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T03:59:55.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:54 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T03:59:55.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:54 smithi181 ceph-mon[46602]: from='client.? 172.21.15.181:0/2345792334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:59:55.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:54 smithi161 ceph-mon[46918]: osdmap e33: 5 total, 5 up, 5 in 2024-05-10T03:59:55.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:54 smithi161 ceph-mon[46918]: from='client.? 172.21.15.181:0/278000863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb8856d9-daa0-4119-9da6-84cc2c9c5dbc"}]: dispatch 2024-05-10T03:59:55.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:54 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb8856d9-daa0-4119-9da6-84cc2c9c5dbc"}]: dispatch 2024-05-10T03:59:55.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:54 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb8856d9-daa0-4119-9da6-84cc2c9c5dbc"}]': finished 2024-05-10T03:59:55.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:54 smithi161 ceph-mon[46918]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T03:59:55.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:54 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T03:59:55.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:54 smithi161 ceph-mon[46918]: from='client.? 172.21.15.181:0/2345792334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T03:59:56.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:55 smithi150 ceph-mon[42477]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-10T03:59:56.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:55 smithi181 ceph-mon[46602]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-10T03:59:56.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:55 smithi161 ceph-mon[46918]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-10T03:59:58.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 03:59:58 smithi150 ceph-mon[42477]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-10T03:59:58.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 03:59:58 smithi181 ceph-mon[46602]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-10T03:59:58.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 03:59:58 smithi161 ceph-mon[46918]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:00.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:00 smithi181 ceph-mon[46602]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:00.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:00 smithi181 ceph-mon[46602]: overall HEALTH_OK 2024-05-10T04:00:00.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:00 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T04:00:00.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:00 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:00.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:00 smithi150 ceph-mon[42477]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:00.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:00 smithi150 ceph-mon[42477]: overall HEALTH_OK 2024-05-10T04:00:00.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:00 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T04:00:00.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:00 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:00.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:00 smithi161 ceph-mon[46918]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:00.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:00 smithi161 ceph-mon[46918]: overall HEALTH_OK 2024-05-10T04:00:00.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:00 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T04:00:00.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:00 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:01.186 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:01 smithi181 ceph-mon[46602]: Deploying daemon osd.5 on smithi181 2024-05-10T04:00:01.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:01 smithi150 ceph-mon[42477]: Deploying daemon osd.5 on smithi181 2024-05-10T04:00:01.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:01 smithi161 ceph-mon[46918]: Deploying daemon osd.5 on smithi181 2024-05-10T04:00:02.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:02 smithi150 ceph-mon[42477]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:02.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:02 smithi181 ceph-mon[46602]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:02.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:02 smithi161 ceph-mon[46918]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:04.367 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:04 smithi181 ceph-mon[46602]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:04.367 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:04.367 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:04.367 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:04.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:04 smithi150 ceph-mon[42477]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:04.761 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:04.762 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:04.762 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:04.762 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:04 smithi161 ceph-mon[46918]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:04.763 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:04 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:04.763 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:04 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:04.763 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:04 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:06.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:06.162+0000 7f5dccff9700 1 -- 172.21.15.181:0/1229717200 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5dd0061650 con 0x7f5dbc0a1120 2024-05-10T04:00:06.166 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 5 on host 'smithi181' 2024-05-10T04:00:06.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:06.165+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/1229717200 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5dbc0a1120 msgr2=0x7f5dbc0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:06.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:06.165+0000 7f5dd4d9f700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5dbc0a1120 0x7f5dbc0a3630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000b5c0 tx=0x7f5dc001a040 comp rx=0 tx=0).stop 2024-05-10T04:00:06.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:06.165+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/1229717200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd0101b10 msgr2=0x7f5dd019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:06.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:06.165+0000 7f5dd4d9f700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd0101b10 0x7f5dd019cce0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5db800d950 tx=0x7f5db800dc60 comp rx=0 tx=0).stop 2024-05-10T04:00:06.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:06.165+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/1229717200 shutdown_connections 2024-05-10T04:00:06.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:06.165+0000 7f5dd4d9f700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5dbc0a1120 0x7f5dbc0a3630 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:06.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:06.165+0000 7f5dd4d9f700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5dd01024b0 0x7f5dd019f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:06.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:06.165+0000 7f5dd4d9f700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dd01009e0 0x7f5dd0069780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:06.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:06.165+0000 7f5dd4d9f700 1 --2- 172.21.15.181:0/1229717200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd0101b10 0x7f5dd019cce0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:06.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:06.165+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/1229717200 >> 172.21.15.181:0/1229717200 conn(0x7f5dd00fc020 msgr2=0x7f5dd0104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:00:06.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:06.165+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/1229717200 shutdown_connections 2024-05-10T04:00:06.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:06.165+0000 7f5dd4d9f700 1 -- 172.21.15.181:0/1229717200 wait complete. 2024-05-10T04:00:06.351 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:06 smithi181 ceph-mon[46602]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:06.351 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:06 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:06.352 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:06 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:06.352 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:06 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:06.352 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:06 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:06.352 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:06 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:06.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:06 smithi150 ceph-mon[42477]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:06.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:06 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:06.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:06 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:06.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:06 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:06.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:06 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:06.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:06 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:06.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:06 smithi161 ceph-mon[46918]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:06.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:06 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:06.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:06 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:06.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:06 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:06.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:06 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:06.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:06 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:06.527 DEBUG:teuthology.orchestra.run.smithi181:osd.5> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.5.service 2024-05-10T04:00:06.529 INFO:tasks.cephadm:Deploying osd.6 on smithi150 with /dev/vg_nvme/lv_2... 2024-05-10T04:00:06.529 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T04:00:06.561 INFO:journalctl@ceph.osd.5.smithi181.stdout:-- Logs begin at Fri 2024-05-10 03:40:19 UTC. -- 2024-05-10T04:00:06.762 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:00:07.326 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:07 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:07.326 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:07 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:07.326 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:07 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:07.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:07 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:07.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:07 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:07.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:07 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:07.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:07.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:07.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:08.039 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T04:00:08.040 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T04:00:08.040 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-10T04:00:08.040 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-10T04:00:08.040 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112088 s, 935 MB/s 2024-05-10T04:00:08.040 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-10T04:00:08.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:08 smithi181 ceph-mon[46602]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:08.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:08 smithi181 ceph-mon[46602]: from='osd.5 [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T04:00:08.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:08 smithi181 ceph-mon[46602]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T04:00:08.239 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:00:07 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:00:07.914+0000 7fbff554f7c0 -1 osd.5 0 log_to_monitors true 2024-05-10T04:00:08.328 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:08 smithi150 ceph-mon[42477]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:08.328 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:08 smithi150 ceph-mon[42477]: from='osd.5 [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T04:00:08.328 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:08 smithi150 ceph-mon[42477]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T04:00:08.412 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-05-10T04:00:08.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:08 smithi161 ceph-mon[46918]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:08.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:08 smithi161 ceph-mon[46918]: from='osd.5 [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T04:00:08.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:08 smithi161 ceph-mon[46918]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T04:00:08.635 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:00:09.392 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:09 smithi150 ceph-mon[42477]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T04:00:09.393 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:09 smithi150 ceph-mon[42477]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T04:00:09.393 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:00:09.393 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:09 smithi150 ceph-mon[42477]: from='osd.5 [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:00:09.393 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:09 smithi150 ceph-mon[42477]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:00:09.393 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:09.393 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:09.393 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:09.393 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:09.393 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:09.393 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:09.393 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:09.393 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:09.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:09 smithi181 ceph-mon[46602]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T04:00:09.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:09 smithi181 ceph-mon[46602]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T04:00:09.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:00:09.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:09 smithi181 ceph-mon[46602]: from='osd.5 [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:00:09.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:09 smithi181 ceph-mon[46602]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:00:09.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:09.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:09.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:09.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:09.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:09.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:09.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:09.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:09.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:09 smithi161 ceph-mon[46918]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T04:00:09.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:09 smithi161 ceph-mon[46918]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T04:00:09.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:00:09.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:09 smithi161 ceph-mon[46918]: from='osd.5 [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:00:09.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:09 smithi161 ceph-mon[46918]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:00:09.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:09.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:09.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:09.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:09.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:09.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:09.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:09.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:09.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.578+0000 7ff977238700 1 -- 172.21.15.150:0/461382792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff970100b80 msgr2=0x7ff970100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:09.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.578+0000 7ff977238700 1 --2- 172.21.15.150:0/461382792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff970100b80 0x7ff970100f90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff964009990 tx=0x7ff964009ca0 comp rx=0 tx=0).stop 2024-05-10T04:00:09.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.578+0000 7ff977238700 1 -- 172.21.15.150:0/461382792 shutdown_connections 2024-05-10T04:00:09.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.578+0000 7ff977238700 1 --2- 172.21.15.150:0/461382792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff970102650 0x7ff970108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:09.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.578+0000 7ff977238700 1 --2- 172.21.15.150:0/461382792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff970100b80 0x7ff970100f90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:09.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.578+0000 7ff977238700 1 --2- 172.21.15.150:0/461382792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff970101cb0 0x7ff9701020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:09.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.578+0000 7ff977238700 1 -- 172.21.15.150:0/461382792 >> 172.21.15.150:0/461382792 conn(0x7ff9700fc1c0 msgr2=0x7ff9700fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:00:09.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.578+0000 7ff977238700 1 -- 172.21.15.150:0/461382792 shutdown_connections 2024-05-10T04:00:09.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.578+0000 7ff977238700 1 -- 172.21.15.150:0/461382792 wait complete. 2024-05-10T04:00:09.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.579+0000 7ff977238700 1 Processor -- start 2024-05-10T04:00:09.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.579+0000 7ff977238700 1 -- start start 2024-05-10T04:00:09.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.579+0000 7ff977238700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff970100b80 0x7ff97019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:00:09.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.579+0000 7ff977238700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff970101cb0 0x7ff97019d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:00:09.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff977238700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff970102650 0x7ff97019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:00:09.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff977238700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff970069c40 con 0x7ff970101cb0 2024-05-10T04:00:09.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff977238700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97019fe00 con 0x7ff970102650 2024-05-10T04:00:09.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff977238700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97019ff40 con 0x7ff970100b80 2024-05-10T04:00:09.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff96ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff970101cb0 0x7ff97019d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff974fd4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff970100b80 0x7ff97019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff96ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff970101cb0 0x7ff97019d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38900/0 (socket says 172.21.15.150:38900) 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff96ffff700 1 -- 172.21.15.150:0/3044743155 learned_addr learned my addr 172.21.15.150:0/3044743155 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff9757d5700 1 --2- 172.21.15.150:0/3044743155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff970102650 0x7ff97019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff96ffff700 1 -- 172.21.15.150:0/3044743155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff970100b80 msgr2=0x7ff97019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff96ffff700 1 --2- 172.21.15.150:0/3044743155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff970100b80 0x7ff97019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff96ffff700 1 -- 172.21.15.150:0/3044743155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff970102650 msgr2=0x7ff97019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff96ffff700 1 --2- 172.21.15.150:0/3044743155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff970102650 0x7ff97019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff96ffff700 1 -- 172.21.15.150:0/3044743155 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9640095f0 con 0x7ff970101cb0 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff96ffff700 1 --2- 172.21.15.150:0/3044743155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff970101cb0 0x7ff97019d370 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00c9b0 tx=0x7ff95c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff96dffb700 1 -- 172.21.15.150:0/3044743155 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c007b00 con 0x7ff970101cb0 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff977238700 1 -- 172.21.15.150:0/3044743155 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff970196c60 con 0x7ff970101cb0 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff96dffb700 1 -- 172.21.15.150:0/3044743155 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff95c00ce50 con 0x7ff970101cb0 2024-05-10T04:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.580+0000 7ff96dffb700 1 -- 172.21.15.150:0/3044743155 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c018660 con 0x7ff970101cb0 2024-05-10T04:00:09.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.581+0000 7ff977238700 1 -- 172.21.15.150:0/3044743155 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff970197180 con 0x7ff970101cb0 2024-05-10T04:00:09.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.581+0000 7ff977238700 1 -- 172.21.15.150:0/3044743155 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff970107e10 con 0x7ff970101cb0 2024-05-10T04:00:09.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.583+0000 7ff96dffb700 1 -- 172.21.15.150:0/3044743155 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff95c00f450 con 0x7ff970101cb0 2024-05-10T04:00:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.583+0000 7ff96dffb700 1 --2- 172.21.15.150:0/3044743155 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff9600a1120 0x7ff9600a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:00:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.583+0000 7ff974fd4700 1 --2- 172.21.15.150:0/3044743155 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff9600a1120 0x7ff9600a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:00:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.583+0000 7ff96dffb700 1 -- 172.21.15.150:0/3044743155 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4253+0+0 (secure 0 0 0) 0x7ff95c0cb820 con 0x7ff970101cb0 2024-05-10T04:00:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.584+0000 7ff974fd4700 1 --2- 172.21.15.150:0/3044743155 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff9600a1120 0x7ff9600a3630 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff96400b5c0 tx=0x7ff96401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:00:09.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.588+0000 7ff96dffb700 1 -- 172.21.15.150:0/3044743155 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff95c095f90 con 0x7ff970101cb0 2024-05-10T04:00:09.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:09.795+0000 7ff977238700 1 -- 172.21.15.150:0/3044743155 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff97010c500 con 0x7ff9600a1120 2024-05-10T04:00:09.988 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:00:09 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:00:09.611+0000 7fbfeabb3700 -1 osd.5 0 waiting for initial osdmap 2024-05-10T04:00:09.988 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:00:09 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:00:09.618+0000 7fbfe2962700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:00:10.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:10 smithi150 ceph-mon[42477]: Detected new or changed devices on smithi181 2024-05-10T04:00:10.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:10 smithi150 ceph-mon[42477]: Adjusting osd_memory_target on smithi181 to 8556M 2024-05-10T04:00:10.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:10 smithi150 ceph-mon[42477]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:10.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:10 smithi150 ceph-mon[42477]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:00:10.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:10 smithi150 ceph-mon[42477]: osdmap e36: 6 total, 5 up, 6 in 2024-05-10T04:00:10.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:10 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:00:10.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:10 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:00:10.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:10 smithi150 ceph-mon[42477]: from='osd.5 ' entity='osd.5' 2024-05-10T04:00:10.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:10 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:00:10.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:10 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:00:10.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:10 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:10.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:10 smithi181 ceph-mon[46602]: Detected new or changed devices on smithi181 2024-05-10T04:00:10.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:10 smithi181 ceph-mon[46602]: Adjusting osd_memory_target on smithi181 to 8556M 2024-05-10T04:00:10.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:10 smithi181 ceph-mon[46602]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:10.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:10 smithi181 ceph-mon[46602]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:00:10.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:10 smithi181 ceph-mon[46602]: osdmap e36: 6 total, 5 up, 6 in 2024-05-10T04:00:10.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:10 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:00:10.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:10 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:00:10.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:10 smithi181 ceph-mon[46602]: from='osd.5 ' entity='osd.5' 2024-05-10T04:00:10.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:10 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:00:10.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:10 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:00:10.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:10 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:10.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:10 smithi161 ceph-mon[46918]: Detected new or changed devices on smithi181 2024-05-10T04:00:10.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:10 smithi161 ceph-mon[46918]: Adjusting osd_memory_target on smithi181 to 8556M 2024-05-10T04:00:10.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:10 smithi161 ceph-mon[46918]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:10.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:10 smithi161 ceph-mon[46918]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:00:10.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:10 smithi161 ceph-mon[46918]: osdmap e36: 6 total, 5 up, 6 in 2024-05-10T04:00:10.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:10 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:00:10.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:10 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:00:10.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:10 smithi161 ceph-mon[46918]: from='osd.5 ' entity='osd.5' 2024-05-10T04:00:10.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:10 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:00:10.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:10 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:00:10.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:10 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:11.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:11 smithi150 ceph-mon[42477]: purged_snaps scrub starts 2024-05-10T04:00:11.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:11 smithi150 ceph-mon[42477]: purged_snaps scrub ok 2024-05-10T04:00:11.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:11 smithi150 ceph-mon[42477]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:00:11.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:11 smithi150 ceph-mon[42477]: osd.5 [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780] boot 2024-05-10T04:00:11.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:11 smithi150 ceph-mon[42477]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T04:00:11.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:11 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:00:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:11 smithi181 ceph-mon[46602]: purged_snaps scrub starts 2024-05-10T04:00:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:11 smithi181 ceph-mon[46602]: purged_snaps scrub ok 2024-05-10T04:00:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:11 smithi181 ceph-mon[46602]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:00:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:11 smithi181 ceph-mon[46602]: osd.5 [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780] boot 2024-05-10T04:00:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:11 smithi181 ceph-mon[46602]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T04:00:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:11 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:00:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:11 smithi161 ceph-mon[46918]: purged_snaps scrub starts 2024-05-10T04:00:11.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:11 smithi161 ceph-mon[46918]: purged_snaps scrub ok 2024-05-10T04:00:11.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:11 smithi161 ceph-mon[46918]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:00:11.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:11 smithi161 ceph-mon[46918]: osd.5 [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780] boot 2024-05-10T04:00:11.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:11 smithi161 ceph-mon[46918]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T04:00:11.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:11 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:00:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:12 smithi150 ceph-mon[42477]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:12 smithi150 ceph-mon[42477]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T04:00:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:12 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3348247452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8"}]: dispatch 2024-05-10T04:00:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:12 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3348247452' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8"}]': finished 2024-05-10T04:00:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:12 smithi150 ceph-mon[42477]: osdmap e39: 7 total, 6 up, 7 in 2024-05-10T04:00:12.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:12 smithi181 ceph-mon[46602]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:12 smithi181 ceph-mon[46602]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T04:00:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:12 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3348247452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8"}]: dispatch 2024-05-10T04:00:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:12 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3348247452' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8"}]': finished 2024-05-10T04:00:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:12 smithi181 ceph-mon[46602]: osdmap e39: 7 total, 6 up, 7 in 2024-05-10T04:00:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:12 smithi161 ceph-mon[46918]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:00:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:12 smithi161 ceph-mon[46918]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T04:00:12.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:12 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3348247452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8"}]: dispatch 2024-05-10T04:00:12.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:12 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3348247452' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8"}]': finished 2024-05-10T04:00:12.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:12 smithi161 ceph-mon[46918]: osdmap e39: 7 total, 6 up, 7 in 2024-05-10T04:00:12.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:13.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:13 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3030624848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:00:13.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:13 smithi150 ceph-mon[42477]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T04:00:13.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:13.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:13 smithi150 ceph-mon[42477]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:00:13.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:13 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3030624848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:00:13.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:13 smithi181 ceph-mon[46602]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T04:00:13.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:13.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:13 smithi181 ceph-mon[46602]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:00:13.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:13 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3030624848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:00:13.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:13 smithi161 ceph-mon[46918]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T04:00:13.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:13.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:13 smithi161 ceph-mon[46918]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:00:14.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:14 smithi150 ceph-mon[42477]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T04:00:14.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:14.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:14 smithi181 ceph-mon[46602]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T04:00:14.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:14.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:14 smithi161 ceph-mon[46918]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T04:00:14.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:15.627 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:15 smithi150 ceph-mon[42477]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:00:15.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:15 smithi181 ceph-mon[46602]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:00:15.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:15 smithi161 ceph-mon[46918]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:00:18.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:18 smithi150 ceph-mon[42477]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-10T04:00:18.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:18 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T04:00:18.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:18 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:18.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:18 smithi181 ceph-mon[46602]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-10T04:00:18.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:18 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T04:00:18.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:18 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:18.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:18 smithi161 ceph-mon[46918]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-10T04:00:18.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:18 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T04:00:18.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:18 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:19.210 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:19 smithi150 ceph-mon[42477]: Deploying daemon osd.6 on smithi150 2024-05-10T04:00:19.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:19 smithi181 ceph-mon[46602]: Deploying daemon osd.6 on smithi150 2024-05-10T04:00:19.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:19 smithi161 ceph-mon[46918]: Deploying daemon osd.6 on smithi150 2024-05-10T04:00:20.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:20 smithi150 ceph-mon[42477]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-10T04:00:20.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:20 smithi181 ceph-mon[46602]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-10T04:00:20.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:20 smithi161 ceph-mon[46918]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-10T04:00:22.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:22 smithi150 ceph-mon[42477]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-10T04:00:22.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:22 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:22.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:22 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:22.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:22 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:22.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:22 smithi181 ceph-mon[46602]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-10T04:00:22.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:22 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:22.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:22 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:22.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:22 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:22.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:22 smithi161 ceph-mon[46918]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-10T04:00:22.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:22 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:22.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:22 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:22.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:22 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:23.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:23.489+0000 7ff96dffb700 1 -- 172.21.15.150:0/3044743155 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff97010c500 con 0x7ff9600a1120 2024-05-10T04:00:23.492 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 6 on host 'smithi150' 2024-05-10T04:00:23.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:23.491+0000 7ff977238700 1 -- 172.21.15.150:0/3044743155 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff9600a1120 msgr2=0x7ff9600a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:23.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:23.491+0000 7ff977238700 1 --2- 172.21.15.150:0/3044743155 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff9600a1120 0x7ff9600a3630 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff96400b5c0 tx=0x7ff96401a040 comp rx=0 tx=0).stop 2024-05-10T04:00:23.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:23.491+0000 7ff977238700 1 -- 172.21.15.150:0/3044743155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff970101cb0 msgr2=0x7ff97019d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:23.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:23.491+0000 7ff977238700 1 --2- 172.21.15.150:0/3044743155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff970101cb0 0x7ff97019d370 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00c9b0 tx=0x7ff95c00ccc0 comp rx=0 tx=0).stop 2024-05-10T04:00:23.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:23.491+0000 7ff977238700 1 -- 172.21.15.150:0/3044743155 shutdown_connections 2024-05-10T04:00:23.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:23.491+0000 7ff977238700 1 --2- 172.21.15.150:0/3044743155 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff9600a1120 0x7ff9600a3630 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:23.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:23.491+0000 7ff977238700 1 --2- 172.21.15.150:0/3044743155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff970102650 0x7ff97019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:23.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:23.491+0000 7ff977238700 1 --2- 172.21.15.150:0/3044743155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff970100b80 0x7ff97019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:23.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:23.491+0000 7ff977238700 1 --2- 172.21.15.150:0/3044743155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff970101cb0 0x7ff97019d370 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:23.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:23.491+0000 7ff977238700 1 -- 172.21.15.150:0/3044743155 >> 172.21.15.150:0/3044743155 conn(0x7ff9700fc1c0 msgr2=0x7ff970104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:00:23.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:23.492+0000 7ff977238700 1 -- 172.21.15.150:0/3044743155 shutdown_connections 2024-05-10T04:00:23.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:00:23.492+0000 7ff977238700 1 -- 172.21.15.150:0/3044743155 wait complete. 2024-05-10T04:00:23.874 DEBUG:teuthology.orchestra.run.smithi150:osd.6> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.6.service 2024-05-10T04:00:23.876 INFO:tasks.cephadm:Deploying osd.7 on smithi161 with /dev/vg_nvme/lv_2... 2024-05-10T04:00:23.877 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T04:00:23.905 INFO:journalctl@ceph.osd.6.smithi150.stdout:-- Logs begin at Fri 2024-05-10 03:40:30 UTC. -- 2024-05-10T04:00:24.053 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.053 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.053 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:24.053 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:24.053 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.053 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:23 smithi150 ceph-mon[42477]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:00:24.053 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:24.053 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.053 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.112 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:00:24.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:24.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:24.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:23 smithi181 ceph-mon[46602]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:00:24.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:24.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:24.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:24.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:23 smithi161 ceph-mon[46918]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:00:24.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:24.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:24.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:25.449 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T04:00:25.449 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T04:00:25.450 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-05-10T04:00:25.450 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-05-10T04:00:25.450 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113378 s, 925 MB/s 2024-05-10T04:00:25.450 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-05-10T04:00:25.654 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:00:25 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:00:25.261+0000 7faee9ea77c0 -1 osd.6 0 log_to_monitors true 2024-05-10T04:00:25.834 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-05-10T04:00:26.055 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:00:26.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:26 smithi150 ceph-mon[42477]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-10T04:00:26.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:26 smithi150 ceph-mon[42477]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T04:00:26.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:26.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:26.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:26.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:26.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:26.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:26.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:26.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:26.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:26.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:26 smithi181 ceph-mon[46602]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-10T04:00:26.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:26 smithi181 ceph-mon[46602]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T04:00:26.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:26.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:26.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:26.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:26.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:26.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:26.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:26.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:26.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:26.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:26 smithi161 ceph-mon[46918]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-10T04:00:26.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:26 smithi161 ceph-mon[46918]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T04:00:26.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:26.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:26.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:26.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:26.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:26.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:26.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:26.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:26.494 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:27.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.039+0000 7fa03b57b700 1 -- 172.21.15.161:0/1205590989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa034101cb0 msgr2=0x7fa0341020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:27.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.039+0000 7fa03b57b700 1 --2- 172.21.15.161:0/1205590989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa034101cb0 0x7fa0341020c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa0240098f0 tx=0x7fa024009c00 comp rx=0 tx=0).stop 2024-05-10T04:00:27.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.039+0000 7fa03b57b700 1 -- 172.21.15.161:0/1205590989 shutdown_connections 2024-05-10T04:00:27.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.039+0000 7fa03b57b700 1 --2- 172.21.15.161:0/1205590989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa034102650 0x7fa034108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:27.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.039+0000 7fa03b57b700 1 --2- 172.21.15.161:0/1205590989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa034100b80 0x7fa034100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:27.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.039+0000 7fa03b57b700 1 --2- 172.21.15.161:0/1205590989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa034101cb0 0x7fa0341020c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:27.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.039+0000 7fa03b57b700 1 -- 172.21.15.161:0/1205590989 >> 172.21.15.161:0/1205590989 conn(0x7fa0340fc1c0 msgr2=0x7fa0340fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:00:27.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.039+0000 7fa03b57b700 1 -- 172.21.15.161:0/1205590989 shutdown_connections 2024-05-10T04:00:27.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.039+0000 7fa03b57b700 1 -- 172.21.15.161:0/1205590989 wait complete. 2024-05-10T04:00:27.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.040+0000 7fa03b57b700 1 Processor -- start 2024-05-10T04:00:27.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.040+0000 7fa03b57b700 1 -- start start 2024-05-10T04:00:27.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.040+0000 7fa03b57b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa034100b80 0x7fa03419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:00:27.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.040+0000 7fa03b57b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa034101cb0 0x7fa03419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:00:27.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.040+0000 7fa03b57b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa034102650 0x7fa03419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:00:27.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.040+0000 7fa03b57b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03419fe70 con 0x7fa034102650 2024-05-10T04:00:27.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.040+0000 7fa03b57b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03419ffb0 con 0x7fa034100b80 2024-05-10T04:00:27.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.040+0000 7fa03b57b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0341a00f0 con 0x7fa034101cb0 2024-05-10T04:00:27.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.040+0000 7fa039317700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa034100b80 0x7fa03419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:00:27.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.040+0000 7fa039317700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa034100b80 0x7fa03419ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58506/0 (socket says 172.21.15.161:58506) 2024-05-10T04:00:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.040+0000 7fa039317700 1 -- 172.21.15.161:0/263540357 learned_addr learned my addr 172.21.15.161:0/263540357 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-10T04:00:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa038b16700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa034101cb0 0x7fa03419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:00:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa039b18700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa034102650 0x7fa03419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:00:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa039317700 1 -- 172.21.15.161:0/263540357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa034101cb0 msgr2=0x7fa03419d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa039317700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa034101cb0 0x7fa03419d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa039317700 1 -- 172.21.15.161:0/263540357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa034102650 msgr2=0x7fa03419f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa039317700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa034102650 0x7fa03419f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:27.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa039317700 1 -- 172.21.15.161:0/263540357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02c009d00 con 0x7fa034100b80 2024-05-10T04:00:27.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa039b18700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa034102650 0x7fa03419f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:00:27.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa039317700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa034100b80 0x7fa03419ae90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa03000e940 tx=0x7fa03000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:00:27.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa02a7fc700 1 -- 172.21.15.161:0/263540357 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03000cb70 con 0x7fa034100b80 2024-05-10T04:00:27.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa03b57b700 1 -- 172.21.15.161:0/263540357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa024009520 con 0x7fa034100b80 2024-05-10T04:00:27.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa02a7fc700 1 -- 172.21.15.161:0/263540357 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0300044e0 con 0x7fa034100b80 2024-05-10T04:00:27.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa02a7fc700 1 -- 172.21.15.161:0/263540357 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa030003cf0 con 0x7fa034100b80 2024-05-10T04:00:27.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.041+0000 7fa03b57b700 1 -- 172.21.15.161:0/263540357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0341a06c0 con 0x7fa034100b80 2024-05-10T04:00:27.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.042+0000 7fa03b57b700 1 -- 172.21.15.161:0/263540357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa034107e10 con 0x7fa034100b80 2024-05-10T04:00:27.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.044+0000 7fa02a7fc700 1 -- 172.21.15.161:0/263540357 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa030003e70 con 0x7fa034100b80 2024-05-10T04:00:27.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.044+0000 7fa02a7fc700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa0200a1120 0x7fa0200a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:00:27.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.044+0000 7fa02a7fc700 1 -- 172.21.15.161:0/263540357 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4669+0+0 (secure 0 0 0) 0x7fa030014070 con 0x7fa034100b80 2024-05-10T04:00:27.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.045+0000 7fa038b16700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa0200a1120 0x7fa0200a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:00:27.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.045+0000 7fa038b16700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa0200a1120 0x7fa0200a3630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa024009810 tx=0x7fa024019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:00:27.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.047+0000 7fa02a7fc700 1 -- 172.21.15.161:0/263540357 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa0300964e0 con 0x7fa034100b80 2024-05-10T04:00:27.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:27.253+0000 7fa03b57b700 1 -- 172.21.15.161:0/263540357 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa03410c620 con 0x7fa0200a1120 2024-05-10T04:00:27.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:27 smithi150 ceph-mon[42477]: Detected new or changed devices on smithi150 2024-05-10T04:00:27.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:27 smithi150 ceph-mon[42477]: Adjusting osd_memory_target on smithi150 to 5704M 2024-05-10T04:00:27.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:27 smithi150 ceph-mon[42477]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T04:00:27.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:27 smithi150 ceph-mon[42477]: osdmap e42: 7 total, 6 up, 7 in 2024-05-10T04:00:27.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:27 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:27.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:27 smithi150 ceph-mon[42477]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:00:27.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:27 smithi150 ceph-mon[42477]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:00:27.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:27 smithi150 ceph-mon[42477]: osdmap e43: 7 total, 6 up, 7 in 2024-05-10T04:00:27.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:27 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:27.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:27 smithi181 ceph-mon[46602]: Detected new or changed devices on smithi150 2024-05-10T04:00:27.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:27 smithi181 ceph-mon[46602]: Adjusting osd_memory_target on smithi150 to 5704M 2024-05-10T04:00:27.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:27 smithi181 ceph-mon[46602]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T04:00:27.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:27 smithi181 ceph-mon[46602]: osdmap e42: 7 total, 6 up, 7 in 2024-05-10T04:00:27.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:27 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:27.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:27 smithi181 ceph-mon[46602]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:00:27.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:27 smithi181 ceph-mon[46602]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:00:27.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:27 smithi181 ceph-mon[46602]: osdmap e43: 7 total, 6 up, 7 in 2024-05-10T04:00:27.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:27 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:27.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:27 smithi161 ceph-mon[46918]: Detected new or changed devices on smithi150 2024-05-10T04:00:27.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:27 smithi161 ceph-mon[46918]: Adjusting osd_memory_target on smithi150 to 5704M 2024-05-10T04:00:27.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:27 smithi161 ceph-mon[46918]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T04:00:27.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:27 smithi161 ceph-mon[46918]: osdmap e42: 7 total, 6 up, 7 in 2024-05-10T04:00:27.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:27 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:27.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:27 smithi161 ceph-mon[46918]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:00:27.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:27 smithi161 ceph-mon[46918]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:00:27.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:27 smithi161 ceph-mon[46918]: osdmap e43: 7 total, 6 up, 7 in 2024-05-10T04:00:27.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:27 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:27.964 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:00:27 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:00:27.588+0000 7faedf50b700 -1 osd.6 0 waiting for initial osdmap 2024-05-10T04:00:27.964 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:00:27 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:00:27.599+0000 7faed72ba700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:00:28.355 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:28 smithi161 ceph-mon[46918]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:00:28.355 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:28.355 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:00:28.355 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:00:28.355 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:28.355 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:28 smithi161 ceph-mon[46918]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' 2024-05-10T04:00:28.355 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:28.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:28 smithi150 ceph-mon[42477]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:00:28.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:28.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:00:28.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:00:28.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:28.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:28 smithi150 ceph-mon[42477]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' 2024-05-10T04:00:28.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:28 smithi181 ceph-mon[46602]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:00:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:00:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:00:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:28 smithi181 ceph-mon[46602]: from='osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504]' entity='osd.6' 2024-05-10T04:00:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:29 smithi150 ceph-mon[42477]: purged_snaps scrub starts 2024-05-10T04:00:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:29 smithi150 ceph-mon[42477]: purged_snaps scrub ok 2024-05-10T04:00:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:29 smithi150 ceph-mon[42477]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:00:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:29 smithi150 ceph-mon[42477]: osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504] boot 2024-05-10T04:00:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:29 smithi150 ceph-mon[42477]: osdmap e44: 7 total, 7 up, 7 in 2024-05-10T04:00:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:29 smithi181 ceph-mon[46602]: purged_snaps scrub starts 2024-05-10T04:00:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:29 smithi181 ceph-mon[46602]: purged_snaps scrub ok 2024-05-10T04:00:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:29 smithi181 ceph-mon[46602]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:00:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:29 smithi181 ceph-mon[46602]: osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504] boot 2024-05-10T04:00:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:29 smithi181 ceph-mon[46602]: osdmap e44: 7 total, 7 up, 7 in 2024-05-10T04:00:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:29 smithi161 ceph-mon[46918]: purged_snaps scrub starts 2024-05-10T04:00:29.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:29 smithi161 ceph-mon[46918]: purged_snaps scrub ok 2024-05-10T04:00:29.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:29 smithi161 ceph-mon[46918]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:00:29.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:29 smithi161 ceph-mon[46918]: osd.6 [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504] boot 2024-05-10T04:00:29.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:29 smithi161 ceph-mon[46918]: osdmap e44: 7 total, 7 up, 7 in 2024-05-10T04:00:29.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:00:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:30 smithi150 ceph-mon[42477]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:00:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:30 smithi150 ceph-mon[42477]: from='client.? 172.21.15.161:0/2264450558' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e7b2d25-da3d-4d56-966a-13e6d9c2278e"}]: dispatch 2024-05-10T04:00:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:30 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e7b2d25-da3d-4d56-966a-13e6d9c2278e"}]: dispatch 2024-05-10T04:00:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:30 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e7b2d25-da3d-4d56-966a-13e6d9c2278e"}]': finished 2024-05-10T04:00:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:30 smithi150 ceph-mon[42477]: osdmap e45: 8 total, 7 up, 8 in 2024-05-10T04:00:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:30 smithi150 ceph-mon[42477]: from='client.? 172.21.15.161:0/1786084707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:00:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:30 smithi181 ceph-mon[46602]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:00:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:30 smithi181 ceph-mon[46602]: from='client.? 172.21.15.161:0/2264450558' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e7b2d25-da3d-4d56-966a-13e6d9c2278e"}]: dispatch 2024-05-10T04:00:30.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:30 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e7b2d25-da3d-4d56-966a-13e6d9c2278e"}]: dispatch 2024-05-10T04:00:30.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:30 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e7b2d25-da3d-4d56-966a-13e6d9c2278e"}]': finished 2024-05-10T04:00:30.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:30 smithi181 ceph-mon[46602]: osdmap e45: 8 total, 7 up, 8 in 2024-05-10T04:00:30.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:30.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:30 smithi181 ceph-mon[46602]: from='client.? 172.21.15.161:0/1786084707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:00:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:30 smithi161 ceph-mon[46918]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:00:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:30 smithi161 ceph-mon[46918]: from='client.? 172.21.15.161:0/2264450558' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e7b2d25-da3d-4d56-966a-13e6d9c2278e"}]: dispatch 2024-05-10T04:00:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:30 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e7b2d25-da3d-4d56-966a-13e6d9c2278e"}]: dispatch 2024-05-10T04:00:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:30 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e7b2d25-da3d-4d56-966a-13e6d9c2278e"}]': finished 2024-05-10T04:00:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:30 smithi161 ceph-mon[46918]: osdmap e45: 8 total, 7 up, 8 in 2024-05-10T04:00:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:30 smithi161 ceph-mon[46918]: from='client.? 172.21.15.161:0/1786084707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:00:31.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:31 smithi150 ceph-mon[42477]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T04:00:31.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:31 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:31.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:31 smithi150 ceph-mon[42477]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:31.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:31 smithi181 ceph-mon[46602]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T04:00:31.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:31 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:31.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:31 smithi181 ceph-mon[46602]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:31.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:31 smithi161 ceph-mon[46918]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T04:00:31.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:31 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:31.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:31 smithi161 ceph-mon[46918]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:34.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:34 smithi161 ceph-mon[46918]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:34.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:34 smithi150 ceph-mon[42477]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:34 smithi181 ceph-mon[46602]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:35.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:35 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T04:00:35.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:35 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:35.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:35 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T04:00:35.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:35 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:35.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:35 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T04:00:35.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:35 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:36.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:36 smithi150 ceph-mon[42477]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:36.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:36 smithi150 ceph-mon[42477]: Deploying daemon osd.7 on smithi161 2024-05-10T04:00:36.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:36 smithi181 ceph-mon[46602]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:36.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:36 smithi181 ceph-mon[46602]: Deploying daemon osd.7 on smithi161 2024-05-10T04:00:36.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:36 smithi161 ceph-mon[46918]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:36.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:36 smithi161 ceph-mon[46918]: Deploying daemon osd.7 on smithi161 2024-05-10T04:00:38.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:38 smithi150 ceph-mon[42477]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:38.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:38 smithi181 ceph-mon[46602]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:38.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:38 smithi161 ceph-mon[46918]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:40.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:39 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:40.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:39 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:40.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:39 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:40.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:39 smithi150 ceph-mon[42477]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:40.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:39 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:40.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:39 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:40.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:39 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:40.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:39 smithi181 ceph-mon[46602]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:40.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:39 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:40.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:39 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:40.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:39 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:40.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:39 smithi161 ceph-mon[46918]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:41.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:41.306+0000 7fa02a7fc700 1 -- 172.21.15.161:0/263540357 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa03410c620 con 0x7fa0200a1120 2024-05-10T04:00:41.311 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 7 on host 'smithi161' 2024-05-10T04:00:41.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:41.309+0000 7fa03b57b700 1 -- 172.21.15.161:0/263540357 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa0200a1120 msgr2=0x7fa0200a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:41.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:41.309+0000 7fa03b57b700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa0200a1120 0x7fa0200a3630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa024009810 tx=0x7fa024019040 comp rx=0 tx=0).stop 2024-05-10T04:00:41.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:41.309+0000 7fa03b57b700 1 -- 172.21.15.161:0/263540357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa034100b80 msgr2=0x7fa03419ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:41.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:41.309+0000 7fa03b57b700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa034100b80 0x7fa03419ae90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa03000e940 tx=0x7fa03000ed00 comp rx=0 tx=0).stop 2024-05-10T04:00:41.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:41.309+0000 7fa03b57b700 1 -- 172.21.15.161:0/263540357 shutdown_connections 2024-05-10T04:00:41.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:41.309+0000 7fa03b57b700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa0200a1120 0x7fa0200a3630 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:41.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:41.309+0000 7fa03b57b700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa034102650 0x7fa03419f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:41.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:41.309+0000 7fa03b57b700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa034100b80 0x7fa03419ae90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:41.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:41.309+0000 7fa03b57b700 1 --2- 172.21.15.161:0/263540357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa034101cb0 0x7fa03419d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:41.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:41.309+0000 7fa03b57b700 1 -- 172.21.15.161:0/263540357 >> 172.21.15.161:0/263540357 conn(0x7fa0340fc1c0 msgr2=0x7fa034104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:00:41.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:41.309+0000 7fa03b57b700 1 -- 172.21.15.161:0/263540357 shutdown_connections 2024-05-10T04:00:41.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:00:41.309+0000 7fa03b57b700 1 -- 172.21.15.161:0/263540357 wait complete. 2024-05-10T04:00:41.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:41 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:41.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:41 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:41.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:41 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:41.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:41 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:41.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:41 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:41.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:41 smithi161 ceph-mon[46918]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:41.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:41 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:41.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:41 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:41.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:41 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:41.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:41 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:41.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:41 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:41.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:41 smithi150 ceph-mon[42477]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:41.722 DEBUG:teuthology.orchestra.run.smithi161:osd.7> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.7.service 2024-05-10T04:00:41.724 INFO:tasks.cephadm:Deploying osd.8 on smithi181 with /dev/vg_nvme/lv_2... 2024-05-10T04:00:41.724 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T04:00:41.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:41 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:41.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:41 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:41.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:41 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:41.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:41 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:41.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:41 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:41.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:41 smithi181 ceph-mon[46602]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:41.761 INFO:journalctl@ceph.osd.7.smithi161.stdout:-- Logs begin at Fri 2024-05-10 03:40:24 UTC. -- 2024-05-10T04:00:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:00:42.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:42 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:42.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:42 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:42.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:42 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:42.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:42 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:42.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:42 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:42.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:42 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:42.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:42 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:42.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:42 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:42.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:42 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:42.993 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:00:42 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:00:42.794+0000 7f18b7e317c0 -1 osd.7 0 log_to_monitors true 2024-05-10T04:00:43.308 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T04:00:43.308 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T04:00:43.308 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-05-10T04:00:43.308 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-05-10T04:00:43.308 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.018714 s, 560 MB/s 2024-05-10T04:00:43.308 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-05-10T04:00:43.599 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:43 smithi181 ceph-mon[46602]: from='osd.7 [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T04:00:43.599 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:43 smithi181 ceph-mon[46602]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T04:00:43.599 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:43 smithi181 ceph-mon[46602]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:43.684 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:43 smithi161 ceph-mon[46918]: from='osd.7 [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T04:00:43.684 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:43 smithi161 ceph-mon[46918]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T04:00:43.684 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:43 smithi161 ceph-mon[46918]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:43.700 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_2 2024-05-10T04:00:43.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:43 smithi150 ceph-mon[42477]: from='osd.7 [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T04:00:43.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:43 smithi150 ceph-mon[42477]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T04:00:43.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:43 smithi150 ceph-mon[42477]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:00:44.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T04:00:44.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T04:00:44.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:44.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: from='osd.7 [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:00:44.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:00:44.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: Detected new or changed devices on smithi161 2024-05-10T04:00:44.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:44.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:44.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:44.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:44.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:44.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: Adjusting osd_memory_target on smithi161 to 5704M 2024-05-10T04:00:44.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:44.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:44.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:44.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:44 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:44.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T04:00:44.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T04:00:44.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:44.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: from='osd.7 [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:00:44.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:00:44.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: Detected new or changed devices on smithi161 2024-05-10T04:00:44.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:44.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:44.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:44.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:44.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:44.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: Adjusting osd_memory_target on smithi161 to 5704M 2024-05-10T04:00:44.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:44.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:44.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:44.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:44 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:44.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: from='osd.7 [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: Detected new or changed devices on smithi161 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: Adjusting osd_memory_target on smithi161 to 5704M 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:44.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:44.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:44 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:44.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.974+0000 7f2e0abba700 1 -- 172.21.15.181:0/1900694416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e04101cb0 msgr2=0x7f2e041020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.974+0000 7f2e0abba700 1 --2- 172.21.15.181:0/1900694416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e04101cb0 0x7f2e041020c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009990 tx=0x7f2df0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:00:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.974+0000 7f2e0abba700 1 -- 172.21.15.181:0/1900694416 shutdown_connections 2024-05-10T04:00:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.974+0000 7f2e0abba700 1 --2- 172.21.15.181:0/1900694416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e04102650 0x7f2e04108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.974+0000 7f2e0abba700 1 --2- 172.21.15.181:0/1900694416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e04100b80 0x7f2e04100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.974+0000 7f2e0abba700 1 --2- 172.21.15.181:0/1900694416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e04101cb0 0x7f2e041020c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.974+0000 7f2e0abba700 1 -- 172.21.15.181:0/1900694416 >> 172.21.15.181:0/1900694416 conn(0x7f2e040fc1c0 msgr2=0x7f2e040fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:00:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.974+0000 7f2e0abba700 1 -- 172.21.15.181:0/1900694416 shutdown_connections 2024-05-10T04:00:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.974+0000 7f2e0abba700 1 -- 172.21.15.181:0/1900694416 wait complete. 2024-05-10T04:00:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.975+0000 7f2e0abba700 1 Processor -- start 2024-05-10T04:00:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.975+0000 7f2e0abba700 1 -- start start 2024-05-10T04:00:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.975+0000 7f2e0abba700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e04100b80 0x7f2e04198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:00:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.975+0000 7f2e0abba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e04101cb0 0x7f2e041986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:00:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.975+0000 7f2e0abba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e04102650 0x7f2e04199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.975+0000 7f2e0abba700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e04199ab0 con 0x7f2e04101cb0 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.975+0000 7f2e0abba700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e04199bf0 con 0x7f2e04100b80 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.975+0000 7f2e0abba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e04199db0 con 0x7f2e04102650 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.975+0000 7f2e09157700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e04102650 0x7f2e04199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.975+0000 7f2e09157700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e04102650 0x7f2e04199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38672/0 (socket says 172.21.15.181:38672) 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.975+0000 7f2e09157700 1 -- 172.21.15.181:0/2185077517 learned_addr learned my addr 172.21.15.181:0/2185077517 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.975+0000 7f2e03fff700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e04101cb0 0x7f2e041986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.976+0000 7f2e08956700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e04100b80 0x7f2e04198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.976+0000 7f2e09157700 1 -- 172.21.15.181:0/2185077517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e04100b80 msgr2=0x7f2e04198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.976+0000 7f2e09157700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e04100b80 0x7f2e04198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.976+0000 7f2e09157700 1 -- 172.21.15.181:0/2185077517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e04101cb0 msgr2=0x7f2e041986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.976+0000 7f2e09157700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e04101cb0 0x7f2e041986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.976+0000 7f2e09157700 1 -- 172.21.15.181:0/2185077517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc009c40 con 0x7f2e04102650 2024-05-10T04:00:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.976+0000 7f2e09157700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e04102650 0x7f2e04199450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00c370 tx=0x7f2dfc00c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:00:44.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.976+0000 7f2e01ffb700 1 -- 172.21.15.181:0/2185077517 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc017070 con 0x7f2e04102650 2024-05-10T04:00:44.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.976+0000 7f2e01ffb700 1 -- 172.21.15.181:0/2185077517 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dfc00f040 con 0x7f2e04102650 2024-05-10T04:00:44.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.976+0000 7f2e0abba700 1 -- 172.21.15.181:0/2185077517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df00095f0 con 0x7f2e04102650 2024-05-10T04:00:44.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.976+0000 7f2e01ffb700 1 -- 172.21.15.181:0/2185077517 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc00e050 con 0x7f2e04102650 2024-05-10T04:00:44.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.976+0000 7f2e0abba700 1 -- 172.21.15.181:0/2185077517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e041a06a0 con 0x7f2e04102650 2024-05-10T04:00:44.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.977+0000 7f2e0abba700 1 -- 172.21.15.181:0/2185077517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e040ffe80 con 0x7f2e04102650 2024-05-10T04:00:44.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.978+0000 7f2e01ffb700 1 -- 172.21.15.181:0/2185077517 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2dfc00cb60 con 0x7f2e04102650 2024-05-10T04:00:44.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.979+0000 7f2e01ffb700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2df40a11a0 0x7f2df40a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:00:44.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.979+0000 7f2e01ffb700 1 -- 172.21.15.181:0/2185077517 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5117+0+0 (secure 0 0 0) 0x7f2dfc0cbc00 con 0x7f2e04102650 2024-05-10T04:00:44.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.980+0000 7f2e08956700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2df40a11a0 0x7f2df40a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:00:44.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.980+0000 7f2e08956700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2df40a11a0 0x7f2df40a36b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2e04101bb0 tx=0x7f2df8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:00:44.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:44.983+0000 7f2e01ffb700 1 -- 172.21.15.181:0/2185077517 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2dfc095ff0 con 0x7f2e04102650 2024-05-10T04:00:45.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:45.184+0000 7f2e0abba700 1 -- 172.21.15.181:0/2185077517 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e040fe2c0 con 0x7f2df40a11a0 2024-05-10T04:00:45.242 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:00:44 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:00:44.771+0000 7f18adc96700 -1 osd.7 0 waiting for initial osdmap 2024-05-10T04:00:45.242 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:00:44 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:00:44.791+0000 7f18a5244700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:00:45.570 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:45 smithi181 ceph-mon[46602]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:00:45.570 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:45 smithi181 ceph-mon[46602]: osdmap e48: 8 total, 7 up, 8 in 2024-05-10T04:00:45.570 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:45 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:45.570 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:45 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:45.570 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:45 smithi181 ceph-mon[46602]: from='osd.7 ' entity='osd.7' 2024-05-10T04:00:45.570 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:45 smithi181 ceph-mon[46602]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:45.570 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:45 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:00:45.570 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:45 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:00:45.570 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:45 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:45.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:45 smithi150 ceph-mon[42477]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:00:45.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:45 smithi150 ceph-mon[42477]: osdmap e48: 8 total, 7 up, 8 in 2024-05-10T04:00:45.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:45 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:45.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:45 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:45.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:45 smithi150 ceph-mon[42477]: from='osd.7 ' entity='osd.7' 2024-05-10T04:00:45.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:45 smithi150 ceph-mon[42477]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:45.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:45 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:00:45.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:45 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:00:45.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:45 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:45.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:45 smithi161 ceph-mon[46918]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:00:45.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:45 smithi161 ceph-mon[46918]: osdmap e48: 8 total, 7 up, 8 in 2024-05-10T04:00:45.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:45 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:45.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:45 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:45.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:45 smithi161 ceph-mon[46918]: from='osd.7 ' entity='osd.7' 2024-05-10T04:00:45.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:45 smithi161 ceph-mon[46918]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:00:45.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:45 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:00:45.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:45 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:00:45.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:45 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:46.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:46 smithi181 ceph-mon[46602]: purged_snaps scrub starts 2024-05-10T04:00:46.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:46 smithi181 ceph-mon[46602]: purged_snaps scrub ok 2024-05-10T04:00:46.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:46 smithi181 ceph-mon[46602]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:00:46.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:46 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:46.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:46 smithi181 ceph-mon[46602]: osd.7 [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649] boot 2024-05-10T04:00:46.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:46 smithi181 ceph-mon[46602]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T04:00:46.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:46 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:46.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:46 smithi150 ceph-mon[42477]: purged_snaps scrub starts 2024-05-10T04:00:46.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:46 smithi150 ceph-mon[42477]: purged_snaps scrub ok 2024-05-10T04:00:46.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:46 smithi150 ceph-mon[42477]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:00:46.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:46 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:46.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:46 smithi150 ceph-mon[42477]: osd.7 [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649] boot 2024-05-10T04:00:46.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:46 smithi150 ceph-mon[42477]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T04:00:46.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:46 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:46.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:46 smithi161 ceph-mon[46918]: purged_snaps scrub starts 2024-05-10T04:00:46.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:46 smithi161 ceph-mon[46918]: purged_snaps scrub ok 2024-05-10T04:00:46.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:46 smithi161 ceph-mon[46918]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:00:46.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:46 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:46.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:46 smithi161 ceph-mon[46918]: osd.7 [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649] boot 2024-05-10T04:00:46.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:46 smithi161 ceph-mon[46918]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T04:00:46.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:46 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:00:47.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:47 smithi150 ceph-mon[42477]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T04:00:47.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:47 smithi150 ceph-mon[42477]: from='client.? 172.21.15.181:0/415296768' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "259306a4-2c3c-4aff-9f6b-38cf3256b090"}]: dispatch 2024-05-10T04:00:47.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:47 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "259306a4-2c3c-4aff-9f6b-38cf3256b090"}]: dispatch 2024-05-10T04:00:47.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:47 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "259306a4-2c3c-4aff-9f6b-38cf3256b090"}]': finished 2024-05-10T04:00:47.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:47 smithi150 ceph-mon[42477]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T04:00:47.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:47 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:00:47.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:47 smithi150 ceph-mon[42477]: pgmap v140: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:47.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:47 smithi181 ceph-mon[46602]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T04:00:47.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:47 smithi181 ceph-mon[46602]: from='client.? 172.21.15.181:0/415296768' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "259306a4-2c3c-4aff-9f6b-38cf3256b090"}]: dispatch 2024-05-10T04:00:47.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:47 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "259306a4-2c3c-4aff-9f6b-38cf3256b090"}]: dispatch 2024-05-10T04:00:47.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:47 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "259306a4-2c3c-4aff-9f6b-38cf3256b090"}]': finished 2024-05-10T04:00:47.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:47 smithi181 ceph-mon[46602]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T04:00:47.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:47 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:00:47.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:47 smithi181 ceph-mon[46602]: pgmap v140: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:47.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:47 smithi161 ceph-mon[46918]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T04:00:47.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:47 smithi161 ceph-mon[46918]: from='client.? 172.21.15.181:0/415296768' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "259306a4-2c3c-4aff-9f6b-38cf3256b090"}]: dispatch 2024-05-10T04:00:47.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:47 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "259306a4-2c3c-4aff-9f6b-38cf3256b090"}]: dispatch 2024-05-10T04:00:47.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:47 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "259306a4-2c3c-4aff-9f6b-38cf3256b090"}]': finished 2024-05-10T04:00:47.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:47 smithi161 ceph-mon[46918]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T04:00:47.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:47 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:00:47.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:47 smithi161 ceph-mon[46918]: pgmap v140: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:48.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:48 smithi150 ceph-mon[42477]: from='client.? 172.21.15.181:0/3291491642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:00:48.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:48 smithi150 ceph-mon[42477]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T04:00:48.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:48 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:00:48.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:48 smithi181 ceph-mon[46602]: from='client.? 172.21.15.181:0/3291491642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:00:48.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:48 smithi181 ceph-mon[46602]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T04:00:48.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:48 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:00:48.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:48 smithi161 ceph-mon[46918]: from='client.? 172.21.15.181:0/3291491642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:00:48.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:48 smithi161 ceph-mon[46918]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T04:00:48.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:48 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:00:50.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:50 smithi150 ceph-mon[42477]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T04:00:50.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:50 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:00:50.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:50 smithi150 ceph-mon[42477]: pgmap v143: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:50.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:50 smithi181 ceph-mon[46602]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T04:00:50.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:50 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:00:50.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:50 smithi181 ceph-mon[46602]: pgmap v143: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:50.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:50 smithi161 ceph-mon[46918]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T04:00:50.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:50 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:00:50.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:50 smithi161 ceph-mon[46918]: pgmap v143: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:52.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:52 smithi150 ceph-mon[42477]: pgmap v144: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:52.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:52 smithi181 ceph-mon[46602]: pgmap v144: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:52.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:52 smithi161 ceph-mon[46918]: pgmap v144: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:53.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:53 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T04:00:53.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:53 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:53.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:53 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T04:00:53.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:53 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:53.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:53 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T04:00:53.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:53 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:54.441 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:54 smithi181 ceph-mon[46602]: Deploying daemon osd.8 on smithi181 2024-05-10T04:00:54.442 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:54 smithi181 ceph-mon[46602]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:54.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:54 smithi150 ceph-mon[42477]: Deploying daemon osd.8 on smithi181 2024-05-10T04:00:54.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:54 smithi150 ceph-mon[42477]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:54 smithi161 ceph-mon[46918]: Deploying daemon osd.8 on smithi181 2024-05-10T04:00:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:54 smithi161 ceph-mon[46918]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:56.377 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:56 smithi181 ceph-mon[46602]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:56.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:56 smithi150 ceph-mon[42477]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:56.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:56 smithi161 ceph-mon[46918]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:58.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:57 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:58.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:57 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:58.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:57 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:58.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:57 smithi150 ceph-mon[42477]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:58.228 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:57 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:58.228 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:57 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:58.228 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:57 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:58.228 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:57 smithi181 ceph-mon[46602]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:58.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:57 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:58.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:57 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:58.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:57 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:58.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:57 smithi161 ceph-mon[46918]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:59.241 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 8 on host 'smithi181' 2024-05-10T04:00:59.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:59.237+0000 7f2e01ffb700 1 -- 172.21.15.181:0/2185077517 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2e040fe2c0 con 0x7f2df40a11a0 2024-05-10T04:00:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:59.240+0000 7f2e0abba700 1 -- 172.21.15.181:0/2185077517 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2df40a11a0 msgr2=0x7f2df40a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:59.240+0000 7f2e0abba700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2df40a11a0 0x7f2df40a36b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2e04101bb0 tx=0x7f2df8009260 comp rx=0 tx=0).stop 2024-05-10T04:00:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:59.240+0000 7f2e0abba700 1 -- 172.21.15.181:0/2185077517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e04102650 msgr2=0x7f2e04199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:00:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:59.240+0000 7f2e0abba700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e04102650 0x7f2e04199450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00c370 tx=0x7f2dfc00c680 comp rx=0 tx=0).stop 2024-05-10T04:00:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:59.240+0000 7f2e0abba700 1 -- 172.21.15.181:0/2185077517 shutdown_connections 2024-05-10T04:00:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:59.240+0000 7f2e0abba700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2df40a11a0 0x7f2df40a36b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:59.240+0000 7f2e0abba700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e04102650 0x7f2e04199450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:59.240+0000 7f2e0abba700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e04100b80 0x7f2e04198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:59.240+0000 7f2e0abba700 1 --2- 172.21.15.181:0/2185077517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e04101cb0 0x7f2e041986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:00:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:59.240+0000 7f2e0abba700 1 -- 172.21.15.181:0/2185077517 >> 172.21.15.181:0/2185077517 conn(0x7f2e040fc1c0 msgr2=0x7f2e040fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:00:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:59.240+0000 7f2e0abba700 1 -- 172.21.15.181:0/2185077517 shutdown_connections 2024-05-10T04:00:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:00:59.240+0000 7f2e0abba700 1 -- 172.21.15.181:0/2185077517 wait complete. 2024-05-10T04:00:59.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:59.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:59.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:59 smithi181 ceph-mon[46602]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:59.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:00:59 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:59.628 DEBUG:teuthology.orchestra.run.smithi181:osd.8> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.8.service 2024-05-10T04:00:59.630 INFO:tasks.cephadm:Deploying osd.9 on smithi150 with /dev/vg_nvme/lv_1... 2024-05-10T04:00:59.631 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T04:00:59.675 INFO:journalctl@ceph.osd.8.smithi181.stdout:-- Logs begin at Fri 2024-05-10 03:40:19 UTC. -- 2024-05-10T04:00:59.677 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:59 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.677 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:59 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.677 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:59 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:59.678 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:59 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:59.678 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:59 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.678 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:59 smithi150 ceph-mon[42477]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:59.678 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:59 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.678 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:59 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.678 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:00:59 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:59.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:59 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:59 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:59 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:00:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:59 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:00:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:59 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:59 smithi161 ceph-mon[46918]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:00:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:59 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:59 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:00:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:00:59 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:00:59.875 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:01:01.224 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T04:01:01.225 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T04:01:01.225 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-10T04:01:01.225 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-10T04:01:01.225 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113511 s, 924 MB/s 2024-05-10T04:01:01.225 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-10T04:01:01.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:00 smithi181 ceph-mon[46602]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T04:01:01.240 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:01:00 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:01:00.918+0000 7f523a39b7c0 -1 osd.8 0 log_to_monitors true 2024-05-10T04:01:01.240 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:00 smithi150 ceph-mon[42477]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T04:01:01.245 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:00 smithi161 ceph-mon[46918]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T04:01:01.647 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2024-05-10T04:01:01.875 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:01:02.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: osdmap e54: 9 total, 8 up, 9 in 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:02.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:01:02.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: osdmap e55: 9 total, 8 up, 9 in 2024-05-10T04:01:02.244 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:02.326 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T04:01:02.326 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: osdmap e54: 9 total, 8 up, 9 in 2024-05-10T04:01:02.326 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:02.326 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:01:02.326 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:01:02.326 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:02.326 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:02.326 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:02.326 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:02.327 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:02.327 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:02.327 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:02.327 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:02.327 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:02.327 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:01:02.327 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: osdmap e55: 9 total, 8 up, 9 in 2024-05-10T04:01:02.327 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:02.442 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T04:01:02.442 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: osdmap e54: 9 total, 8 up, 9 in 2024-05-10T04:01:02.442 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:02.442 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:01:02.442 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:01:02.443 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:02.443 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:02.443 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:02.443 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:02.443 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:02.443 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:02.443 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:02.443 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:02.443 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:02.443 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:01:02.443 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: osdmap e55: 9 total, 8 up, 9 in 2024-05-10T04:01:02.443 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:02.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:01:02 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:01:02.438+0000 7f522f9ff700 -1 osd.8 0 waiting for initial osdmap 2024-05-10T04:01:02.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:01:02 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:01:02.453+0000 7f52277ae700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:01:02.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.879+0000 7f50b9e06700 1 -- 172.21.15.150:0/644667663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50b4107ab0 msgr2=0x7f50b4109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:02.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.879+0000 7f50b9e06700 1 --2- 172.21.15.150:0/644667663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50b4107ab0 0x7f50b4109ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00b0a0 tx=0x7f50ac00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:01:02.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.879+0000 7f50b9e06700 1 -- 172.21.15.150:0/644667663 shutdown_connections 2024-05-10T04:01:02.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.879+0000 7f50b9e06700 1 --2- 172.21.15.150:0/644667663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50b4107ab0 0x7f50b4109ed0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:02.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.879+0000 7f50b9e06700 1 --2- 172.21.15.150:0/644667663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50b40fffd0 0x7f50b41003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:02.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.879+0000 7f50b9e06700 1 --2- 172.21.15.150:0/644667663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b4105150 0x7f50b4107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:02.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.879+0000 7f50b9e06700 1 -- 172.21.15.150:0/644667663 >> 172.21.15.150:0/644667663 conn(0x7f50b40fb950 msgr2=0x7f50b40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:01:02.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.879+0000 7f50b9e06700 1 -- 172.21.15.150:0/644667663 shutdown_connections 2024-05-10T04:01:02.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.879+0000 7f50b9e06700 1 -- 172.21.15.150:0/644667663 wait complete. 2024-05-10T04:01:02.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.880+0000 7f50b9e06700 1 Processor -- start 2024-05-10T04:01:02.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.880+0000 7f50b9e06700 1 -- start start 2024-05-10T04:01:02.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.880+0000 7f50b9e06700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b40fffd0 0x7f50b41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:02.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.880+0000 7f50b9e06700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50b4105150 0x7f50b4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:02.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.880+0000 7f50b9e06700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50b4107ab0 0x7f50b41994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:02.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.880+0000 7f50b9e06700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4199b20 con 0x7f50b40fffd0 2024-05-10T04:01:02.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.880+0000 7f50b9e06700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4199c60 con 0x7f50b4107ab0 2024-05-10T04:01:02.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.880+0000 7f50b9e06700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4199e20 con 0x7f50b4105150 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.880+0000 7f50b37fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b40fffd0 0x7f50b41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.880+0000 7f50b37fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b40fffd0 0x7f50b41981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37490/0 (socket says 172.21.15.150:37490) 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.880+0000 7f50b37fe700 1 -- 172.21.15.150:0/10801182 learned_addr learned my addr 172.21.15.150:0/10801182 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.881+0000 7f50b2ffd700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50b4105150 0x7f50b4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.881+0000 7f50b3fff700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50b4107ab0 0x7f50b41994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.881+0000 7f50b37fe700 1 -- 172.21.15.150:0/10801182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50b4105150 msgr2=0x7f50b4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.881+0000 7f50b37fe700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50b4105150 0x7f50b4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.881+0000 7f50b37fe700 1 -- 172.21.15.150:0/10801182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50b4107ab0 msgr2=0x7f50b41994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.881+0000 7f50b37fe700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50b4107ab0 0x7f50b41994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.881+0000 7f50b37fe700 1 -- 172.21.15.150:0/10801182 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac009d00 con 0x7f50b40fffd0 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.881+0000 7f50b37fe700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b40fffd0 0x7f50b41981d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f50a800e970 tx=0x7f50a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.881+0000 7f50b0ff9700 1 -- 172.21.15.150:0/10801182 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8009790 con 0x7f50b40fffd0 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.881+0000 7f50b9e06700 1 -- 172.21.15.150:0/10801182 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b419a100 con 0x7f50b40fffd0 2024-05-10T04:01:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.881+0000 7f50b9e06700 1 -- 172.21.15.150:0/10801182 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b41a08e0 con 0x7f50b40fffd0 2024-05-10T04:01:02.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.881+0000 7f50b0ff9700 1 -- 172.21.15.150:0/10801182 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50a800eec0 con 0x7f50b40fffd0 2024-05-10T04:01:02.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.881+0000 7f50b0ff9700 1 -- 172.21.15.150:0/10801182 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8005560 con 0x7f50b40fffd0 2024-05-10T04:01:02.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.882+0000 7f50b9e06700 1 -- 172.21.15.150:0/10801182 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a0005320 con 0x7f50b40fffd0 2024-05-10T04:01:02.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.884+0000 7f50b0ff9700 1 -- 172.21.15.150:0/10801182 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f50a800cbd0 con 0x7f50b40fffd0 2024-05-10T04:01:02.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.885+0000 7f50b0ff9700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f509c0a1120 0x7f509c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:02.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.885+0000 7f50b2ffd700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f509c0a1120 0x7f509c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:02.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.885+0000 7f50b0ff9700 1 -- 172.21.15.150:0/10801182 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5549+0+0 (secure 0 0 0) 0x7f50a8014070 con 0x7f50b40fffd0 2024-05-10T04:01:02.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.885+0000 7f50b2ffd700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f509c0a1120 0x7f509c0a3630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f50b41991a0 tx=0x7f50a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:01:02.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:02.888+0000 7f50b0ff9700 1 -- 172.21.15.150:0/10801182 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f50a8096410 con 0x7f50b40fffd0 2024-05-10T04:01:03.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:03.098+0000 7f50b9e06700 1 -- 172.21.15.150:0/10801182 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f50a0000bf0 con 0x7f509c0a1120 2024-05-10T04:01:03.457 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:02 smithi150 ceph-mon[42477]: Detected new or changed devices on smithi181 2024-05-10T04:01:03.457 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:02 smithi150 ceph-mon[42477]: Adjusting osd_memory_target on smithi181 to 5704M 2024-05-10T04:01:03.457 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:02 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:03.457 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:02 smithi150 ceph-mon[42477]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' 2024-05-10T04:01:03.457 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:02 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:03.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:02 smithi181 ceph-mon[46602]: Detected new or changed devices on smithi181 2024-05-10T04:01:03.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:02 smithi181 ceph-mon[46602]: Adjusting osd_memory_target on smithi181 to 5704M 2024-05-10T04:01:03.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:02 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:03.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:02 smithi181 ceph-mon[46602]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' 2024-05-10T04:01:03.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:02 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:03.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:02 smithi161 ceph-mon[46918]: Detected new or changed devices on smithi181 2024-05-10T04:01:03.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:02 smithi161 ceph-mon[46918]: Adjusting osd_memory_target on smithi181 to 5704M 2024-05-10T04:01:03.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:02 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:03.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:02 smithi161 ceph-mon[46918]: from='osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813]' entity='osd.8' 2024-05-10T04:01:03.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:02 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:04.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:04 smithi150 ceph-mon[42477]: purged_snaps scrub starts 2024-05-10T04:01:04.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:04 smithi150 ceph-mon[42477]: purged_snaps scrub ok 2024-05-10T04:01:04.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:04 smithi150 ceph-mon[42477]: osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813] boot 2024-05-10T04:01:04.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:04 smithi150 ceph-mon[42477]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T04:01:04.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:04.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:04 smithi150 ceph-mon[42477]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:01:04.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:04 smithi150 ceph-mon[42477]: from='client.14451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:01:04.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:01:04.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:01:04.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:04 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:04.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:04 smithi150 ceph-mon[42477]: osdmap e57: 9 total, 9 up, 9 in 2024-05-10T04:01:04.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:04 smithi181 ceph-mon[46602]: purged_snaps scrub starts 2024-05-10T04:01:04.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:04 smithi181 ceph-mon[46602]: purged_snaps scrub ok 2024-05-10T04:01:04.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:04 smithi181 ceph-mon[46602]: osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813] boot 2024-05-10T04:01:04.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:04 smithi181 ceph-mon[46602]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T04:01:04.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:04.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:04 smithi181 ceph-mon[46602]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:01:04.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:04 smithi181 ceph-mon[46602]: from='client.14451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:01:04.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:01:04.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:01:04.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:04 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:04.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:04 smithi181 ceph-mon[46602]: osdmap e57: 9 total, 9 up, 9 in 2024-05-10T04:01:04.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:04 smithi161 ceph-mon[46918]: purged_snaps scrub starts 2024-05-10T04:01:04.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:04 smithi161 ceph-mon[46918]: purged_snaps scrub ok 2024-05-10T04:01:04.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:04 smithi161 ceph-mon[46918]: osd.8 [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813] boot 2024-05-10T04:01:04.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:04 smithi161 ceph-mon[46918]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T04:01:04.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:04 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:01:04.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:04 smithi161 ceph-mon[46918]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:01:04.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:04 smithi161 ceph-mon[46918]: from='client.14451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:01:04.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:04 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:01:04.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:04 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:01:04.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:04 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:04.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:04 smithi161 ceph-mon[46918]: osdmap e57: 9 total, 9 up, 9 in 2024-05-10T04:01:05.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:05 smithi150 ceph-mon[42477]: osdmap e58: 9 total, 9 up, 9 in 2024-05-10T04:01:05.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:05 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3355111093' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db540925-be5f-4601-8cb9-392f6b547d68"}]: dispatch 2024-05-10T04:01:05.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:05 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3355111093' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db540925-be5f-4601-8cb9-392f6b547d68"}]': finished 2024-05-10T04:01:05.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:05 smithi150 ceph-mon[42477]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T04:01:05.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:05 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:05.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:05 smithi150 ceph-mon[42477]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:01:05.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:05 smithi181 ceph-mon[46602]: osdmap e58: 9 total, 9 up, 9 in 2024-05-10T04:01:05.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:05 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3355111093' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db540925-be5f-4601-8cb9-392f6b547d68"}]: dispatch 2024-05-10T04:01:05.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:05 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3355111093' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db540925-be5f-4601-8cb9-392f6b547d68"}]': finished 2024-05-10T04:01:05.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:05 smithi181 ceph-mon[46602]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T04:01:05.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:05 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:05.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:05 smithi181 ceph-mon[46602]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:01:05.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:05 smithi161 ceph-mon[46918]: osdmap e58: 9 total, 9 up, 9 in 2024-05-10T04:01:05.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:05 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3355111093' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db540925-be5f-4601-8cb9-392f6b547d68"}]: dispatch 2024-05-10T04:01:05.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:05 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3355111093' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db540925-be5f-4601-8cb9-392f6b547d68"}]': finished 2024-05-10T04:01:05.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:05 smithi161 ceph-mon[46918]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T04:01:05.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:05 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:05.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:05 smithi161 ceph-mon[46918]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:01:06.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:06 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1118372975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:01:06.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:06 smithi150 ceph-mon[42477]: osdmap e60: 10 total, 9 up, 10 in 2024-05-10T04:01:06.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:06 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:06.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:06 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1118372975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:01:06.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:06 smithi181 ceph-mon[46602]: osdmap e60: 10 total, 9 up, 10 in 2024-05-10T04:01:06.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:06 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:06.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:06 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1118372975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:01:06.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:06 smithi161 ceph-mon[46918]: osdmap e60: 10 total, 9 up, 10 in 2024-05-10T04:01:06.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:06 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:08.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:07 smithi150 ceph-mon[42477]: osdmap e61: 10 total, 9 up, 10 in 2024-05-10T04:01:08.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:07 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:08.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:07 smithi150 ceph-mon[42477]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-05-10T04:01:08.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:07 smithi181 ceph-mon[46602]: osdmap e61: 10 total, 9 up, 10 in 2024-05-10T04:01:08.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:07 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:08.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:07 smithi181 ceph-mon[46602]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-05-10T04:01:08.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:07 smithi161 ceph-mon[46918]: osdmap e61: 10 total, 9 up, 10 in 2024-05-10T04:01:08.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:08.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:07 smithi161 ceph-mon[46918]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-05-10T04:01:10.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:10 smithi150 ceph-mon[42477]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-10T04:01:10.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:10 smithi181 ceph-mon[46602]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-10T04:01:10.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:10 smithi161 ceph-mon[46918]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-10T04:01:11.349 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:11 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T04:01:11.349 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:11 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:11 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T04:01:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:11 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:11 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T04:01:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:11 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:12 smithi150 ceph-mon[42477]: Deploying daemon osd.9 on smithi150 2024-05-10T04:01:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:12 smithi150 ceph-mon[42477]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-10T04:01:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:12 smithi181 ceph-mon[46602]: Deploying daemon osd.9 on smithi150 2024-05-10T04:01:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:12 smithi181 ceph-mon[46602]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-10T04:01:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:12 smithi161 ceph-mon[46918]: Deploying daemon osd.9 on smithi150 2024-05-10T04:01:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:12 smithi161 ceph-mon[46918]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-10T04:01:14.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:14 smithi150 ceph-mon[42477]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T04:01:14.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:14 smithi181 ceph-mon[46602]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T04:01:14.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:14 smithi161 ceph-mon[46918]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T04:01:15.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:15.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:15.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:15.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:15 smithi150 ceph-mon[42477]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-10T04:01:15.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:15.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:15.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:15.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:15 smithi181 ceph-mon[46602]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-10T04:01:15.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:15.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:15.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:15.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:15 smithi161 ceph-mon[46918]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-10T04:01:16.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:16.547+0000 7f50b0ff9700 1 -- 172.21.15.150:0/10801182 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f50a0000bf0 con 0x7f509c0a1120 2024-05-10T04:01:16.550 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 9 on host 'smithi150' 2024-05-10T04:01:16.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:16.549+0000 7f50b9e06700 1 -- 172.21.15.150:0/10801182 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f509c0a1120 msgr2=0x7f509c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:16.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:16.549+0000 7f50b9e06700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f509c0a1120 0x7f509c0a3630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f50b41991a0 tx=0x7f50a4009260 comp rx=0 tx=0).stop 2024-05-10T04:01:16.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:16.549+0000 7f50b9e06700 1 -- 172.21.15.150:0/10801182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b40fffd0 msgr2=0x7f50b41981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:16.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:16.549+0000 7f50b9e06700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b40fffd0 0x7f50b41981d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f50a800e970 tx=0x7f50a800ec80 comp rx=0 tx=0).stop 2024-05-10T04:01:16.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:16.549+0000 7f50b9e06700 1 -- 172.21.15.150:0/10801182 shutdown_connections 2024-05-10T04:01:16.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:16.549+0000 7f50b9e06700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f509c0a1120 0x7f509c0a3630 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:16.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:16.549+0000 7f50b9e06700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50b4107ab0 0x7f50b41994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:16.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:16.549+0000 7f50b9e06700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b40fffd0 0x7f50b41981d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:16.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:16.549+0000 7f50b9e06700 1 --2- 172.21.15.150:0/10801182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50b4105150 0x7f50b4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:16.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:16.549+0000 7f50b9e06700 1 -- 172.21.15.150:0/10801182 >> 172.21.15.150:0/10801182 conn(0x7f50b40fb950 msgr2=0x7f50b41060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:01:16.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:16.549+0000 7f50b9e06700 1 -- 172.21.15.150:0/10801182 shutdown_connections 2024-05-10T04:01:16.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:16.549+0000 7f50b9e06700 1 -- 172.21.15.150:0/10801182 wait complete. 2024-05-10T04:01:17.070 DEBUG:teuthology.orchestra.run.smithi150:osd.9> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.9.service 2024-05-10T04:01:17.074 INFO:tasks.cephadm:Deploying osd.10 on smithi161 with /dev/vg_nvme/lv_1... 2024-05-10T04:01:17.074 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T04:01:17.106 INFO:journalctl@ceph.osd.9.smithi150.stdout:-- Logs begin at Fri 2024-05-10 03:40:30 UTC. -- 2024-05-10T04:01:17.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.121 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.122 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:17.122 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:17.122 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.122 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:17.122 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.122 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.156 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.156 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.156 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:17.156 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:17.156 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.156 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:17.156 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.156 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:17.319 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:01:18.196 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:17 smithi161 ceph-mon[46918]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:01:18.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:17 smithi150 ceph-mon[42477]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:01:18.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:17 smithi181 ceph-mon[46602]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:01:18.713 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:01:18 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:01:18.347+0000 7f12ca1177c0 -1 osd.9 0 log_to_monitors true 2024-05-10T04:01:18.732 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T04:01:18.732 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T04:01:18.732 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-05-10T04:01:18.732 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-05-10T04:01:18.732 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014704 s, 713 MB/s 2024-05-10T04:01:18.732 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-05-10T04:01:19.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:18 smithi161 ceph-mon[46918]: from='osd.9 [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T04:01:19.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:18 smithi161 ceph-mon[46918]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T04:01:19.158 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-05-10T04:01:19.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:18 smithi150 ceph-mon[42477]: from='osd.9 [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T04:01:19.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:18 smithi150 ceph-mon[42477]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T04:01:19.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:18 smithi181 ceph-mon[46602]: from='osd.9 [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T04:01:19.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:18 smithi181 ceph-mon[46602]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T04:01:19.423 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:01:20.205 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T04:01:20.205 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: osdmap e62: 10 total, 9 up, 10 in 2024-05-10T04:01:20.205 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:20.205 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='osd.9 [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:01:20.205 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:01:20.205 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:01:20.205 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: Detected new or changed devices on smithi150 2024-05-10T04:01:20.205 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:20.205 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:20.205 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:20.205 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:20.206 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:20.206 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:20.206 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:20.206 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:20.206 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:20.206 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:19 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:20.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T04:01:20.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: osdmap e62: 10 total, 9 up, 10 in 2024-05-10T04:01:20.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:20.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='osd.9 [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:01:20.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:01:20.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:01:20.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: Detected new or changed devices on smithi150 2024-05-10T04:01:20.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:20.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:20.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:20.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:20.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:20.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:20.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:20.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:20.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:20.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:19 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:20.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T04:01:20.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: osdmap e62: 10 total, 9 up, 10 in 2024-05-10T04:01:20.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:20.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='osd.9 [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:01:20.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:01:20.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:01:20.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: Detected new or changed devices on smithi150 2024-05-10T04:01:20.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:20.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:20.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:20.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:20.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:20.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:20.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:20.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:20.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:20.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:19 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:20.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.423+0000 7f40f52ea700 1 -- 172.21.15.161:0/346406983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40f0102640 msgr2=0x7f40f0109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:20.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.423+0000 7f40f52ea700 1 --2- 172.21.15.161:0/346406983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40f0102640 0x7f40f0109110 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f40e800b0a0 tx=0x7f40e800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:01:20.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.423+0000 7f40f52ea700 1 -- 172.21.15.161:0/346406983 shutdown_connections 2024-05-10T04:01:20.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.423+0000 7f40f52ea700 1 --2- 172.21.15.161:0/346406983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40f0102640 0x7f40f0109110 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:20.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.423+0000 7f40f52ea700 1 --2- 172.21.15.161:0/346406983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f0101140 0x7f40f0101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:20.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.423+0000 7f40f52ea700 1 --2- 172.21.15.161:0/346406983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40f0101b20 0x7f40f0101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:20.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.423+0000 7f40f52ea700 1 -- 172.21.15.161:0/346406983 >> 172.21.15.161:0/346406983 conn(0x7f40f00fb970 msgr2=0x7f40f00fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:01:20.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.423+0000 7f40f52ea700 1 -- 172.21.15.161:0/346406983 shutdown_connections 2024-05-10T04:01:20.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.424+0000 7f40f52ea700 1 -- 172.21.15.161:0/346406983 wait complete. 2024-05-10T04:01:20.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.424+0000 7f40f52ea700 1 Processor -- start 2024-05-10T04:01:20.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.424+0000 7f40f52ea700 1 -- start start 2024-05-10T04:01:20.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.424+0000 7f40f52ea700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40f0101140 0x7f40f0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:20.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40f52ea700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40f0101b20 0x7f40f0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40f52ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f0102640 0x7f40f01994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40f52ea700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0199b50 con 0x7f40f0101140 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40f52ea700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0199c90 con 0x7f40f0101b20 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40f52ea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0199e50 con 0x7f40f0102640 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40ee7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40f0101b20 0x7f40f0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40ee7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40f0101b20 0x7f40f0198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39424/0 (socket says 172.21.15.161:39424) 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40ee7fc700 1 -- 172.21.15.161:0/954065094 learned_addr learned my addr 172.21.15.161:0/954065094 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40eeffd700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40f0101140 0x7f40f0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40ef7fe700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f0102640 0x7f40f01994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40ee7fc700 1 -- 172.21.15.161:0/954065094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f0102640 msgr2=0x7f40f01994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40ee7fc700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f0102640 0x7f40f01994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40ee7fc700 1 -- 172.21.15.161:0/954065094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40f0101140 msgr2=0x7f40f0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40ee7fc700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40f0101140 0x7f40f0198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:20.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40ee7fc700 1 -- 172.21.15.161:0/954065094 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e8009d00 con 0x7f40f0101b20 2024-05-10T04:01:20.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40eeffd700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40f0101140 0x7f40f0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:01:20.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40ee7fc700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40f0101b20 0x7f40f0198740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f40e000e9a0 tx=0x7f40e000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:01:20.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40e7fff700 1 -- 172.21.15.161:0/954065094 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0009790 con 0x7f40f0101b20 2024-05-10T04:01:20.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40f52ea700 1 -- 172.21.15.161:0/954065094 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f019a130 con 0x7f40f0101b20 2024-05-10T04:01:20.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40e7fff700 1 -- 172.21.15.161:0/954065094 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40e0010430 con 0x7f40f0101b20 2024-05-10T04:01:20.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40e7fff700 1 -- 172.21.15.161:0/954065094 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0018680 con 0x7f40f0101b20 2024-05-10T04:01:20.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.425+0000 7f40f52ea700 1 -- 172.21.15.161:0/954065094 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f01a0910 con 0x7f40f0101b20 2024-05-10T04:01:20.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.426+0000 7f40f52ea700 1 -- 172.21.15.161:0/954065094 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f00ff630 con 0x7f40f0101b20 2024-05-10T04:01:20.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.427+0000 7f40e7fff700 1 -- 172.21.15.161:0/954065094 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f40e00105a0 con 0x7f40f0101b20 2024-05-10T04:01:20.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.428+0000 7f40e7fff700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f40dc0a1120 0x7f40dc0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:20.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.428+0000 7f40e7fff700 1 -- 172.21.15.161:0/954065094 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5981+0+0 (secure 0 0 0) 0x7f40e0014070 con 0x7f40f0101b20 2024-05-10T04:01:20.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.428+0000 7f40eeffd700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f40dc0a1120 0x7f40dc0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:20.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.429+0000 7f40eeffd700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f40dc0a1120 0x7f40dc0a3630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f40f0069a50 tx=0x7f40d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:01:20.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.432+0000 7f40e7fff700 1 -- 172.21.15.161:0/954065094 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f40e0096400 con 0x7f40f0101b20 2024-05-10T04:01:20.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:20.633+0000 7f40f52ea700 1 -- 172.21.15.161:0/954065094 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f40f00fd960 con 0x7f40dc0a1120 2024-05-10T04:01:20.713 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:01:20 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:01:20.385+0000 7f12bff7c700 -1 osd.9 0 waiting for initial osdmap 2024-05-10T04:01:20.714 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:01:20 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:01:20.402+0000 7f12b752a700 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:01:21.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:20 smithi161 ceph-mon[46918]: Adjusting osd_memory_target on smithi150 to 4278M 2024-05-10T04:01:21.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:20 smithi161 ceph-mon[46918]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:01:21.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:20 smithi161 ceph-mon[46918]: osdmap e63: 10 total, 9 up, 10 in 2024-05-10T04:01:21.148 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:21.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:21.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:20 smithi161 ceph-mon[46918]: from='osd.9 ' entity='osd.9' 2024-05-10T04:01:21.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:01:21.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:01:21.149 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:21.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:20 smithi150 ceph-mon[42477]: Adjusting osd_memory_target on smithi150 to 4278M 2024-05-10T04:01:21.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:20 smithi150 ceph-mon[42477]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:01:21.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:20 smithi150 ceph-mon[42477]: osdmap e63: 10 total, 9 up, 10 in 2024-05-10T04:01:21.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:21.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:21.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:20 smithi150 ceph-mon[42477]: from='osd.9 ' entity='osd.9' 2024-05-10T04:01:21.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:01:21.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:01:21.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:21.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:20 smithi181 ceph-mon[46602]: Adjusting osd_memory_target on smithi150 to 4278M 2024-05-10T04:01:21.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:20 smithi181 ceph-mon[46602]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:01:21.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:20 smithi181 ceph-mon[46602]: osdmap e63: 10 total, 9 up, 10 in 2024-05-10T04:01:21.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:21.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:21.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:20 smithi181 ceph-mon[46602]: from='osd.9 ' entity='osd.9' 2024-05-10T04:01:21.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:01:21.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:01:21.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:22.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:21 smithi150 ceph-mon[42477]: purged_snaps scrub starts 2024-05-10T04:01:22.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:21 smithi150 ceph-mon[42477]: purged_snaps scrub ok 2024-05-10T04:01:22.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:21 smithi150 ceph-mon[42477]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:01:22.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:22.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:21 smithi150 ceph-mon[42477]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:01:22.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:21 smithi150 ceph-mon[42477]: osd.9 [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934] boot 2024-05-10T04:01:22.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:21 smithi150 ceph-mon[42477]: osdmap e64: 10 total, 10 up, 10 in 2024-05-10T04:01:22.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:21 smithi181 ceph-mon[46602]: purged_snaps scrub starts 2024-05-10T04:01:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:21 smithi181 ceph-mon[46602]: purged_snaps scrub ok 2024-05-10T04:01:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:21 smithi181 ceph-mon[46602]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:01:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:21 smithi181 ceph-mon[46602]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:01:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:21 smithi181 ceph-mon[46602]: osd.9 [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934] boot 2024-05-10T04:01:22.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:21 smithi181 ceph-mon[46602]: osdmap e64: 10 total, 10 up, 10 in 2024-05-10T04:01:22.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:22.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:21 smithi161 ceph-mon[46918]: purged_snaps scrub starts 2024-05-10T04:01:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:21 smithi161 ceph-mon[46918]: purged_snaps scrub ok 2024-05-10T04:01:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:21 smithi161 ceph-mon[46918]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:01:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:21 smithi161 ceph-mon[46918]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:01:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:21 smithi161 ceph-mon[46918]: osd.9 [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934] boot 2024-05-10T04:01:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:21 smithi161 ceph-mon[46918]: osdmap e64: 10 total, 10 up, 10 in 2024-05-10T04:01:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:01:23.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:22 smithi150 ceph-mon[42477]: from='client.? 172.21.15.161:0/2275162909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3281319-74d6-4de1-ba77-6671f6d2d5fc"}]: dispatch 2024-05-10T04:01:23.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:22 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3281319-74d6-4de1-ba77-6671f6d2d5fc"}]: dispatch 2024-05-10T04:01:23.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:22 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3281319-74d6-4de1-ba77-6671f6d2d5fc"}]': finished 2024-05-10T04:01:23.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:22 smithi150 ceph-mon[42477]: osdmap e65: 11 total, 10 up, 11 in 2024-05-10T04:01:23.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:22 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:23.237 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:22 smithi181 ceph-mon[46602]: from='client.? 172.21.15.161:0/2275162909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3281319-74d6-4de1-ba77-6671f6d2d5fc"}]: dispatch 2024-05-10T04:01:23.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:22 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3281319-74d6-4de1-ba77-6671f6d2d5fc"}]: dispatch 2024-05-10T04:01:23.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:22 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3281319-74d6-4de1-ba77-6671f6d2d5fc"}]': finished 2024-05-10T04:01:23.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:22 smithi181 ceph-mon[46602]: osdmap e65: 11 total, 10 up, 11 in 2024-05-10T04:01:23.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:22 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:23.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:22 smithi161 ceph-mon[46918]: from='client.? 172.21.15.161:0/2275162909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3281319-74d6-4de1-ba77-6671f6d2d5fc"}]: dispatch 2024-05-10T04:01:23.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:22 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3281319-74d6-4de1-ba77-6671f6d2d5fc"}]: dispatch 2024-05-10T04:01:23.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:22 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3281319-74d6-4de1-ba77-6671f6d2d5fc"}]': finished 2024-05-10T04:01:23.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:22 smithi161 ceph-mon[46918]: osdmap e65: 11 total, 10 up, 11 in 2024-05-10T04:01:23.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:22 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:24.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:23 smithi150 ceph-mon[42477]: from='client.? 172.21.15.161:0/928254357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:01:24.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:23 smithi150 ceph-mon[42477]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:24.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:23 smithi181 ceph-mon[46602]: from='client.? 172.21.15.161:0/928254357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:01:24.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:23 smithi181 ceph-mon[46602]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:24.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:23 smithi161 ceph-mon[46918]: from='client.? 172.21.15.161:0/928254357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:01:24.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:23 smithi161 ceph-mon[46918]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:26.431 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:26 smithi161 ceph-mon[46918]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:26.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:26 smithi150 ceph-mon[42477]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:26.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:26 smithi181 ceph-mon[46602]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:28.213 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:28 smithi161 ceph-mon[46918]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:28.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:28 smithi150 ceph-mon[42477]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:28 smithi181 ceph-mon[46602]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:29.428 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T04:01:29.428 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:29.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T04:01:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T04:01:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:30.394 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:30 smithi161 ceph-mon[46918]: Deploying daemon osd.10 on smithi161 2024-05-10T04:01:30.394 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:30 smithi161 ceph-mon[46918]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:30 smithi150 ceph-mon[42477]: Deploying daemon osd.10 on smithi161 2024-05-10T04:01:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:30 smithi150 ceph-mon[42477]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:30 smithi181 ceph-mon[46602]: Deploying daemon osd.10 on smithi161 2024-05-10T04:01:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:30 smithi181 ceph-mon[46602]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:32 smithi150 ceph-mon[42477]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:32 smithi181 ceph-mon[46602]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:32 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:32 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:32 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:32 smithi161 ceph-mon[46918]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:32 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:32 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:32 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:34.229 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:34 smithi161 ceph-mon[46918]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:34.229 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:34 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:34.230 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:34 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:34.230 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:34 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:34.230 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:34 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:34.230 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:34 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:34.337 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 10 on host 'smithi161' 2024-05-10T04:01:34.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:34.334+0000 7f40e7fff700 1 -- 172.21.15.161:0/954065094 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f40f00fd960 con 0x7f40dc0a1120 2024-05-10T04:01:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:34.337+0000 7f40f52ea700 1 -- 172.21.15.161:0/954065094 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f40dc0a1120 msgr2=0x7f40dc0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:34.337+0000 7f40f52ea700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f40dc0a1120 0x7f40dc0a3630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f40f0069a50 tx=0x7f40d8009260 comp rx=0 tx=0).stop 2024-05-10T04:01:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:34.337+0000 7f40f52ea700 1 -- 172.21.15.161:0/954065094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40f0101b20 msgr2=0x7f40f0198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:34.337+0000 7f40f52ea700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40f0101b20 0x7f40f0198740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f40e000e9a0 tx=0x7f40e000ecb0 comp rx=0 tx=0).stop 2024-05-10T04:01:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:34.337+0000 7f40f52ea700 1 -- 172.21.15.161:0/954065094 shutdown_connections 2024-05-10T04:01:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:34.337+0000 7f40f52ea700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f40dc0a1120 0x7f40dc0a3630 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:34.337+0000 7f40f52ea700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f0102640 0x7f40f01994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:34.337+0000 7f40f52ea700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40f0101140 0x7f40f0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:34.337+0000 7f40f52ea700 1 --2- 172.21.15.161:0/954065094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40f0101b20 0x7f40f0198740 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:34.337+0000 7f40f52ea700 1 -- 172.21.15.161:0/954065094 >> 172.21.15.161:0/954065094 conn(0x7f40f00fb970 msgr2=0x7f40f00fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:01:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:34.337+0000 7f40f52ea700 1 -- 172.21.15.161:0/954065094 shutdown_connections 2024-05-10T04:01:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:01:34.337+0000 7f40f52ea700 1 -- 172.21.15.161:0/954065094 wait complete. 2024-05-10T04:01:34.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:34 smithi150 ceph-mon[42477]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:34.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:34 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:34.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:34 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:34.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:34 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:34.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:34 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:34.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:34 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:34 smithi181 ceph-mon[46602]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:34 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:34 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:34 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:34 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:34 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:34.705 DEBUG:teuthology.orchestra.run.smithi161:osd.10> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.10.service 2024-05-10T04:01:34.708 INFO:tasks.cephadm:Deploying osd.11 on smithi181 with /dev/vg_nvme/lv_1... 2024-05-10T04:01:34.708 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T04:01:34.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:-- Logs begin at Fri 2024-05-10 03:40:24 UTC. -- 2024-05-10T04:01:34.944 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:01:35.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:35 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:35.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:35 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:35.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:35 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:35.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:35 smithi161 ceph-mon[46918]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:35.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:35 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:35.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:35 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:35.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:35 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:35.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:35 smithi150 ceph-mon[42477]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:35.722 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:35 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:35.722 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:35 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:35.722 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:35 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:35.722 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:35 smithi181 ceph-mon[46602]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:36.238 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T04:01:36.239 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T04:01:36.239 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-05-10T04:01:36.239 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-05-10T04:01:36.239 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112676 s, 931 MB/s 2024-05-10T04:01:36.239 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-05-10T04:01:36.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:01:36 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:01:36.095+0000 7fbf75aac7c0 -1 osd.10 0 log_to_monitors true 2024-05-10T04:01:36.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:36 smithi161 ceph-mon[46918]: from='osd.10 [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T04:01:36.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:36 smithi161 ceph-mon[46918]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T04:01:36.540 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:36 smithi181 ceph-mon[46602]: from='osd.10 [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T04:01:36.541 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:36 smithi181 ceph-mon[46602]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T04:01:36.594 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_1 2024-05-10T04:01:36.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:36 smithi150 ceph-mon[42477]: from='osd.10 [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T04:01:36.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:36 smithi150 ceph-mon[42477]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T04:01:36.816 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:01:37.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T04:01:37.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: osdmap e66: 11 total, 10 up, 11 in 2024-05-10T04:01:37.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:37.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='osd.10 [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:01:37.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:01:37.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: Detected new or changed devices on smithi161 2024-05-10T04:01:37.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:37.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:37.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:37.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:37.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:37.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:37.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: Adjusting osd_memory_target on smithi161 to 4278M 2024-05-10T04:01:37.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:37.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:37.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:37.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:37.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:37 smithi181 ceph-mon[46602]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T04:01:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: osdmap e66: 11 total, 10 up, 11 in 2024-05-10T04:01:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='osd.10 [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:01:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:01:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: Detected new or changed devices on smithi161 2024-05-10T04:01:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:37.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:37.716 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:37.716 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: Adjusting osd_memory_target on smithi161 to 4278M 2024-05-10T04:01:37.716 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:37.716 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:37.716 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:37.716 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:37.716 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:37 smithi150 ceph-mon[42477]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:37.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T04:01:37.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: osdmap e66: 11 total, 10 up, 11 in 2024-05-10T04:01:37.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:37.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='osd.10 [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:01:37.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:01:37.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: Detected new or changed devices on smithi161 2024-05-10T04:01:37.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:37.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:37.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:37.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:37.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:37.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:37.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: Adjusting osd_memory_target on smithi161 to 4278M 2024-05-10T04:01:37.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:37.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:37.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:37.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:37.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:37 smithi161 ceph-mon[46918]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.792+0000 7fe634609700 1 -- 172.21.15.181:0/3534942173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c102650 msgr2=0x7fe62c108f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.792+0000 7fe634609700 1 --2- 172.21.15.181:0/3534942173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c102650 0x7fe62c108f00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe62400b0a0 tx=0x7fe62400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:01:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.792+0000 7fe634609700 1 -- 172.21.15.181:0/3534942173 shutdown_connections 2024-05-10T04:01:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.792+0000 7fe634609700 1 --2- 172.21.15.181:0/3534942173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c102650 0x7fe62c108f00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.792+0000 7fe634609700 1 --2- 172.21.15.181:0/3534942173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe62c100b80 0x7fe62c100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.792+0000 7fe634609700 1 --2- 172.21.15.181:0/3534942173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe62c101cb0 0x7fe62c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.792+0000 7fe634609700 1 -- 172.21.15.181:0/3534942173 >> 172.21.15.181:0/3534942173 conn(0x7fe62c0fc1c0 msgr2=0x7fe62c0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:01:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.792+0000 7fe634609700 1 -- 172.21.15.181:0/3534942173 shutdown_connections 2024-05-10T04:01:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.792+0000 7fe634609700 1 -- 172.21.15.181:0/3534942173 wait complete. 2024-05-10T04:01:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.792+0000 7fe634609700 1 Processor -- start 2024-05-10T04:01:37.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.792+0000 7fe634609700 1 -- start start 2024-05-10T04:01:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe634609700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe62c100b80 0x7fe62c104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe634609700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c101cb0 0x7fe62c102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe634609700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe62c102650 0x7fe62c103650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe634609700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c103d80 con 0x7fe62c102650 2024-05-10T04:01:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe634609700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c1a21f0 con 0x7fe62c100b80 2024-05-10T04:01:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe634609700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c1a2330 con 0x7fe62c101cb0 2024-05-10T04:01:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe631ba4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c101cb0 0x7fe62c102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe631ba4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c101cb0 0x7fe62c102ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46802/0 (socket says 172.21.15.181:46802) 2024-05-10T04:01:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe631ba4700 1 -- 172.21.15.181:0/2799588360 learned_addr learned my addr 172.21.15.181:0/2799588360 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T04:01:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe6323a5700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe62c100b80 0x7fe62c104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe632ba6700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe62c102650 0x7fe62c103650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe631ba4700 1 -- 172.21.15.181:0/2799588360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe62c100b80 msgr2=0x7fe62c104b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:37.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe631ba4700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe62c100b80 0x7fe62c104b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe631ba4700 1 -- 172.21.15.181:0/2799588360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe62c102650 msgr2=0x7fe62c103650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe631ba4700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe62c102650 0x7fe62c103650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe631ba4700 1 -- 172.21.15.181:0/2799588360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe624009d00 con 0x7fe62c101cb0 2024-05-10T04:01:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe632ba6700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe62c102650 0x7fe62c103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:01:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe631ba4700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c101cb0 0x7fe62c102ff0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00d980 tx=0x7fe61c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:01:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe6237fe700 1 -- 172.21.15.181:0/2799588360 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c004220 con 0x7fe62c101cb0 2024-05-10T04:01:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe6237fe700 1 -- 172.21.15.181:0/2799588360 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe61c007640 con 0x7fe62c101cb0 2024-05-10T04:01:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe634609700 1 -- 172.21.15.181:0/2799588360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62c1a2590 con 0x7fe62c101cb0 2024-05-10T04:01:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.793+0000 7fe6237fe700 1 -- 172.21.15.181:0/2799588360 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c005680 con 0x7fe62c101cb0 2024-05-10T04:01:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.794+0000 7fe634609700 1 -- 172.21.15.181:0/2799588360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62c1a2a90 con 0x7fe62c101cb0 2024-05-10T04:01:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.794+0000 7fe634609700 1 -- 172.21.15.181:0/2799588360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62c0ffe80 con 0x7fe62c101cb0 2024-05-10T04:01:37.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.796+0000 7fe6237fe700 1 -- 172.21.15.181:0/2799588360 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe61c004380 con 0x7fe62c101cb0 2024-05-10T04:01:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.796+0000 7fe6237fe700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe6180a1120 0x7fe6180a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.797+0000 7fe6323a5700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe6180a1120 0x7fe6180a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.797+0000 7fe6237fe700 1 -- 172.21.15.181:0/2799588360 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe61c0cc3d0 con 0x7fe62c101cb0 2024-05-10T04:01:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.797+0000 7fe6323a5700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe6180a1120 0x7fe6180a3630 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe62c104860 tx=0x7fe62800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:01:37.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:37.800+0000 7fe6237fe700 1 -- 172.21.15.181:0/2799588360 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe61c0962b0 con 0x7fe62c101cb0 2024-05-10T04:01:38.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:38.003+0000 7fe634609700 1 -- 172.21.15.181:0/2799588360 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe62c0fe240 con 0x7fe6180a1120 2024-05-10T04:01:38.242 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:01:37 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:01:37.818+0000 7fbf6b110700 -1 osd.10 0 waiting for initial osdmap 2024-05-10T04:01:38.242 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:01:37 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:01:37.837+0000 7fbf62ebf700 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:01:38.678 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:38 smithi181 ceph-mon[46602]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:01:38.678 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:38 smithi181 ceph-mon[46602]: osdmap e67: 11 total, 10 up, 11 in 2024-05-10T04:01:38.678 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:38 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:38.678 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:38 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:38.679 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:38 smithi181 ceph-mon[46602]: from='osd.10 ' entity='osd.10' 2024-05-10T04:01:38.679 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:38 smithi181 ceph-mon[46602]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:01:38.679 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:38 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:01:38.679 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:38 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:01:38.679 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:38 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:38.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:38 smithi150 ceph-mon[42477]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:01:38.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:38 smithi150 ceph-mon[42477]: osdmap e67: 11 total, 10 up, 11 in 2024-05-10T04:01:38.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:38 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:38.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:38 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:38.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:38 smithi150 ceph-mon[42477]: from='osd.10 ' entity='osd.10' 2024-05-10T04:01:38.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:38 smithi150 ceph-mon[42477]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:01:38.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:38 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:01:38.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:38 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:01:38.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:38 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:38.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:38 smithi161 ceph-mon[46918]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:01:38.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:38 smithi161 ceph-mon[46918]: osdmap e67: 11 total, 10 up, 11 in 2024-05-10T04:01:38.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:38 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:38.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:38 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:38.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:38 smithi161 ceph-mon[46918]: from='osd.10 ' entity='osd.10' 2024-05-10T04:01:38.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:38 smithi161 ceph-mon[46918]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:01:38.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:38 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:01:38.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:38 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:01:38.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:38 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:39.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:39 smithi150 ceph-mon[42477]: purged_snaps scrub starts 2024-05-10T04:01:39.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:39 smithi150 ceph-mon[42477]: purged_snaps scrub ok 2024-05-10T04:01:39.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:39 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:39.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:39 smithi150 ceph-mon[42477]: osd.10 [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933] boot 2024-05-10T04:01:39.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:39 smithi150 ceph-mon[42477]: osdmap e68: 11 total, 11 up, 11 in 2024-05-10T04:01:39.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:39 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:39.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:39 smithi150 ceph-mon[42477]: osdmap e69: 11 total, 11 up, 11 in 2024-05-10T04:01:39.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:39 smithi150 ceph-mon[42477]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:39.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:39 smithi181 ceph-mon[46602]: purged_snaps scrub starts 2024-05-10T04:01:39.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:39 smithi181 ceph-mon[46602]: purged_snaps scrub ok 2024-05-10T04:01:39.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:39 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:39.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:39 smithi181 ceph-mon[46602]: osd.10 [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933] boot 2024-05-10T04:01:39.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:39 smithi181 ceph-mon[46602]: osdmap e68: 11 total, 11 up, 11 in 2024-05-10T04:01:39.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:39 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:39.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:39 smithi181 ceph-mon[46602]: osdmap e69: 11 total, 11 up, 11 in 2024-05-10T04:01:39.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:39 smithi181 ceph-mon[46602]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:39.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:39 smithi161 ceph-mon[46918]: purged_snaps scrub starts 2024-05-10T04:01:39.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:39 smithi161 ceph-mon[46918]: purged_snaps scrub ok 2024-05-10T04:01:39.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:39 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:39.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:39 smithi161 ceph-mon[46918]: osd.10 [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933] boot 2024-05-10T04:01:39.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:39 smithi161 ceph-mon[46918]: osdmap e68: 11 total, 11 up, 11 in 2024-05-10T04:01:39.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:39 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:01:39.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:39 smithi161 ceph-mon[46918]: osdmap e69: 11 total, 11 up, 11 in 2024-05-10T04:01:39.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:39 smithi161 ceph-mon[46918]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:01:40.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:40 smithi150 ceph-mon[42477]: osdmap e70: 11 total, 11 up, 11 in 2024-05-10T04:01:40.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:40 smithi150 ceph-mon[42477]: from='client.? 172.21.15.181:0/1998872091' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a60f21a5-86c6-43b9-aea1-fa8517d04b1d"}]: dispatch 2024-05-10T04:01:40.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:40 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a60f21a5-86c6-43b9-aea1-fa8517d04b1d"}]: dispatch 2024-05-10T04:01:40.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:40 smithi150 ceph-mon[42477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a60f21a5-86c6-43b9-aea1-fa8517d04b1d"}]': finished 2024-05-10T04:01:40.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:40 smithi150 ceph-mon[42477]: osdmap e71: 12 total, 11 up, 12 in 2024-05-10T04:01:40.715 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:40 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:40.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:40 smithi181 ceph-mon[46602]: osdmap e70: 11 total, 11 up, 11 in 2024-05-10T04:01:40.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:40 smithi181 ceph-mon[46602]: from='client.? 172.21.15.181:0/1998872091' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a60f21a5-86c6-43b9-aea1-fa8517d04b1d"}]: dispatch 2024-05-10T04:01:40.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:40 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a60f21a5-86c6-43b9-aea1-fa8517d04b1d"}]: dispatch 2024-05-10T04:01:40.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:40 smithi181 ceph-mon[46602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a60f21a5-86c6-43b9-aea1-fa8517d04b1d"}]': finished 2024-05-10T04:01:40.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:40 smithi181 ceph-mon[46602]: osdmap e71: 12 total, 11 up, 12 in 2024-05-10T04:01:40.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:40 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:40.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:40 smithi161 ceph-mon[46918]: osdmap e70: 11 total, 11 up, 11 in 2024-05-10T04:01:40.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:40 smithi161 ceph-mon[46918]: from='client.? 172.21.15.181:0/1998872091' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a60f21a5-86c6-43b9-aea1-fa8517d04b1d"}]: dispatch 2024-05-10T04:01:40.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:40 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a60f21a5-86c6-43b9-aea1-fa8517d04b1d"}]: dispatch 2024-05-10T04:01:40.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:40 smithi161 ceph-mon[46918]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a60f21a5-86c6-43b9-aea1-fa8517d04b1d"}]': finished 2024-05-10T04:01:40.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:40 smithi161 ceph-mon[46918]: osdmap e71: 12 total, 11 up, 12 in 2024-05-10T04:01:40.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:40 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:41.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:41 smithi150 ceph-mon[42477]: from='client.? 172.21.15.181:0/3602675115' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:01:41.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:41 smithi150 ceph-mon[42477]: osdmap e72: 12 total, 11 up, 12 in 2024-05-10T04:01:41.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:41 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:41.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:41 smithi150 ceph-mon[42477]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:01:41.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:41 smithi181 ceph-mon[46602]: from='client.? 172.21.15.181:0/3602675115' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:01:41.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:41 smithi181 ceph-mon[46602]: osdmap e72: 12 total, 11 up, 12 in 2024-05-10T04:01:41.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:41 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:41.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:41 smithi181 ceph-mon[46602]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:01:41.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:41 smithi161 ceph-mon[46918]: from='client.? 172.21.15.181:0/3602675115' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:01:41.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:41 smithi161 ceph-mon[46918]: osdmap e72: 12 total, 11 up, 12 in 2024-05-10T04:01:41.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:41 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:41.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:41 smithi161 ceph-mon[46918]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:01:43.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:42 smithi150 ceph-mon[42477]: osdmap e73: 12 total, 11 up, 12 in 2024-05-10T04:01:43.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:42 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:43.237 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:42 smithi181 ceph-mon[46602]: osdmap e73: 12 total, 11 up, 12 in 2024-05-10T04:01:43.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:42 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:43.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:42 smithi161 ceph-mon[46918]: osdmap e73: 12 total, 11 up, 12 in 2024-05-10T04:01:43.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:42 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:43.923 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:43 smithi181 ceph-mon[46602]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T04:01:44.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:43 smithi150 ceph-mon[42477]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T04:01:44.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:43 smithi161 ceph-mon[46918]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T04:01:46.375 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:46 smithi181 ceph-mon[46602]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-10T04:01:46.375 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:46 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T04:01:46.375 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:46 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:46.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:46 smithi150 ceph-mon[42477]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-10T04:01:46.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:46 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T04:01:46.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:46 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:46.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:46 smithi161 ceph-mon[46918]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-10T04:01:46.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:46 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T04:01:46.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:46 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:47.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:47 smithi181 ceph-mon[46602]: Deploying daemon osd.11 on smithi181 2024-05-10T04:01:47.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:47 smithi150 ceph-mon[42477]: Deploying daemon osd.11 on smithi181 2024-05-10T04:01:47.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:47 smithi161 ceph-mon[46918]: Deploying daemon osd.11 on smithi181 2024-05-10T04:01:48.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:48 smithi181 ceph-mon[46602]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-10T04:01:48.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:48 smithi150 ceph-mon[42477]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-10T04:01:48.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:48 smithi161 ceph-mon[46918]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-10T04:01:50.296 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:50 smithi181 ceph-mon[46602]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-10T04:01:50.296 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:50 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:50.296 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:50 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:50.296 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:50 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:50.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:50 smithi150 ceph-mon[42477]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-10T04:01:50.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:50 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:50.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:50 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:50.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:50 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:50.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:50 smithi161 ceph-mon[46918]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-10T04:01:50.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:50 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:50.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:50 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:50.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:50 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:52.053+0000 7fe6237fe700 1 -- 172.21.15.181:0/2799588360 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fe62c0fe240 con 0x7fe6180a1120 2024-05-10T04:01:52.056 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 11 on host 'smithi181' 2024-05-10T04:01:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:52.056+0000 7fe634609700 1 -- 172.21.15.181:0/2799588360 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe6180a1120 msgr2=0x7fe6180a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:52.056+0000 7fe634609700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe6180a1120 0x7fe6180a3630 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe62c104860 tx=0x7fe62800b410 comp rx=0 tx=0).stop 2024-05-10T04:01:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:52.056+0000 7fe634609700 1 -- 172.21.15.181:0/2799588360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c101cb0 msgr2=0x7fe62c102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:52.056+0000 7fe634609700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c101cb0 0x7fe62c102ff0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00d980 tx=0x7fe61c00dc90 comp rx=0 tx=0).stop 2024-05-10T04:01:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:52.056+0000 7fe634609700 1 -- 172.21.15.181:0/2799588360 shutdown_connections 2024-05-10T04:01:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:52.056+0000 7fe634609700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe6180a1120 0x7fe6180a3630 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:52.056+0000 7fe634609700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe62c102650 0x7fe62c103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:52.056+0000 7fe634609700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe62c100b80 0x7fe62c104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:52.056+0000 7fe634609700 1 --2- 172.21.15.181:0/2799588360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c101cb0 0x7fe62c102ff0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:52.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:52.056+0000 7fe634609700 1 -- 172.21.15.181:0/2799588360 >> 172.21.15.181:0/2799588360 conn(0x7fe62c0fc1c0 msgr2=0x7fe62c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:01:52.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:52.056+0000 7fe634609700 1 -- 172.21.15.181:0/2799588360 shutdown_connections 2024-05-10T04:01:52.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:01:52.056+0000 7fe634609700 1 -- 172.21.15.181:0/2799588360 wait complete. 2024-05-10T04:01:52.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:52 smithi181 ceph-mon[46602]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:01:52.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:52 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:52 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:52 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:52.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:52 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:52.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:52 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:52 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:52 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:52 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:52.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:52 smithi150 ceph-mon[42477]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:01:52.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:52.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:52.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:52 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:52.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:52 smithi161 ceph-mon[46918]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:01:52.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:52 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:52 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:52 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:52.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:52 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:52.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:52 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:52 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:52 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:52.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:52 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:01:52.506 DEBUG:teuthology.orchestra.run.smithi181:osd.11> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.11.service 2024-05-10T04:01:52.509 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-10T04:01:52.509 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-10T04:01:52.539 INFO:journalctl@ceph.osd.11.smithi181.stdout:-- Logs begin at Fri 2024-05-10 03:40:19 UTC. -- 2024-05-10T04:01:52.746 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:01:53.350 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:53 smithi181 ceph-mon[46602]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:01:53.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:53 smithi150 ceph-mon[42477]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:01:53.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:53 smithi161 ceph-mon[46918]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:01:53.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.653+0000 7fa7f6e46700 1 -- 172.21.15.150:0/685427308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7f0102620 msgr2=0x7fa7f0109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:53.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.653+0000 7fa7f6e46700 1 --2- 172.21.15.150:0/685427308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7f0102620 0x7fa7f0109110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800b0a0 tx=0x7fa7e800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:01:53.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.653+0000 7fa7f6e46700 1 -- 172.21.15.150:0/685427308 shutdown_connections 2024-05-10T04:01:53.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.653+0000 7fa7f6e46700 1 --2- 172.21.15.150:0/685427308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7f0102620 0x7fa7f0109110 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:53.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.653+0000 7fa7f6e46700 1 --2- 172.21.15.150:0/685427308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f0101140 0x7fa7f0101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:53.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.653+0000 7fa7f6e46700 1 --2- 172.21.15.150:0/685427308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7f0101b20 0x7fa7f0101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:53.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.653+0000 7fa7f6e46700 1 -- 172.21.15.150:0/685427308 >> 172.21.15.150:0/685427308 conn(0x7fa7f00fb950 msgr2=0x7fa7f00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:01:53.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.653+0000 7fa7f6e46700 1 -- 172.21.15.150:0/685427308 shutdown_connections 2024-05-10T04:01:53.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.653+0000 7fa7f6e46700 1 -- 172.21.15.150:0/685427308 wait complete. 2024-05-10T04:01:53.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.653+0000 7fa7f6e46700 1 Processor -- start 2024-05-10T04:01:53.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.653+0000 7fa7f6e46700 1 -- start start 2024-05-10T04:01:53.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7f6e46700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7f0101140 0x7fa7f01980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:53.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7f6e46700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7f0101b20 0x7fa7f0198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:53.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7f6e46700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f0102620 0x7fa7f01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:53.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7f6e46700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f0199a40 con 0x7fa7f0101b20 2024-05-10T04:01:53.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7f6e46700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f0199b80 con 0x7fa7f0102620 2024-05-10T04:01:53.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7f6e46700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f0199d40 con 0x7fa7f0101140 2024-05-10T04:01:53.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7effff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7f0101b20 0x7fa7f0198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:53.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7effff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7f0101b20 0x7fa7f0198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46852/0 (socket says 172.21.15.150:46852) 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7effff700 1 -- 172.21.15.150:0/2715462074 learned_addr learned my addr 172.21.15.150:0/2715462074 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7f4be2700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7f0101140 0x7fa7f01980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7f53e3700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f0102620 0x7fa7f01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7effff700 1 -- 172.21.15.150:0/2715462074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7f0101140 msgr2=0x7fa7f01980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7effff700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7f0101140 0x7fa7f01980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7effff700 1 -- 172.21.15.150:0/2715462074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f0102620 msgr2=0x7fa7f01993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7effff700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f0102620 0x7fa7f01993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7effff700 1 -- 172.21.15.150:0/2715462074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e8009d00 con 0x7fa7f0101b20 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7effff700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7f0101b20 0x7fa7f0198630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00d980 tx=0x7fa7dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7edffb700 1 -- 172.21.15.150:0/2715462074 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc004220 con 0x7fa7f0101b20 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7edffb700 1 -- 172.21.15.150:0/2715462074 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7dc007640 con 0x7fa7f0101b20 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7f6e46700 1 -- 172.21.15.150:0/2715462074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f019a020 con 0x7fa7f0101b20 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.654+0000 7fa7edffb700 1 -- 172.21.15.150:0/2715462074 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc005610 con 0x7fa7f0101b20 2024-05-10T04:01:53.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.655+0000 7fa7f6e46700 1 -- 172.21.15.150:0/2715462074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f01a0800 con 0x7fa7f0101b20 2024-05-10T04:01:53.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.656+0000 7fa7f6e46700 1 -- 172.21.15.150:0/2715462074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f00ff610 con 0x7fa7f0101b20 2024-05-10T04:01:53.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.657+0000 7fa7edffb700 1 -- 172.21.15.150:0/2715462074 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa7dc004750 con 0x7fa7f0101b20 2024-05-10T04:01:53.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.657+0000 7fa7edffb700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa7e00a1120 0x7fa7e00a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:53.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.657+0000 7fa7f4be2700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa7e00a1120 0x7fa7e00a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:53.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.657+0000 7fa7edffb700 1 -- 172.21.15.150:0/2715462074 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa7dc0cc1e0 con 0x7fa7f0101b20 2024-05-10T04:01:53.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.658+0000 7fa7f4be2700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa7e00a1120 0x7fa7e00a3630 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0068a20 tx=0x7fa7e4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:01:53.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.662+0000 7fa7edffb700 1 -- 172.21.15.150:0/2715462074 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa7dc095f50 con 0x7fa7f0101b20 2024-05-10T04:01:53.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.854+0000 7fa7f6e46700 1 -- 172.21.15.150:0/2715462074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fa7f004edc0 con 0x7fa7f0101b20 2024-05-10T04:01:53.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.855+0000 7fa7edffb700 1 -- 172.21.15.150:0/2715462074 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) v1 ==== 74+0+133 (secure 0 0 0) 0x7fa7dc099570 con 0x7fa7f0101b20 2024-05-10T04:01:53.857 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:01:53.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.857+0000 7fa7f6e46700 1 -- 172.21.15.150:0/2715462074 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa7e00a1120 msgr2=0x7fa7e00a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:53.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.857+0000 7fa7f6e46700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa7e00a1120 0x7fa7e00a3630 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0068a20 tx=0x7fa7e4009190 comp rx=0 tx=0).stop 2024-05-10T04:01:53.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.857+0000 7fa7f6e46700 1 -- 172.21.15.150:0/2715462074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7f0101b20 msgr2=0x7fa7f0198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:53.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.857+0000 7fa7f6e46700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7f0101b20 0x7fa7f0198630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00d980 tx=0x7fa7dc00dc90 comp rx=0 tx=0).stop 2024-05-10T04:01:53.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.857+0000 7fa7f6e46700 1 -- 172.21.15.150:0/2715462074 shutdown_connections 2024-05-10T04:01:53.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.857+0000 7fa7f6e46700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa7e00a1120 0x7fa7e00a3630 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:53.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.857+0000 7fa7f6e46700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f0102620 0x7fa7f01993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:53.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.857+0000 7fa7f6e46700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7f0101140 0x7fa7f01980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:53.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.857+0000 7fa7f6e46700 1 --2- 172.21.15.150:0/2715462074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7f0101b20 0x7fa7f0198630 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:53.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.857+0000 7fa7f6e46700 1 -- 172.21.15.150:0/2715462074 >> 172.21.15.150:0/2715462074 conn(0x7fa7f00fb950 msgr2=0x7fa7f00fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:01:53.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.857+0000 7fa7f6e46700 1 -- 172.21.15.150:0/2715462074 shutdown_connections 2024-05-10T04:01:53.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:53.857+0000 7fa7f6e46700 1 -- 172.21.15.150:0/2715462074 wait complete. 2024-05-10T04:01:53.988 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:01:53 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:01:53.644+0000 7f1a38d6e7c0 -1 osd.11 0 log_to_monitors true 2024-05-10T04:01:54.258 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1715313698,"num_in_osds":12,"osd_in_since":1715313699,"num_remapped_pgs":0} 2024-05-10T04:01:54.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:54 smithi181 ceph-mon[46602]: from='osd.11 [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T04:01:54.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:54 smithi181 ceph-mon[46602]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T04:01:54.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:54 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2715462074' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:01:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:54 smithi161 ceph-mon[46918]: from='osd.11 [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T04:01:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:54 smithi161 ceph-mon[46918]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T04:01:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:54 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2715462074' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:01:54.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:54 smithi150 ceph-mon[42477]: from='osd.11 [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T04:01:54.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:54 smithi150 ceph-mon[42477]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T04:01:54.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:54 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2715462074' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:01:55.259 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-10T04:01:55.483 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:01:55.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T04:01:55.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: osdmap e74: 12 total, 11 up, 12 in 2024-05-10T04:01:55.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:55.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='osd.11 [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:01:55.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:01:55.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: Detected new or changed devices on smithi181 2024-05-10T04:01:55.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:55.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:55.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:55.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:55.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:55.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:55.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: Adjusting osd_memory_target on smithi181 to 4278M 2024-05-10T04:01:55.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:55.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:55.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:55.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:55.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:55 smithi161 ceph-mon[46918]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:01:55.584 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T04:01:55.584 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: osdmap e74: 12 total, 11 up, 12 in 2024-05-10T04:01:55.584 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:55.584 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='osd.11 [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:01:55.585 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:01:55.585 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: Detected new or changed devices on smithi181 2024-05-10T04:01:55.585 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:55.585 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:55.585 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:55.585 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:55.585 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:55.585 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:55.585 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: Adjusting osd_memory_target on smithi181 to 4278M 2024-05-10T04:01:55.585 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:55.585 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:55.585 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:55.585 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:55.586 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:55 smithi150 ceph-mon[42477]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:01:55.686 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T04:01:55.686 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: osdmap e74: 12 total, 11 up, 12 in 2024-05-10T04:01:55.686 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:55.686 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='osd.11 [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:01:55.686 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:01:55.686 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: Detected new or changed devices on smithi181 2024-05-10T04:01:55.686 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:55.686 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:55.686 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:55.687 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:55.687 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:55.687 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:01:55.687 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: Adjusting osd_memory_target on smithi181 to 4278M 2024-05-10T04:01:55.687 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:55.687 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:01:55.687 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:01:55.687 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:01:55.687 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:55 smithi181 ceph-mon[46602]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:01:55.988 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:01:55 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:01:55.682+0000 7f1a2e3d2700 -1 osd.11 0 waiting for initial osdmap 2024-05-10T04:01:55.988 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:01:55 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:01:55.699+0000 7f1a26181700 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:01:56.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.411+0000 7f825f398700 1 -- 172.21.15.150:0/1781876032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8258101cb0 msgr2=0x7f82581020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.411+0000 7f825f398700 1 --2- 172.21.15.150:0/1781876032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8258101cb0 0x7f82581020c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8254009990 tx=0x7f8254009ca0 comp rx=0 tx=0).stop 2024-05-10T04:01:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.411+0000 7f825f398700 1 -- 172.21.15.150:0/1781876032 shutdown_connections 2024-05-10T04:01:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.411+0000 7f825f398700 1 --2- 172.21.15.150:0/1781876032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8258102650 0x7f8258108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.411+0000 7f825f398700 1 --2- 172.21.15.150:0/1781876032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8258100b80 0x7f8258100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.411+0000 7f825f398700 1 --2- 172.21.15.150:0/1781876032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8258101cb0 0x7f82581020c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.411+0000 7f825f398700 1 -- 172.21.15.150:0/1781876032 >> 172.21.15.150:0/1781876032 conn(0x7f82580fc1c0 msgr2=0x7f82580fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:01:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.411+0000 7f825f398700 1 -- 172.21.15.150:0/1781876032 shutdown_connections 2024-05-10T04:01:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.411+0000 7f825f398700 1 -- 172.21.15.150:0/1781876032 wait complete. 2024-05-10T04:01:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.412+0000 7f825f398700 1 Processor -- start 2024-05-10T04:01:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.412+0000 7f825f398700 1 -- start start 2024-05-10T04:01:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.412+0000 7f825f398700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8258100b80 0x7f8258104ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.412+0000 7f825f398700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8258101cb0 0x7f8258102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.412+0000 7f825f398700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8258102650 0x7f8258103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.412+0000 7f825f398700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8258103da0 con 0x7f8258102650 2024-05-10T04:01:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.412+0000 7f825f398700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82581a2160 con 0x7f8258101cb0 2024-05-10T04:01:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.412+0000 7f825f398700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82581a22a0 con 0x7f8258100b80 2024-05-10T04:01:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.412+0000 7f825d935700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8258102650 0x7f8258103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.412+0000 7f825d935700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8258102650 0x7f8258103620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46876/0 (socket says 172.21.15.150:46876) 2024-05-10T04:01:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.412+0000 7f825d935700 1 -- 172.21.15.150:0/962283453 learned_addr learned my addr 172.21.15.150:0/962283453 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:01:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.412+0000 7f825d134700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8258100b80 0x7f8258104ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.412+0000 7f825c933700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8258101cb0 0x7f8258102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.413+0000 7f825d935700 1 -- 172.21.15.150:0/962283453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8258100b80 msgr2=0x7f8258104ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.413+0000 7f825d935700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8258100b80 0x7f8258104ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.413+0000 7f825d935700 1 -- 172.21.15.150:0/962283453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8258101cb0 msgr2=0x7f8258102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.413+0000 7f825d935700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8258101cb0 0x7f8258102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.413+0000 7f825d935700 1 -- 172.21.15.150:0/962283453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82540095f0 con 0x7f8258102650 2024-05-10T04:01:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.413+0000 7f825d134700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8258100b80 0x7f8258104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:01:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.413+0000 7f825d935700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8258102650 0x7f8258103620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f825000ada0 tx=0x7f825000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:01:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.413+0000 7f824e7fc700 1 -- 172.21.15.150:0/962283453 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000f070 con 0x7f8258102650 2024-05-10T04:01:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.413+0000 7f824e7fc700 1 -- 172.21.15.150:0/962283453 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8250007b30 con 0x7f8258102650 2024-05-10T04:01:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.413+0000 7f824e7fc700 1 -- 172.21.15.150:0/962283453 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500053e0 con 0x7f8258102650 2024-05-10T04:01:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.413+0000 7f825f398700 1 -- 172.21.15.150:0/962283453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82581a24d0 con 0x7f8258102650 2024-05-10T04:01:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.413+0000 7f825f398700 1 -- 172.21.15.150:0/962283453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82581a2980 con 0x7f8258102650 2024-05-10T04:01:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.414+0000 7f825f398700 1 -- 172.21.15.150:0/962283453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82580ffe80 con 0x7f8258102650 2024-05-10T04:01:56.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.416+0000 7f824e7fc700 1 -- 172.21.15.150:0/962283453 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f825000c980 con 0x7f8258102650 2024-05-10T04:01:56.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.417+0000 7f824e7fc700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82440a11a0 0x7f82440a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:56.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.417+0000 7f825d134700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82440a11a0 0x7f82440a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:56.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.417+0000 7f824e7fc700 1 -- 172.21.15.150:0/962283453 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f82500cbf30 con 0x7f8258102650 2024-05-10T04:01:56.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.417+0000 7f825d134700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82440a11a0 0x7f82440a36b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8258104810 tx=0x7f8248009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:01:56.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.420+0000 7f824e7fc700 1 -- 172.21.15.150:0/962283453 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f82500f9d60 con 0x7f8258102650 2024-05-10T04:01:56.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.614+0000 7f825f398700 1 -- 172.21.15.150:0/962283453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f825804edc0 con 0x7f8258102650 2024-05-10T04:01:56.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.615+0000 7f824e7fc700 1 -- 172.21.15.150:0/962283453 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f8250099700 con 0x7f8258102650 2024-05-10T04:01:56.618 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:01:56.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.617+0000 7f825f398700 1 -- 172.21.15.150:0/962283453 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82440a11a0 msgr2=0x7f82440a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:56.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.617+0000 7f825f398700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82440a11a0 0x7f82440a36b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8258104810 tx=0x7f8248009260 comp rx=0 tx=0).stop 2024-05-10T04:01:56.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.617+0000 7f825f398700 1 -- 172.21.15.150:0/962283453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8258102650 msgr2=0x7f8258103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:56.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.617+0000 7f825f398700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8258102650 0x7f8258103620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f825000ada0 tx=0x7f825000c330 comp rx=0 tx=0).stop 2024-05-10T04:01:56.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.617+0000 7f825f398700 1 -- 172.21.15.150:0/962283453 shutdown_connections 2024-05-10T04:01:56.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.617+0000 7f825f398700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82440a11a0 0x7f82440a36b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.617+0000 7f825f398700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8258102650 0x7f8258103620 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.617+0000 7f825f398700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8258100b80 0x7f8258104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.617+0000 7f825f398700 1 --2- 172.21.15.150:0/962283453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8258101cb0 0x7f8258102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.617+0000 7f825f398700 1 -- 172.21.15.150:0/962283453 >> 172.21.15.150:0/962283453 conn(0x7f82580fc1c0 msgr2=0x7f82580fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:01:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.617+0000 7f825f398700 1 -- 172.21.15.150:0/962283453 shutdown_connections 2024-05-10T04:01:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:56.617+0000 7f825f398700 1 -- 172.21.15.150:0/962283453 wait complete. 2024-05-10T04:01:56.640 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:56 smithi150 ceph-mon[42477]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:01:56.640 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:56 smithi150 ceph-mon[42477]: osdmap e75: 12 total, 11 up, 12 in 2024-05-10T04:01:56.640 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:56 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:56.640 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:56 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:56.641 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:56 smithi150 ceph-mon[42477]: from='osd.11 ' entity='osd.11' 2024-05-10T04:01:56.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:56 smithi181 ceph-mon[46602]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:01:56.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:56 smithi181 ceph-mon[46602]: osdmap e75: 12 total, 11 up, 12 in 2024-05-10T04:01:56.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:56 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:56.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:56 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:56.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:56 smithi181 ceph-mon[46602]: from='osd.11 ' entity='osd.11' 2024-05-10T04:01:56.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:56 smithi161 ceph-mon[46918]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:01:56.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:56 smithi161 ceph-mon[46918]: osdmap e75: 12 total, 11 up, 12 in 2024-05-10T04:01:56.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:56 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:56.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:56 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:56.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:56 smithi161 ceph-mon[46918]: from='osd.11 ' entity='osd.11' 2024-05-10T04:01:56.981 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1715313698,"num_in_osds":12,"osd_in_since":1715313699,"num_remapped_pgs":0} 2024-05-10T04:01:57.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:57 smithi150 ceph-mon[42477]: purged_snaps scrub starts 2024-05-10T04:01:57.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:57 smithi150 ceph-mon[42477]: purged_snaps scrub ok 2024-05-10T04:01:57.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:57 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:57.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:57 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/962283453' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:01:57.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:57 smithi150 ceph-mon[42477]: osd.11 [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921] boot 2024-05-10T04:01:57.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:57 smithi150 ceph-mon[42477]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T04:01:57.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:57 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:57.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:57 smithi150 ceph-mon[42477]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:01:57.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:57 smithi181 ceph-mon[46602]: purged_snaps scrub starts 2024-05-10T04:01:57.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:57 smithi181 ceph-mon[46602]: purged_snaps scrub ok 2024-05-10T04:01:57.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:57 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:57.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:57 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/962283453' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:01:57.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:57 smithi181 ceph-mon[46602]: osd.11 [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921] boot 2024-05-10T04:01:57.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:57 smithi181 ceph-mon[46602]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T04:01:57.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:57 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:57.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:57 smithi181 ceph-mon[46602]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:01:57.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:57 smithi161 ceph-mon[46918]: purged_snaps scrub starts 2024-05-10T04:01:57.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:57 smithi161 ceph-mon[46918]: purged_snaps scrub ok 2024-05-10T04:01:57.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:57 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:57.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:57 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/962283453' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:01:57.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:57 smithi161 ceph-mon[46918]: osd.11 [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921] boot 2024-05-10T04:01:57.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:57 smithi161 ceph-mon[46918]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T04:01:57.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:57 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:01:57.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:57 smithi161 ceph-mon[46918]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:01:57.983 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-10T04:01:58.208 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:01:59.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.153+0000 7f8d78ddf700 1 -- 172.21.15.150:0/1978240736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d74101cb0 msgr2=0x7f8d741020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.153+0000 7f8d78ddf700 1 --2- 172.21.15.150:0/1978240736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d74101cb0 0x7f8d741020c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009990 tx=0x7f8d68009ca0 comp rx=0 tx=0).stop 2024-05-10T04:01:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.153+0000 7f8d78ddf700 1 -- 172.21.15.150:0/1978240736 shutdown_connections 2024-05-10T04:01:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.153+0000 7f8d78ddf700 1 --2- 172.21.15.150:0/1978240736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d74102650 0x7f8d74108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.153+0000 7f8d78ddf700 1 --2- 172.21.15.150:0/1978240736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d74100b80 0x7f8d74100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.153+0000 7f8d78ddf700 1 --2- 172.21.15.150:0/1978240736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d74101cb0 0x7f8d741020c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.153+0000 7f8d78ddf700 1 -- 172.21.15.150:0/1978240736 >> 172.21.15.150:0/1978240736 conn(0x7f8d740fc1c0 msgr2=0x7f8d740fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:01:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.153+0000 7f8d78ddf700 1 -- 172.21.15.150:0/1978240736 shutdown_connections 2024-05-10T04:01:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.153+0000 7f8d78ddf700 1 -- 172.21.15.150:0/1978240736 wait complete. 2024-05-10T04:01:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.154+0000 7f8d78ddf700 1 Processor -- start 2024-05-10T04:01:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.154+0000 7f8d78ddf700 1 -- start start 2024-05-10T04:01:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.154+0000 7f8d78ddf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d74100b80 0x7f8d7419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.154+0000 7f8d78ddf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d74101cb0 0x7f8d7419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.154+0000 7f8d78ddf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d74102650 0x7f8d7419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:59.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.154+0000 7f8d78ddf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7419fd90 con 0x7f8d74100b80 2024-05-10T04:01:59.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.154+0000 7f8d78ddf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7419fed0 con 0x7f8d74101cb0 2024-05-10T04:01:59.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.154+0000 7f8d78ddf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d741a0010 con 0x7f8d74102650 2024-05-10T04:01:59.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.154+0000 7f8d7259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d74100b80 0x7f8d7419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:59.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.154+0000 7f8d7259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d74100b80 0x7f8d7419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46894/0 (socket says 172.21.15.150:46894) 2024-05-10T04:01:59.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.154+0000 7f8d7259c700 1 -- 172.21.15.150:0/892026937 learned_addr learned my addr 172.21.15.150:0/892026937 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:01:59.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.154+0000 7f8d71d9b700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d74101cb0 0x7f8d7419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:59.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.154+0000 7f8d72d9d700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d74102650 0x7f8d7419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:59.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.155+0000 7f8d7259c700 1 -- 172.21.15.150:0/892026937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d74102650 msgr2=0x7f8d7419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:59.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.155+0000 7f8d7259c700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d74102650 0x7f8d7419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:59.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.155+0000 7f8d7259c700 1 -- 172.21.15.150:0/892026937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d74101cb0 msgr2=0x7f8d7419d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:59.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.155+0000 7f8d7259c700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d74101cb0 0x7f8d7419d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:59.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.155+0000 7f8d7259c700 1 -- 172.21.15.150:0/892026937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d680095f0 con 0x7f8d74100b80 2024-05-10T04:01:59.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.155+0000 7f8d7259c700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d74100b80 0x7f8d7419adb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400e940 tx=0x7f8d6400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:01:59.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.155+0000 7f8d637fe700 1 -- 172.21.15.150:0/892026937 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400cb70 con 0x7f8d74100b80 2024-05-10T04:01:59.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.155+0000 7f8d78ddf700 1 -- 172.21.15.150:0/892026937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d741a02a0 con 0x7f8d74100b80 2024-05-10T04:01:59.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.155+0000 7f8d637fe700 1 -- 172.21.15.150:0/892026937 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d6400ee90 con 0x7f8d74100b80 2024-05-10T04:01:59.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.155+0000 7f8d637fe700 1 -- 172.21.15.150:0/892026937 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64003cb0 con 0x7f8d74100b80 2024-05-10T04:01:59.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.155+0000 7f8d78ddf700 1 -- 172.21.15.150:0/892026937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d741a07f0 con 0x7f8d74100b80 2024-05-10T04:01:59.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.156+0000 7f8d78ddf700 1 -- 172.21.15.150:0/892026937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d74107da0 con 0x7f8d74100b80 2024-05-10T04:01:59.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.157+0000 7f8d637fe700 1 -- 172.21.15.150:0/892026937 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8d6400ccd0 con 0x7f8d74100b80 2024-05-10T04:01:59.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.158+0000 7f8d637fe700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8d5c0a1120 0x7f8d5c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:01:59.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.158+0000 7f8d71d9b700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8d5c0a1120 0x7f8d5c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:01:59.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.158+0000 7f8d637fe700 1 -- 172.21.15.150:0/892026937 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8d64014070 con 0x7f8d74100b80 2024-05-10T04:01:59.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.158+0000 7f8d71d9b700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8d5c0a1120 0x7f8d5c0a3630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800b2e0 tx=0x7f8d6801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:01:59.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.161+0000 7f8d637fe700 1 -- 172.21.15.150:0/892026937 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8d640963f0 con 0x7f8d74100b80 2024-05-10T04:01:59.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.357+0000 7f8d78ddf700 1 -- 172.21.15.150:0/892026937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f8d740ffe80 con 0x7f8d74100b80 2024-05-10T04:01:59.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.357+0000 7f8d637fe700 1 -- 172.21.15.150:0/892026937 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f8d64099a10 con 0x7f8d74100b80 2024-05-10T04:01:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:01:59.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.359+0000 7f8d78ddf700 1 -- 172.21.15.150:0/892026937 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8d5c0a1120 msgr2=0x7f8d5c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:59.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.359+0000 7f8d78ddf700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8d5c0a1120 0x7f8d5c0a3630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800b2e0 tx=0x7f8d6801a040 comp rx=0 tx=0).stop 2024-05-10T04:01:59.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.359+0000 7f8d78ddf700 1 -- 172.21.15.150:0/892026937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d74100b80 msgr2=0x7f8d7419adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:01:59.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.359+0000 7f8d78ddf700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d74100b80 0x7f8d7419adb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400e940 tx=0x7f8d6400ec50 comp rx=0 tx=0).stop 2024-05-10T04:01:59.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.359+0000 7f8d78ddf700 1 -- 172.21.15.150:0/892026937 shutdown_connections 2024-05-10T04:01:59.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.359+0000 7f8d78ddf700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8d5c0a1120 0x7f8d5c0a3630 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:59.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.359+0000 7f8d78ddf700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d74102650 0x7f8d7419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:59.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.359+0000 7f8d78ddf700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d74100b80 0x7f8d7419adb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:59.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.359+0000 7f8d78ddf700 1 --2- 172.21.15.150:0/892026937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d74101cb0 0x7f8d7419d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:01:59.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.359+0000 7f8d78ddf700 1 -- 172.21.15.150:0/892026937 >> 172.21.15.150:0/892026937 conn(0x7f8d740fc1c0 msgr2=0x7f8d74104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:01:59.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.359+0000 7f8d78ddf700 1 -- 172.21.15.150:0/892026937 shutdown_connections 2024-05-10T04:01:59.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:01:59.359+0000 7f8d78ddf700 1 -- 172.21.15.150:0/892026937 wait complete. 2024-05-10T04:01:59.549 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:59 smithi150 ceph-mon[42477]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T04:01:59.549 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:01:59 smithi150 ceph-mon[42477]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:01:59.731 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1715313716,"num_in_osds":12,"osd_in_since":1715313699,"num_remapped_pgs":0} 2024-05-10T04:01:59.732 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T04:01:59.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:59 smithi181 ceph-mon[46602]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T04:01:59.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:01:59 smithi181 ceph-mon[46602]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:01:59.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:59 smithi161 ceph-mon[46918]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T04:01:59.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:01:59 smithi161 ceph-mon[46918]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:01:59.954 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:02:00.352 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:00 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/892026937' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:02:00.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:00 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/892026937' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:02:00.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:00 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/892026937' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:02:00.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.885+0000 7f3857bb5700 1 -- 172.21.15.150:0/4104552240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3850103800 msgr2=0x7f385010bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:00.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.885+0000 7f3857bb5700 1 --2- 172.21.15.150:0/4104552240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3850103800 0x7f385010bd50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3840009990 tx=0x7f3840009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:00.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.885+0000 7f3857bb5700 1 -- 172.21.15.150:0/4104552240 shutdown_connections 2024-05-10T04:02:00.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.885+0000 7f3857bb5700 1 --2- 172.21.15.150:0/4104552240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f385010c290 0x7f385010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:00.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.885+0000 7f3857bb5700 1 --2- 172.21.15.150:0/4104552240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3850102eb0 0x7f38501032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:00.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.885+0000 7f3857bb5700 1 --2- 172.21.15.150:0/4104552240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3850103800 0x7f385010bd50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:00.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.885+0000 7f3857bb5700 1 -- 172.21.15.150:0/4104552240 >> 172.21.15.150:0/4104552240 conn(0x7f38500fc7a0 msgr2=0x7f38500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:00.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.885+0000 7f3857bb5700 1 -- 172.21.15.150:0/4104552240 shutdown_connections 2024-05-10T04:02:00.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.885+0000 7f3857bb5700 1 -- 172.21.15.150:0/4104552240 wait complete. 2024-05-10T04:02:00.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.886+0000 7f3857bb5700 1 Processor -- start 2024-05-10T04:02:00.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.886+0000 7f3857bb5700 1 -- start start 2024-05-10T04:02:00.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.886+0000 7f3857bb5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3850102eb0 0x7f3850198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:00.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.886+0000 7f3857bb5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3850103800 0x7f3850198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:00.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.886+0000 7f3857bb5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f385010c290 0x7f385019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:00.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.886+0000 7f3857bb5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385019d470 con 0x7f3850102eb0 2024-05-10T04:02:00.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.886+0000 7f3857bb5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385019d5e0 con 0x7f3850103800 2024-05-10T04:02:00.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.886+0000 7f3857bb5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385019d7a0 con 0x7f385010c290 2024-05-10T04:02:00.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.886+0000 7f3855951700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3850102eb0 0x7f3850198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:00.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.886+0000 7f3855951700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3850102eb0 0x7f3850198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51372/0 (socket says 172.21.15.150:51372) 2024-05-10T04:02:00.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.886+0000 7f3855951700 1 -- 172.21.15.150:0/3611767267 learned_addr learned my addr 172.21.15.150:0/3611767267 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:00.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.886+0000 7f3855150700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3850103800 0x7f3850198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:00.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.887+0000 7f3856152700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f385010c290 0x7f385019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.887+0000 7f3855951700 1 -- 172.21.15.150:0/3611767267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f385010c290 msgr2=0x7f385019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.887+0000 7f3855951700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f385010c290 0x7f385019cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.887+0000 7f3855951700 1 -- 172.21.15.150:0/3611767267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3850103800 msgr2=0x7f3850198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.887+0000 7f3855951700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3850103800 0x7f3850198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.887+0000 7f3855951700 1 -- 172.21.15.150:0/3611767267 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38400095f0 con 0x7f3850102eb0 2024-05-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.887+0000 7f3855150700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3850103800 0x7f3850198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.887+0000 7f3855951700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3850102eb0 0x7f3850198480 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f384c00c990 tx=0x7f384c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.887+0000 7f3846ffd700 1 -- 172.21.15.150:0/3611767267 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c007b00 con 0x7f3850102eb0 2024-05-10T04:02:00.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.887+0000 7f3846ffd700 1 -- 172.21.15.150:0/3611767267 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f384c007c60 con 0x7f3850102eb0 2024-05-10T04:02:00.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.887+0000 7f3857bb5700 1 -- 172.21.15.150:0/3611767267 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385019da80 con 0x7f3850102eb0 2024-05-10T04:02:00.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.887+0000 7f3846ffd700 1 -- 172.21.15.150:0/3611767267 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c018660 con 0x7f3850102eb0 2024-05-10T04:02:00.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.887+0000 7f3857bb5700 1 -- 172.21.15.150:0/3611767267 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385019ddb0 con 0x7f3850102eb0 2024-05-10T04:02:00.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.888+0000 7f3857bb5700 1 -- 172.21.15.150:0/3611767267 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3850107450 con 0x7f3850102eb0 2024-05-10T04:02:00.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.889+0000 7f3846ffd700 1 -- 172.21.15.150:0/3611767267 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f384c004020 con 0x7f3850102eb0 2024-05-10T04:02:00.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.890+0000 7f3846ffd700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f383c0a1120 0x7f383c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:00.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.890+0000 7f3855150700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f383c0a1120 0x7f383c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:00.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.890+0000 7f3846ffd700 1 -- 172.21.15.150:0/3611767267 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f384c0cc580 con 0x7f3850102eb0 2024-05-10T04:02:00.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.890+0000 7f3855150700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f383c0a1120 0x7f383c0a3630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f38500ffdc0 tx=0x7f384000b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:00.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:00.894+0000 7f3846ffd700 1 -- 172.21.15.150:0/3611767267 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f384c096240 con 0x7f3850102eb0 2024-05-10T04:02:01.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.087+0000 7f3857bb5700 1 -- 172.21.15.150:0/3611767267 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f385006a0f0 con 0x7f3850102eb0 2024-05-10T04:02:01.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.088+0000 7f3846ffd700 1 -- 172.21.15.150:0/3611767267 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18489 (secure 0 0 0) 0x7f384c099860 con 0x7f3850102eb0 2024-05-10T04:02:01.091 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:01.091 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":77,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:01:58.242469+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:02:01.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.091+0000 7f3857bb5700 1 -- 172.21.15.150:0/3611767267 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f383c0a1120 msgr2=0x7f383c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:01.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.091+0000 7f3857bb5700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f383c0a1120 0x7f383c0a3630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f38500ffdc0 tx=0x7f384000b580 comp rx=0 tx=0).stop 2024-05-10T04:02:01.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.091+0000 7f3857bb5700 1 -- 172.21.15.150:0/3611767267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3850102eb0 msgr2=0x7f3850198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:01.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.091+0000 7f3857bb5700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3850102eb0 0x7f3850198480 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f384c00c990 tx=0x7f384c00cd50 comp rx=0 tx=0).stop 2024-05-10T04:02:01.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.091+0000 7f3857bb5700 1 -- 172.21.15.150:0/3611767267 shutdown_connections 2024-05-10T04:02:01.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.091+0000 7f3857bb5700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f383c0a1120 0x7f383c0a3630 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:01.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.091+0000 7f3857bb5700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f385010c290 0x7f385019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:01.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.091+0000 7f3857bb5700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3850102eb0 0x7f3850198480 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:01.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.091+0000 7f3857bb5700 1 --2- 172.21.15.150:0/3611767267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3850103800 0x7f3850198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:01.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.091+0000 7f3857bb5700 1 -- 172.21.15.150:0/3611767267 >> 172.21.15.150:0/3611767267 conn(0x7f38500fc7a0 msgr2=0x7f38500fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:01.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.091+0000 7f3857bb5700 1 -- 172.21.15.150:0/3611767267 shutdown_connections 2024-05-10T04:02:01.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:01.091+0000 7f3857bb5700 1 -- 172.21.15.150:0/3611767267 wait complete. 2024-05-10T04:02:01.454 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:01 smithi150 ceph-mon[42477]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:01.454 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:01 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3611767267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:01.456 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T04:02:01.457 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-10T04:02:01.679 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:02:01.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:01 smithi181 ceph-mon[46602]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:01.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:01 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3611767267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:01.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:01 smithi161 ceph-mon[46918]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:01.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:01 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3611767267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:02.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.598+0000 7f7414664700 1 -- 172.21.15.150:0/664183685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740c100a40 msgr2=0x7f740c10ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:02.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.598+0000 7f7414664700 1 --2- 172.21.15.150:0/664183685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740c100a40 0x7f740c10ead0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f740400b0a0 tx=0x7f740400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:02.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.598+0000 7f7414664700 1 -- 172.21.15.150:0/664183685 shutdown_connections 2024-05-10T04:02:02.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.598+0000 7f7414664700 1 --2- 172.21.15.150:0/664183685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740c100a40 0x7f740c10ead0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:02.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.598+0000 7f7414664700 1 --2- 172.21.15.150:0/664183685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f740c106c60 0x7f740c107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:02.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.598+0000 7f7414664700 1 --2- 172.21.15.150:0/664183685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f740c1000f0 0x7f740c100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:02.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.598+0000 7f7414664700 1 -- 172.21.15.150:0/664183685 >> 172.21.15.150:0/664183685 conn(0x7f740c0fbed0 msgr2=0x7f740c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:02.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.598+0000 7f7414664700 1 -- 172.21.15.150:0/664183685 shutdown_connections 2024-05-10T04:02:02.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.598+0000 7f7414664700 1 -- 172.21.15.150:0/664183685 wait complete. 2024-05-10T04:02:02.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.599+0000 7f7414664700 1 Processor -- start 2024-05-10T04:02:02.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.599+0000 7f7414664700 1 -- start start 2024-05-10T04:02:02.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.599+0000 7f7414664700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f740c1000f0 0x7f740c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:02.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.599+0000 7f7414664700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740c100a40 0x7f740c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:02.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.599+0000 7f7414664700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f740c106c60 0x7f740c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:02.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.599+0000 7f7414664700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c198bd0 con 0x7f740c100a40 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.599+0000 7f7414664700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c198d40 con 0x7f740c1000f0 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.599+0000 7f7414664700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c198f00 con 0x7f740c106c60 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.599+0000 7f7411bff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740c100a40 0x7f740c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.599+0000 7f7411bff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740c100a40 0x7f740c19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51392/0 (socket says 172.21.15.150:51392) 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.599+0000 7f7411bff700 1 -- 172.21.15.150:0/1988891857 learned_addr learned my addr 172.21.15.150:0/1988891857 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.599+0000 7f7412400700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f740c1000f0 0x7f740c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.600+0000 7f7412c01700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f740c106c60 0x7f740c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.600+0000 7f7411bff700 1 -- 172.21.15.150:0/1988891857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f740c106c60 msgr2=0x7f740c1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.600+0000 7f7411bff700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f740c106c60 0x7f740c1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.600+0000 7f7411bff700 1 -- 172.21.15.150:0/1988891857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f740c1000f0 msgr2=0x7f740c19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.600+0000 7f7411bff700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f740c1000f0 0x7f740c19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.600+0000 7f7411bff700 1 -- 172.21.15.150:0/1988891857 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7404009d00 con 0x7f740c100a40 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.600+0000 7f7412c01700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f740c106c60 0x7f740c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:02.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.600+0000 7f7411bff700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740c100a40 0x7f740c19eb70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00d950 tx=0x7f73fc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:02.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.600+0000 7f74037fe700 1 -- 172.21.15.150:0/1988891857 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc004220 con 0x7f740c100a40 2024-05-10T04:02:02.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.600+0000 7f74037fe700 1 -- 172.21.15.150:0/1988891857 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73fc007620 con 0x7f740c100a40 2024-05-10T04:02:02.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.600+0000 7f7414664700 1 -- 172.21.15.150:0/1988891857 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c1991e0 con 0x7f740c100a40 2024-05-10T04:02:02.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.600+0000 7f74037fe700 1 -- 172.21.15.150:0/1988891857 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc005690 con 0x7f740c100a40 2024-05-10T04:02:02.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.600+0000 7f7414664700 1 -- 172.21.15.150:0/1988891857 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c1a5640 con 0x7f740c100a40 2024-05-10T04:02:02.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.601+0000 7f7414664700 1 -- 172.21.15.150:0/1988891857 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c10a220 con 0x7f740c100a40 2024-05-10T04:02:02.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.602+0000 7f74037fe700 1 -- 172.21.15.150:0/1988891857 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f73fc004750 con 0x7f740c100a40 2024-05-10T04:02:02.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.603+0000 7f74037fe700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f73f80a11a0 0x7f73f80a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:02.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.603+0000 7f7412400700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f73f80a11a0 0x7f73f80a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:02.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.603+0000 7f74037fe700 1 -- 172.21.15.150:0/1988891857 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f73fc0cc670 con 0x7f740c100a40 2024-05-10T04:02:02.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.603+0000 7f7412400700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f73f80a11a0 0x7f73f80a36b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f740c199900 tx=0x7f7408009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:02.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.607+0000 7f74037fe700 1 -- 172.21.15.150:0/1988891857 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f73fc0962a0 con 0x7f740c100a40 2024-05-10T04:02:02.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.801+0000 7f7414664700 1 -- 172.21.15.150:0/1988891857 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f740c199cf0 con 0x7f740c100a40 2024-05-10T04:02:02.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.802+0000 7f74037fe700 1 -- 172.21.15.150:0/1988891857 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73fc0998c0 con 0x7f740c100a40 2024-05-10T04:02:02.804 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:02:02.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.804+0000 7f7414664700 1 -- 172.21.15.150:0/1988891857 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f73f80a11a0 msgr2=0x7f73f80a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:02.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.804+0000 7f7414664700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f73f80a11a0 0x7f73f80a36b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f740c199900 tx=0x7f7408009260 comp rx=0 tx=0).stop 2024-05-10T04:02:02.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.804+0000 7f7414664700 1 -- 172.21.15.150:0/1988891857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740c100a40 msgr2=0x7f740c19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:02.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.804+0000 7f7414664700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740c100a40 0x7f740c19eb70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00d950 tx=0x7f73fc00dc60 comp rx=0 tx=0).stop 2024-05-10T04:02:02.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.804+0000 7f7414664700 1 -- 172.21.15.150:0/1988891857 shutdown_connections 2024-05-10T04:02:02.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.804+0000 7f7414664700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f73f80a11a0 0x7f73f80a36b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:02.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.804+0000 7f7414664700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f740c106c60 0x7f740c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:02.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.804+0000 7f7414664700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f740c1000f0 0x7f740c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:02.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.804+0000 7f7414664700 1 --2- 172.21.15.150:0/1988891857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740c100a40 0x7f740c19eb70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:02.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.804+0000 7f7414664700 1 -- 172.21.15.150:0/1988891857 >> 172.21.15.150:0/1988891857 conn(0x7f740c0fbed0 msgr2=0x7f740c1080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:02.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.804+0000 7f7414664700 1 -- 172.21.15.150:0/1988891857 shutdown_connections 2024-05-10T04:02:02.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:02.804+0000 7f7414664700 1 -- 172.21.15.150:0/1988891857 wait complete. 2024-05-10T04:02:03.079 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:02 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1988891857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:03.172 INFO:tasks.cephadm:Adding mds.a on smithi150 2024-05-10T04:02:03.172 INFO:tasks.cephadm:Adding mds.d on smithi150 2024-05-10T04:02:03.172 INFO:tasks.cephadm:Adding mds.g on smithi150 2024-05-10T04:02:03.172 INFO:tasks.cephadm:Adding mds.j on smithi150 2024-05-10T04:02:03.172 INFO:tasks.cephadm:Adding mds.b on smithi161 2024-05-10T04:02:03.172 INFO:tasks.cephadm:Adding mds.e on smithi161 2024-05-10T04:02:03.172 INFO:tasks.cephadm:Adding mds.h on smithi161 2024-05-10T04:02:03.173 INFO:tasks.cephadm:Adding mds.k on smithi161 2024-05-10T04:02:03.173 INFO:tasks.cephadm:Adding mds.c on smithi181 2024-05-10T04:02:03.173 INFO:tasks.cephadm:Adding mds.f on smithi181 2024-05-10T04:02:03.173 INFO:tasks.cephadm:Adding mds.i on smithi181 2024-05-10T04:02:03.173 INFO:tasks.cephadm:Adding mds.l on smithi181 2024-05-10T04:02:03.173 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l' 2024-05-10T04:02:03.207 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:02 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1988891857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:03.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:02 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1988891857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:03.414 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:02:04.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:03 smithi150 ceph-mon[42477]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:04.217 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:03 smithi181 ceph-mon[46602]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:04.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:03 smithi161 ceph-mon[46918]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:04.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.427+0000 7f16bca17700 1 -- 172.21.15.181:0/704361163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b8101d50 msgr2=0x7f16b8102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:04.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.427+0000 7f16bca17700 1 --2- 172.21.15.181:0/704361163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b8101d50 0x7f16b8102160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009990 tx=0x7f16a8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:04.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.427+0000 7f16bca17700 1 -- 172.21.15.181:0/704361163 shutdown_connections 2024-05-10T04:02:04.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.427+0000 7f16bca17700 1 --2- 172.21.15.181:0/704361163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16b8103820 0x7f16b810a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:04.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.427+0000 7f16bca17700 1 --2- 172.21.15.181:0/704361163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b8101d50 0x7f16b8102160 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:04.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.427+0000 7f16bca17700 1 --2- 172.21.15.181:0/704361163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16b8102e80 0x7f16b8103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:04.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.427+0000 7f16bca17700 1 -- 172.21.15.181:0/704361163 >> 172.21.15.181:0/704361163 conn(0x7f16b80fd390 msgr2=0x7f16b80ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:04.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.427+0000 7f16bca17700 1 -- 172.21.15.181:0/704361163 shutdown_connections 2024-05-10T04:02:04.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.427+0000 7f16bca17700 1 -- 172.21.15.181:0/704361163 wait complete. 2024-05-10T04:02:04.432 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.428+0000 7f16bca17700 1 Processor -- start 2024-05-10T04:02:04.432 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.428+0000 7f16bca17700 1 -- start start 2024-05-10T04:02:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.428+0000 7f16bca17700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16b8101d50 0x7f16b81981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.428+0000 7f16bca17700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16b8102e80 0x7f16b81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.428+0000 7f16bca17700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b8103820 0x7f16b8199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.428+0000 7f16bca17700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8199af0 con 0x7f16b8102e80 2024-05-10T04:02:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.428+0000 7f16bca17700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8199c30 con 0x7f16b8101d50 2024-05-10T04:02:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.428+0000 7f16bca17700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8199df0 con 0x7f16b8103820 2024-05-10T04:02:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.428+0000 7f16b6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b8103820 0x7f16b8199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.428+0000 7f16b6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b8103820 0x7f16b8199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47488/0 (socket says 172.21.15.181:47488) 2024-05-10T04:02:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.428+0000 7f16b6d9d700 1 -- 172.21.15.181:0/3623694258 learned_addr learned my addr 172.21.15.181:0/3623694258 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T04:02:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.428+0000 7f16b659c700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16b8101d50 0x7f16b81981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.428+0000 7f16b5d9b700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16b8102e80 0x7f16b81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.429+0000 7f16b6d9d700 1 -- 172.21.15.181:0/3623694258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16b8101d50 msgr2=0x7f16b81981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.429+0000 7f16b6d9d700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16b8101d50 0x7f16b81981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.429+0000 7f16b6d9d700 1 -- 172.21.15.181:0/3623694258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16b8102e80 msgr2=0x7f16b81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.429+0000 7f16b6d9d700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16b8102e80 0x7f16b81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.429+0000 7f16b6d9d700 1 -- 172.21.15.181:0/3623694258 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a80095f0 con 0x7f16b8103820 2024-05-10T04:02:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.429+0000 7f16b6d9d700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b8103820 0x7f16b8199490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f16b0007ee0 tx=0x7f16b000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.429+0000 7f16a77fe700 1 -- 172.21.15.181:0/3623694258 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0017070 con 0x7f16b8103820 2024-05-10T04:02:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.429+0000 7f16a77fe700 1 -- 172.21.15.181:0/3623694258 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16b000f040 con 0x7f16b8103820 2024-05-10T04:02:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.429+0000 7f16bca17700 1 -- 172.21.15.181:0/3623694258 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b819a0d0 con 0x7f16b8103820 2024-05-10T04:02:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.429+0000 7f16a77fe700 1 -- 172.21.15.181:0/3623694258 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0005580 con 0x7f16b8103820 2024-05-10T04:02:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.429+0000 7f16bca17700 1 -- 172.21.15.181:0/3623694258 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b81a08b0 con 0x7f16b8103820 2024-05-10T04:02:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.430+0000 7f16bca17700 1 -- 172.21.15.181:0/3623694258 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b8108f70 con 0x7f16b8103820 2024-05-10T04:02:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.432+0000 7f16a77fe700 1 -- 172.21.15.181:0/3623694258 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f16b000db20 con 0x7f16b8103820 2024-05-10T04:02:04.435 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.433+0000 7f16a77fe700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f16a00a1120 0x7f16a00a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:04.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.433+0000 7f16a77fe700 1 -- 172.21.15.181:0/3623694258 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f16b00cc150 con 0x7f16b8103820 2024-05-10T04:02:04.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.433+0000 7f16b659c700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f16a00a1120 0x7f16a00a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:04.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.434+0000 7f16b659c700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f16a00a1120 0x7f16a00a3630 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f16a800b5c0 tx=0x7f16a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:04.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.436+0000 7f16a77fe700 1 -- 172.21.15.181:0/3623694258 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f16b0095d80 con 0x7f16b8103820 2024-05-10T04:02:04.641 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.639+0000 7f16bca17700 1 -- 172.21.15.181:0/3623694258 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f16b810d790 con 0x7f16a00a1120 2024-05-10T04:02:04.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.648+0000 7f16a77fe700 1 -- 172.21.15.181:0/3623694258 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f16b810d790 con 0x7f16a00a1120 2024-05-10T04:02:04.651 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mds.all update... 2024-05-10T04:02:04.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.650+0000 7f16bca17700 1 -- 172.21.15.181:0/3623694258 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f16a00a1120 msgr2=0x7f16a00a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:04.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.650+0000 7f16bca17700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f16a00a1120 0x7f16a00a3630 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f16a800b5c0 tx=0x7f16a8005fb0 comp rx=0 tx=0).stop 2024-05-10T04:02:04.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.650+0000 7f16bca17700 1 -- 172.21.15.181:0/3623694258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b8103820 msgr2=0x7f16b8199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:04.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.650+0000 7f16bca17700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b8103820 0x7f16b8199490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f16b0007ee0 tx=0x7f16b000d3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:04.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.650+0000 7f16bca17700 1 -- 172.21.15.181:0/3623694258 shutdown_connections 2024-05-10T04:02:04.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.650+0000 7f16bca17700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f16a00a1120 0x7f16a00a3630 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:04.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.650+0000 7f16bca17700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b8103820 0x7f16b8199490 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:04.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.650+0000 7f16bca17700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16b8101d50 0x7f16b81981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:04.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.650+0000 7f16bca17700 1 --2- 172.21.15.181:0/3623694258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16b8102e80 0x7f16b81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:04.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.650+0000 7f16bca17700 1 -- 172.21.15.181:0/3623694258 >> 172.21.15.181:0/3623694258 conn(0x7f16b80fd390 msgr2=0x7f16b81060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:04.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.651+0000 7f16bca17700 1 -- 172.21.15.181:0/3623694258 shutdown_connections 2024-05-10T04:02:04.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:02:04.651+0000 7f16bca17700 1 -- 172.21.15.181:0/3623694258 wait complete. 2024-05-10T04:02:05.028 DEBUG:teuthology.orchestra.run.smithi150:mds.a> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.a.service 2024-05-10T04:02:05.031 DEBUG:teuthology.orchestra.run.smithi150:mds.d> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.d.service 2024-05-10T04:02:05.062 INFO:journalctl@ceph.mds.a.smithi150.stdout:-- Logs begin at Fri 2024-05-10 03:40:30 UTC. -- 2024-05-10T04:02:05.064 DEBUG:teuthology.orchestra.run.smithi150:mds.g> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.g.service 2024-05-10T04:02:05.092 INFO:journalctl@ceph.mds.d.smithi150.stdout:-- Logs begin at Fri 2024-05-10 03:40:30 UTC. -- 2024-05-10T04:02:05.093 DEBUG:teuthology.orchestra.run.smithi150:mds.j> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.j.service 2024-05-10T04:02:05.119 INFO:journalctl@ceph.mds.g.smithi150.stdout:-- Logs begin at Fri 2024-05-10 03:40:30 UTC. -- 2024-05-10T04:02:05.120 DEBUG:teuthology.orchestra.run.smithi161:mds.b> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.b.service 2024-05-10T04:02:05.123 DEBUG:teuthology.orchestra.run.smithi161:mds.e> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.e.service 2024-05-10T04:02:05.145 INFO:journalctl@ceph.mds.j.smithi150.stdout:-- Logs begin at Fri 2024-05-10 03:40:30 UTC. -- 2024-05-10T04:02:05.160 INFO:journalctl@ceph.mds.b.smithi161.stdout:-- Logs begin at Fri 2024-05-10 03:40:24 UTC. -- 2024-05-10T04:02:05.161 DEBUG:teuthology.orchestra.run.smithi161:mds.h> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.h.service 2024-05-10T04:02:05.190 INFO:journalctl@ceph.mds.e.smithi161.stdout:-- Logs begin at Fri 2024-05-10 03:40:24 UTC. -- 2024-05-10T04:02:05.192 DEBUG:teuthology.orchestra.run.smithi161:mds.k> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.k.service 2024-05-10T04:02:05.220 INFO:journalctl@ceph.mds.h.smithi161.stdout:-- Logs begin at Fri 2024-05-10 03:40:24 UTC. -- 2024-05-10T04:02:05.223 DEBUG:teuthology.orchestra.run.smithi181:mds.c> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.c.service 2024-05-10T04:02:05.225 DEBUG:teuthology.orchestra.run.smithi181:mds.f> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.f.service 2024-05-10T04:02:05.248 INFO:journalctl@ceph.mds.k.smithi161.stdout:-- Logs begin at Fri 2024-05-10 03:40:24 UTC. -- 2024-05-10T04:02:05.257 INFO:journalctl@ceph.mds.c.smithi181.stdout:-- Logs begin at Fri 2024-05-10 03:40:19 UTC. -- 2024-05-10T04:02:05.258 DEBUG:teuthology.orchestra.run.smithi181:mds.i> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.i.service 2024-05-10T04:02:05.287 INFO:journalctl@ceph.mds.f.smithi181.stdout:-- Logs begin at Fri 2024-05-10 03:40:19 UTC. -- 2024-05-10T04:02:05.288 DEBUG:teuthology.orchestra.run.smithi181:mds.l> sudo journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.l.service 2024-05-10T04:02:05.316 INFO:journalctl@ceph.mds.i.smithi181.stdout:-- Logs begin at Fri 2024-05-10 03:40:19 UTC. -- 2024-05-10T04:02:05.317 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-10T04:02:05.318 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:02:05.346 INFO:journalctl@ceph.mds.l.smithi181.stdout:-- Logs begin at Fri 2024-05-10 03:40:19 UTC. -- 2024-05-10T04:02:05.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.474+0000 7f640cbf9700 1 -- 172.21.15.150:0/1124895140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408102580 msgr2=0x7f6408108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:05.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.474+0000 7f640cbf9700 1 --2- 172.21.15.150:0/1124895140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408102580 0x7f6408108e30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f640000b0a0 tx=0x7f640000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.474+0000 7f640cbf9700 1 -- 172.21.15.150:0/1124895140 shutdown_connections 2024-05-10T04:02:05.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.474+0000 7f640cbf9700 1 --2- 172.21.15.150:0/1124895140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408102580 0x7f6408108e30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.474+0000 7f640cbf9700 1 --2- 172.21.15.150:0/1124895140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6408100ac0 0x7f6408100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.474+0000 7f640cbf9700 1 --2- 172.21.15.150:0/1124895140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6408101be0 0x7f6408101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.474+0000 7f640cbf9700 1 -- 172.21.15.150:0/1124895140 >> 172.21.15.150:0/1124895140 conn(0x7f64080fc0d0 msgr2=0x7f64080fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:05.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.474+0000 7f640cbf9700 1 -- 172.21.15.150:0/1124895140 shutdown_connections 2024-05-10T04:02:05.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.474+0000 7f640cbf9700 1 -- 172.21.15.150:0/1124895140 wait complete. 2024-05-10T04:02:05.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.475+0000 7f640cbf9700 1 Processor -- start 2024-05-10T04:02:05.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.475+0000 7f640cbf9700 1 -- start start 2024-05-10T04:02:05.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.475+0000 7f640cbf9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6408100ac0 0x7f6408068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:05.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f640cbf9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6408101be0 0x7f640819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:05.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f640cbf9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408102580 0x7f640819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:05.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f640cbf9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640819fb90 con 0x7f6408102580 2024-05-10T04:02:05.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f640cbf9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640819fd00 con 0x7f6408101be0 2024-05-10T04:02:05.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f640cbf9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640819fec0 con 0x7f6408100ac0 2024-05-10T04:02:05.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f6406d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408102580 0x7f640819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:05.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f6406d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408102580 0x7f640819f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51420/0 (socket says 172.21.15.150:51420) 2024-05-10T04:02:05.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f6406d9d700 1 -- 172.21.15.150:0/1969887615 learned_addr learned my addr 172.21.15.150:0/1969887615 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:05.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f640659c700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6408100ac0 0x7f6408068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:05.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f6405d9b700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6408101be0 0x7f640819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:05.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f6406d9d700 1 -- 172.21.15.150:0/1969887615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6408100ac0 msgr2=0x7f6408068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:05.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f6406d9d700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6408100ac0 0x7f6408068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f6406d9d700 1 -- 172.21.15.150:0/1969887615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6408101be0 msgr2=0x7f640819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:05.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f6406d9d700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6408101be0 0x7f640819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f6406d9d700 1 -- 172.21.15.150:0/1969887615 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6400009d00 con 0x7f6408102580 2024-05-10T04:02:05.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f6405d9b700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6408101be0 0x7f640819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:05.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f6406d9d700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408102580 0x7f640819f440 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6400006070 tx=0x7f64000078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:05.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f63f77fe700 1 -- 172.21.15.150:0/1969887615 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640000e040 con 0x7f6408102580 2024-05-10T04:02:05.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f63f77fe700 1 -- 172.21.15.150:0/1969887615 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f640000be40 con 0x7f6408102580 2024-05-10T04:02:05.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f640cbf9700 1 -- 172.21.15.150:0/1969887615 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64081a00e0 con 0x7f6408102580 2024-05-10T04:02:05.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f63f77fe700 1 -- 172.21.15.150:0/1969887615 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64000052a0 con 0x7f6408102580 2024-05-10T04:02:05.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.476+0000 7f640cbf9700 1 -- 172.21.15.150:0/1969887615 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64081a0560 con 0x7f6408102580 2024-05-10T04:02:05.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.477+0000 7f640cbf9700 1 -- 172.21.15.150:0/1969887615 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6408107cd0 con 0x7f6408102580 2024-05-10T04:02:05.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.478+0000 7f63f77fe700 1 -- 172.21.15.150:0/1969887615 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f640001d470 con 0x7f6408102580 2024-05-10T04:02:05.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.479+0000 7f63f77fe700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f63f00a11a0 0x7f63f00a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:05.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.479+0000 7f640659c700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f63f00a11a0 0x7f63f00a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:05.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.479+0000 7f63f77fe700 1 -- 172.21.15.150:0/1969887615 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f64000cd6f0 con 0x7f6408102580 2024-05-10T04:02:05.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.480+0000 7f640659c700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f63f00a11a0 0x7f63f00a36b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6408101ae0 tx=0x7f63fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:05.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.483+0000 7f63f77fe700 1 -- 172.21.15.150:0/1969887615 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6400097310 con 0x7f6408102580 2024-05-10T04:02:05.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.682+0000 7f640cbf9700 1 -- 172.21.15.150:0/1969887615 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f640810c2d0 con 0x7f6408102580 2024-05-10T04:02:05.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.683+0000 7f63f77fe700 1 -- 172.21.15.150:0/1969887615 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18489 (secure 0 0 0) 0x7f6400024350 con 0x7f6408102580 2024-05-10T04:02:05.686 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:05.686 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":77,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:01:58.242469+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:02:05.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.686+0000 7f640cbf9700 1 -- 172.21.15.150:0/1969887615 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f63f00a11a0 msgr2=0x7f63f00a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:05.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.686+0000 7f640cbf9700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f63f00a11a0 0x7f63f00a36b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6408101ae0 tx=0x7f63fc009260 comp rx=0 tx=0).stop 2024-05-10T04:02:05.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.686+0000 7f640cbf9700 1 -- 172.21.15.150:0/1969887615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408102580 msgr2=0x7f640819f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:05.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.686+0000 7f640cbf9700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408102580 0x7f640819f440 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6400006070 tx=0x7f64000078e0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.686+0000 7f640cbf9700 1 -- 172.21.15.150:0/1969887615 shutdown_connections 2024-05-10T04:02:05.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.686+0000 7f640cbf9700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f63f00a11a0 0x7f63f00a36b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.686+0000 7f640cbf9700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408102580 0x7f640819f440 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.686+0000 7f640cbf9700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6408100ac0 0x7f6408068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.686+0000 7f640cbf9700 1 --2- 172.21.15.150:0/1969887615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6408101be0 0x7f640819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.686+0000 7f640cbf9700 1 -- 172.21.15.150:0/1969887615 >> 172.21.15.150:0/1969887615 conn(0x7f64080fc0d0 msgr2=0x7f6408104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:05.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.686+0000 7f640cbf9700 1 -- 172.21.15.150:0/1969887615 shutdown_connections 2024-05-10T04:02:05.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.686+0000 7f640cbf9700 1 -- 172.21.15.150:0/1969887615 wait complete. 2024-05-10T04:02:05.703 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T04:02:05.703 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:02:05.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.836+0000 7f0ef399d700 1 -- 172.21.15.150:0/2062140755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eec100570 msgr2=0x7f0eec10a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:05.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.836+0000 7f0ef399d700 1 --2- 172.21.15.150:0/2062140755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eec100570 0x7f0eec10a190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400b0a0 tx=0x7f0ee400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.836+0000 7f0ef399d700 1 -- 172.21.15.150:0/2062140755 shutdown_connections 2024-05-10T04:02:05.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.836+0000 7f0ef399d700 1 --2- 172.21.15.150:0/2062140755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eec100570 0x7f0eec10a190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.836+0000 7f0ef399d700 1 --2- 172.21.15.150:0/2062140755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eec1031e0 0x7f0eec1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.836+0000 7f0ef399d700 1 --2- 172.21.15.150:0/2062140755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eec0ffc20 0x7f0eec100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.836+0000 7f0ef399d700 1 -- 172.21.15.150:0/2062140755 >> 172.21.15.150:0/2062140755 conn(0x7f0eec0fb8f0 msgr2=0x7f0eec0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.836+0000 7f0ef399d700 1 -- 172.21.15.150:0/2062140755 shutdown_connections 2024-05-10T04:02:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.836+0000 7f0ef399d700 1 -- 172.21.15.150:0/2062140755 wait complete. 2024-05-10T04:02:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.837+0000 7f0ef399d700 1 Processor -- start 2024-05-10T04:02:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.837+0000 7f0ef399d700 1 -- start start 2024-05-10T04:02:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.837+0000 7f0ef399d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eec0ffc20 0x7f0eec197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.837+0000 7f0ef399d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eec100570 0x7f0eec198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.837+0000 7f0ef399d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eec1031e0 0x7f0eec19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.837+0000 7f0ef399d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec19fa90 con 0x7f0eec100570 2024-05-10T04:02:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.837+0000 7f0ef399d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec19fbd0 con 0x7f0eec0ffc20 2024-05-10T04:02:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.837+0000 7f0ef399d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec19fd60 con 0x7f0eec1031e0 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.837+0000 7f0ef0f38700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eec100570 0x7f0eec198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.837+0000 7f0ef0f38700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eec100570 0x7f0eec198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51448/0 (socket says 172.21.15.150:51448) 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.837+0000 7f0ef0f38700 1 -- 172.21.15.150:0/604786234 learned_addr learned my addr 172.21.15.150:0/604786234 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.837+0000 7f0ef1f3a700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eec1031e0 0x7f0eec19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.838+0000 7f0ef1739700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eec0ffc20 0x7f0eec197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.838+0000 7f0ef0f38700 1 -- 172.21.15.150:0/604786234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eec1031e0 msgr2=0x7f0eec19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.838+0000 7f0ef0f38700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eec1031e0 0x7f0eec19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.838+0000 7f0ef0f38700 1 -- 172.21.15.150:0/604786234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eec0ffc20 msgr2=0x7f0eec197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.838+0000 7f0ef0f38700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eec0ffc20 0x7f0eec197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.838+0000 7f0ef0f38700 1 -- 172.21.15.150:0/604786234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee4009d00 con 0x7f0eec100570 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.838+0000 7f0ef0f38700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eec100570 0x7f0eec198530 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800d980 tx=0x7f0ee800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.838+0000 7f0ede7fc700 1 -- 172.21.15.150:0/604786234 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80098d0 con 0x7f0eec100570 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.838+0000 7f0ede7fc700 1 -- 172.21.15.150:0/604786234 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ee8010460 con 0x7f0eec100570 2024-05-10T04:02:05.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.838+0000 7f0ef399d700 1 -- 172.21.15.150:0/604786234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eec19ffe0 con 0x7f0eec100570 2024-05-10T04:02:05.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.838+0000 7f0ede7fc700 1 -- 172.21.15.150:0/604786234 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee800f620 con 0x7f0eec100570 2024-05-10T04:02:05.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.838+0000 7f0ef399d700 1 -- 172.21.15.150:0/604786234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec1a0490 con 0x7f0eec100570 2024-05-10T04:02:05.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.839+0000 7f0ef399d700 1 -- 172.21.15.150:0/604786234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eec109030 con 0x7f0eec100570 2024-05-10T04:02:05.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.840+0000 7f0ede7fc700 1 -- 172.21.15.150:0/604786234 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0ee8010860 con 0x7f0eec100570 2024-05-10T04:02:05.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.841+0000 7f0ede7fc700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0ed80a11c0 0x7f0ed80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:05.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.841+0000 7f0ede7fc700 1 -- 172.21.15.150:0/604786234 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0ee80cc170 con 0x7f0eec100570 2024-05-10T04:02:05.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.841+0000 7f0ef1739700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0ed80a11c0 0x7f0ed80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:05.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.841+0000 7f0ef1739700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0ed80a11c0 0x7f0ed80a36d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0eec068120 tx=0x7f0ee000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:05.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:05.845+0000 7f0ede7fc700 1 -- 172.21.15.150:0/604786234 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0ee8095da0 con 0x7f0eec100570 2024-05-10T04:02:05.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:05 smithi181 ceph-mon[46602]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:02:05.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:05 smithi181 ceph-mon[46602]: Saving service mds.all spec with placement smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2024-05-10T04:02:05.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:05 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:05.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:05 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:02:05.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:05 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:05.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:05 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:02:05.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:05 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:05.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:05 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:05.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:05 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:05.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:05 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:05.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:05 smithi181 ceph-mon[46602]: Deploying daemon mds.l on smithi181 2024-05-10T04:02:05.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:05 smithi181 ceph-mon[46602]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:05.963 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:05 smithi150 ceph-mon[42477]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:02:05.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:05 smithi150 ceph-mon[42477]: Saving service mds.all spec with placement smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2024-05-10T04:02:05.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:05 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:05.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:05 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:02:05.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:05 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:05.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:05 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:02:05.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:05 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:05.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:05 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:05.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:05 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:05.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:05 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:05.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:05 smithi150 ceph-mon[42477]: Deploying daemon mds.l on smithi181 2024-05-10T04:02:05.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:05 smithi150 ceph-mon[42477]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:05.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:05 smithi161 ceph-mon[46918]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:02:05.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:05 smithi161 ceph-mon[46918]: Saving service mds.all spec with placement smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2024-05-10T04:02:05.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:05 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:05.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:05 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:02:05.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:05 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:05.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:05 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:02:05.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:05 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:05.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:05 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:05.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:05 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:05.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:05 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:05.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:05 smithi161 ceph-mon[46918]: Deploying daemon mds.l on smithi181 2024-05-10T04:02:05.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:05 smithi161 ceph-mon[46918]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:06.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.042+0000 7f0ef399d700 1 -- 172.21.15.150:0/604786234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0eec0678b0 con 0x7f0eec100570 2024-05-10T04:02:06.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.042+0000 7f0ede7fc700 1 -- 172.21.15.150:0/604786234 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ee80993c0 con 0x7f0eec100570 2024-05-10T04:02:06.045 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:02:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.045+0000 7f0ef399d700 1 -- 172.21.15.150:0/604786234 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0ed80a11c0 msgr2=0x7f0ed80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.045+0000 7f0ef399d700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0ed80a11c0 0x7f0ed80a36d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0eec068120 tx=0x7f0ee000a3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.045+0000 7f0ef399d700 1 -- 172.21.15.150:0/604786234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eec100570 msgr2=0x7f0eec198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.045+0000 7f0ef399d700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eec100570 0x7f0eec198530 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800d980 tx=0x7f0ee800dc90 comp rx=0 tx=0).stop 2024-05-10T04:02:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.045+0000 7f0ef399d700 1 -- 172.21.15.150:0/604786234 shutdown_connections 2024-05-10T04:02:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.045+0000 7f0ef399d700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0ed80a11c0 0x7f0ed80a36d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.045+0000 7f0ef399d700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eec1031e0 0x7f0eec19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.045+0000 7f0ef399d700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eec0ffc20 0x7f0eec197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.045+0000 7f0ef399d700 1 --2- 172.21.15.150:0/604786234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eec100570 0x7f0eec198530 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.045+0000 7f0ef399d700 1 -- 172.21.15.150:0/604786234 >> 172.21.15.150:0/604786234 conn(0x7f0eec0fb8f0 msgr2=0x7f0eec106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.045+0000 7f0ef399d700 1 -- 172.21.15.150:0/604786234 shutdown_connections 2024-05-10T04:02:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.045+0000 7f0ef399d700 1 -- 172.21.15.150:0/604786234 wait complete. 2024-05-10T04:02:06.061 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:02:06.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.191+0000 7f830392e700 1 -- 172.21.15.150:0/352495626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82fc1050a0 msgr2=0x7f82fc1074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:06.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.191+0000 7f830392e700 1 --2- 172.21.15.150:0/352495626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82fc1050a0 0x7f82fc1074c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009990 tx=0x7f82f8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.191+0000 7f830392e700 1 -- 172.21.15.150:0/352495626 shutdown_connections 2024-05-10T04:02:06.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.191+0000 7f830392e700 1 --2- 172.21.15.150:0/352495626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82fc107a00 0x7f82fc109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.191+0000 7f830392e700 1 --2- 172.21.15.150:0/352495626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82fc0fff30 0x7f82fc100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.191+0000 7f830392e700 1 --2- 172.21.15.150:0/352495626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82fc1050a0 0x7f82fc1074c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.191+0000 7f830392e700 1 -- 172.21.15.150:0/352495626 >> 172.21.15.150:0/352495626 conn(0x7f82fc0fb880 msgr2=0x7f82fc0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:06.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.191+0000 7f830392e700 1 -- 172.21.15.150:0/352495626 shutdown_connections 2024-05-10T04:02:06.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.191+0000 7f830392e700 1 -- 172.21.15.150:0/352495626 wait complete. 2024-05-10T04:02:06.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f830392e700 1 Processor -- start 2024-05-10T04:02:06.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f830392e700 1 -- start start 2024-05-10T04:02:06.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f830392e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82fc0fff30 0x7f82fc197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:06.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f830392e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82fc1050a0 0x7f82fc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:06.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f830392e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82fc107a00 0x7f82fc19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:06.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f830392e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc19fa70 con 0x7f82fc107a00 2024-05-10T04:02:06.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f830392e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc19fbe0 con 0x7f82fc1050a0 2024-05-10T04:02:06.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f830392e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc19fda0 con 0x7f82fc0fff30 2024-05-10T04:02:06.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f8301ecb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82fc107a00 0x7f82fc19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:06.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f8301ecb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82fc107a00 0x7f82fc19f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51470/0 (socket says 172.21.15.150:51470) 2024-05-10T04:02:06.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f8301ecb700 1 -- 172.21.15.150:0/3269410713 learned_addr learned my addr 172.21.15.150:0/3269410713 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:06.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f83016ca700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82fc0fff30 0x7f82fc197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f8300ec9700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82fc1050a0 0x7f82fc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f8301ecb700 1 -- 172.21.15.150:0/3269410713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82fc0fff30 msgr2=0x7f82fc197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f8301ecb700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82fc0fff30 0x7f82fc197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f8301ecb700 1 -- 172.21.15.150:0/3269410713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82fc1050a0 msgr2=0x7f82fc1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f8301ecb700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82fc1050a0 0x7f82fc1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f8301ecb700 1 -- 172.21.15.150:0/3269410713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f80095f0 con 0x7f82fc107a00 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f8300ec9700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82fc1050a0 0x7f82fc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.192+0000 7f8301ecb700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82fc107a00 0x7f82fc19f460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f82f4007ee0 tx=0x7f82f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.193+0000 7f82ee7fc700 1 -- 172.21.15.150:0/3269410713 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4017070 con 0x7f82fc107a00 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.193+0000 7f82ee7fc700 1 -- 172.21.15.150:0/3269410713 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82f400f040 con 0x7f82fc107a00 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.193+0000 7f830392e700 1 -- 172.21.15.150:0/3269410713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc1a0080 con 0x7f82fc107a00 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.193+0000 7f82ee7fc700 1 -- 172.21.15.150:0/3269410713 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4005740 con 0x7f82fc107a00 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.193+0000 7f830392e700 1 -- 172.21.15.150:0/3269410713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc1a0540 con 0x7f82fc107a00 2024-05-10T04:02:06.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.194+0000 7f830392e700 1 -- 172.21.15.150:0/3269410713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc0fd430 con 0x7f82fc107a00 2024-05-10T04:02:06.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.195+0000 7f82ee7fc700 1 -- 172.21.15.150:0/3269410713 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f82f400dc60 con 0x7f82fc107a00 2024-05-10T04:02:06.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.196+0000 7f82ee7fc700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82e80a1120 0x7f82e80a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:06.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.196+0000 7f83016ca700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82e80a1120 0x7f82e80a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:06.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.196+0000 7f82ee7fc700 1 -- 172.21.15.150:0/3269410713 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f82f40cc240 con 0x7f82fc107a00 2024-05-10T04:02:06.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.196+0000 7f83016ca700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82e80a1120 0x7f82e80a3630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82fc103020 tx=0x7f82f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:06.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.200+0000 7f82ee7fc700 1 -- 172.21.15.150:0/3269410713 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f82f4095e70 con 0x7f82fc107a00 2024-05-10T04:02:06.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.444+0000 7f830392e700 1 -- 172.21.15.150:0/3269410713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82fc068730 con 0x7f82fc107a00 2024-05-10T04:02:06.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.444+0000 7f82ee7fc700 1 -- 172.21.15.150:0/3269410713 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f82f4099490 con 0x7f82fc107a00 2024-05-10T04:02:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:06.447 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-10T04:02:06.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.446+0000 7f830392e700 1 -- 172.21.15.150:0/3269410713 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82e80a1120 msgr2=0x7f82e80a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:06.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.446+0000 7f830392e700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82e80a1120 0x7f82e80a3630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82fc103020 tx=0x7f82f0009260 comp rx=0 tx=0).stop 2024-05-10T04:02:06.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.446+0000 7f830392e700 1 -- 172.21.15.150:0/3269410713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82fc107a00 msgr2=0x7f82fc19f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:06.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.446+0000 7f830392e700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82fc107a00 0x7f82fc19f460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f82f4007ee0 tx=0x7f82f400d3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.446+0000 7f830392e700 1 -- 172.21.15.150:0/3269410713 shutdown_connections 2024-05-10T04:02:06.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.446+0000 7f830392e700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82e80a1120 0x7f82e80a3630 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.446+0000 7f830392e700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82fc107a00 0x7f82fc19f460 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.446+0000 7f830392e700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82fc0fff30 0x7f82fc197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.446+0000 7f830392e700 1 --2- 172.21.15.150:0/3269410713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82fc1050a0 0x7f82fc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:06.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.446+0000 7f830392e700 1 -- 172.21.15.150:0/3269410713 >> 172.21.15.150:0/3269410713 conn(0x7f82fc0fb880 msgr2=0x7f82fc105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:06.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.447+0000 7f830392e700 1 -- 172.21.15.150:0/3269410713 shutdown_connections 2024-05-10T04:02:06.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:06.447+0000 7f830392e700 1 -- 172.21.15.150:0/3269410713 wait complete. 2024-05-10T04:02:06.450 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 1 2024-05-10T04:02:06.517 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:02:06 smithi181 systemd[1]: Starting Ceph mds.l for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:02:06.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:06 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1969887615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:06.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:06 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/604786234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:06.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:06 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3269410713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:06.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:06 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1969887615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:06.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:06 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/604786234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:06.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:06 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3269410713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:06.988 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:02:06 smithi181 ceph-mds[83363]: -- 172.21.15.181:0/2403882821 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5555f1a321a0 con 0x5555f1a92d00 2024-05-10T04:02:06.988 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:02:06 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-l[83340]: starting mds.l at 2024-05-10T04:02:06.988 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:02:06 smithi181 bash[83325]: 20b6e8af45d20f516f2831135863156539ba9cb8ce4667bf681468ffaddf88c0 2024-05-10T04:02:06.988 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:02:06 smithi181 systemd[1]: Started Ceph mds.l for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T04:02:06.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:06 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1969887615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:06.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:06 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/604786234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:06.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:06 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3269410713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:07.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:07 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.181:6832/2987418700,v1:172.21.15.181:6833/2987418700] up:boot 2024-05-10T04:02:07.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:07 smithi161 ceph-mon[46918]: fsmap 1 up:standby 2024-05-10T04:02:07.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T04:02:07.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:07.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:07.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:07.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:07.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:07.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:07 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:07.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:07 smithi161 ceph-mon[46918]: Deploying daemon mds.k on smithi161 2024-05-10T04:02:07.782 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:07 smithi161 ceph-mon[46918]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:07.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:07 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.181:6832/2987418700,v1:172.21.15.181:6833/2987418700] up:boot 2024-05-10T04:02:07.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:07 smithi150 ceph-mon[42477]: fsmap 1 up:standby 2024-05-10T04:02:07.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:07 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T04:02:07.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:07 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:07.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:07 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:07.965 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:07 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:07.965 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:07 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:07.965 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:07 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:07.965 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:07 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:07.965 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:07 smithi150 ceph-mon[42477]: Deploying daemon mds.k on smithi161 2024-05-10T04:02:07.965 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:07 smithi150 ceph-mon[42477]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:07.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:07 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.181:6832/2987418700,v1:172.21.15.181:6833/2987418700] up:boot 2024-05-10T04:02:07.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:07 smithi181 ceph-mon[46602]: fsmap 1 up:standby 2024-05-10T04:02:07.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:07 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T04:02:07.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:07 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:07.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:07 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:07.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:07 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:07.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:07 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:07.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:07 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:07.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:07 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:07.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:07 smithi181 ceph-mon[46602]: Deploying daemon mds.k on smithi161 2024-05-10T04:02:07.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:07 smithi181 ceph-mon[46602]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:08.033 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:02:08 smithi161 systemd[1]: Starting Ceph mds.k for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:02:08.463 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:02:08.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.626+0000 7ff00d4f9700 1 -- 172.21.15.150:0/1196590186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0081026b0 msgr2=0x7ff008104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:08.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.626+0000 7ff00d4f9700 1 --2- 172.21.15.150:0/1196590186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0081026b0 0x7ff008104ad0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feff8009990 tx=0x7feff8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:08.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.626+0000 7ff00d4f9700 1 -- 172.21.15.150:0/1196590186 shutdown_connections 2024-05-10T04:02:08.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.626+0000 7ff00d4f9700 1 --2- 172.21.15.150:0/1196590186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff008107970 0x7ff008109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:08.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.626+0000 7ff00d4f9700 1 --2- 172.21.15.150:0/1196590186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0081026b0 0x7ff008104ad0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:08.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.626+0000 7ff00d4f9700 1 --2- 172.21.15.150:0/1196590186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff008105010 0x7ff008107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:08.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.626+0000 7ff00d4f9700 1 -- 172.21.15.150:0/1196590186 >> 172.21.15.150:0/1196590186 conn(0x7ff0080fc0d0 msgr2=0x7ff0080fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:08.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.626+0000 7ff00d4f9700 1 -- 172.21.15.150:0/1196590186 shutdown_connections 2024-05-10T04:02:08.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.626+0000 7ff00d4f9700 1 -- 172.21.15.150:0/1196590186 wait complete. 2024-05-10T04:02:08.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.626+0000 7ff00d4f9700 1 Processor -- start 2024-05-10T04:02:08.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.626+0000 7ff00d4f9700 1 -- start start 2024-05-10T04:02:08.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff00d4f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0081026b0 0x7ff008197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:08.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff00d4f9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff008105010 0x7ff008198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:08.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff00d4f9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff008107970 0x7ff00819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:08.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff00d4f9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00819fad0 con 0x7ff008105010 2024-05-10T04:02:08.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff00d4f9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00819fc40 con 0x7ff008107970 2024-05-10T04:02:08.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff00d4f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00819fe00 con 0x7ff0081026b0 2024-05-10T04:02:08.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff0067fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff008105010 0x7ff008198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:08.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff006ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0081026b0 0x7ff008197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:08.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff0067fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff008105010 0x7ff008198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51474/0 (socket says 172.21.15.150:51474) 2024-05-10T04:02:08.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff0067fc700 1 -- 172.21.15.150:0/2732328098 learned_addr learned my addr 172.21.15.150:0/2732328098 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:08.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff0077fe700 1 --2- 172.21.15.150:0/2732328098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff008107970 0x7ff00819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:08.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff0067fc700 1 -- 172.21.15.150:0/2732328098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0081026b0 msgr2=0x7ff008197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:08.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff0067fc700 1 --2- 172.21.15.150:0/2732328098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0081026b0 0x7ff008197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:08.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff0067fc700 1 -- 172.21.15.150:0/2732328098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff008107970 msgr2=0x7ff00819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:08.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff0067fc700 1 --2- 172.21.15.150:0/2732328098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff008107970 0x7ff00819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:08.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff0067fc700 1 -- 172.21.15.150:0/2732328098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc009520 con 0x7ff008105010 2024-05-10T04:02:08.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7ff0067fc700 1 --2- 172.21.15.150:0/2732328098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff008105010 0x7ff008198530 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feffc00eb30 tx=0x7feffc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:08.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.627+0000 7fefeffff700 1 -- 172.21.15.150:0/2732328098 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc009880 con 0x7ff008105010 2024-05-10T04:02:08.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.628+0000 7ff00d4f9700 1 -- 172.21.15.150:0/2732328098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff80095f0 con 0x7ff008105010 2024-05-10T04:02:08.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.628+0000 7fefeffff700 1 -- 172.21.15.150:0/2732328098 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feffc0044e0 con 0x7ff008105010 2024-05-10T04:02:08.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.628+0000 7fefeffff700 1 -- 172.21.15.150:0/2732328098 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc003d80 con 0x7ff008105010 2024-05-10T04:02:08.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.628+0000 7ff00d4f9700 1 -- 172.21.15.150:0/2732328098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0081a0460 con 0x7ff008105010 2024-05-10T04:02:08.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.629+0000 7ff00d4f9700 1 -- 172.21.15.150:0/2732328098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0080fdc80 con 0x7ff008105010 2024-05-10T04:02:08.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.631+0000 7fefeffff700 1 -- 172.21.15.150:0/2732328098 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7feffc003f00 con 0x7ff008105010 2024-05-10T04:02:08.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.632+0000 7fefeffff700 1 --2- 172.21.15.150:0/2732328098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feff00a1120 0x7feff00a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:08.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.632+0000 7ff006ffd700 1 --2- 172.21.15.150:0/2732328098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feff00a1120 0x7feff00a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:08.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.632+0000 7fefeffff700 1 -- 172.21.15.150:0/2732328098 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feffc014070 con 0x7ff008105010 2024-05-10T04:02:08.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.632+0000 7ff006ffd700 1 --2- 172.21.15.150:0/2732328098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feff00a1120 0x7feff00a3630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feff800b2e0 tx=0x7feff801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:08.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.635+0000 7fefeffff700 1 -- 172.21.15.150:0/2732328098 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feffc096700 con 0x7ff008105010 2024-05-10T04:02:08.743 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:02:08 smithi161 bash[82471]: ba0cc00b4d61e14c9f4e3d3e6f3f3ba14c59e40820d883602e3b8c4c7c239a4b 2024-05-10T04:02:08.744 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:02:08 smithi161 systemd[1]: Started Ceph mds.k for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T04:02:08.744 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:02:08 smithi161 ceph-mds[82509]: -- 172.21.15.161:0/2510133426 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56236b46e1a0 con 0x56236b4ced00 2024-05-10T04:02:08.744 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:02:08 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-k[82486]: starting mds.k at 2024-05-10T04:02:08.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.898+0000 7ff00d4f9700 1 -- 172.21.15.150:0/2732328098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0080694d0 con 0x7ff008105010 2024-05-10T04:02:08.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.898+0000 7fefeffff700 1 -- 172.21.15.150:0/2732328098 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7feffc00fe60 con 0x7ff008105010 2024-05-10T04:02:08.901 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:08.901 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-10T04:02:08.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.901+0000 7ff00d4f9700 1 -- 172.21.15.150:0/2732328098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feff00a1120 msgr2=0x7feff00a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:08.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.901+0000 7ff00d4f9700 1 --2- 172.21.15.150:0/2732328098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feff00a1120 0x7feff00a3630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feff800b2e0 tx=0x7feff801a040 comp rx=0 tx=0).stop 2024-05-10T04:02:08.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.901+0000 7ff00d4f9700 1 -- 172.21.15.150:0/2732328098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff008105010 msgr2=0x7ff008198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:08.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.901+0000 7ff00d4f9700 1 --2- 172.21.15.150:0/2732328098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff008105010 0x7ff008198530 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feffc00eb30 tx=0x7feffc00ee40 comp rx=0 tx=0).stop 2024-05-10T04:02:08.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.901+0000 7ff00d4f9700 1 -- 172.21.15.150:0/2732328098 shutdown_connections 2024-05-10T04:02:08.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.901+0000 7ff00d4f9700 1 --2- 172.21.15.150:0/2732328098 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feff00a1120 0x7feff00a3630 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:08.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.901+0000 7ff00d4f9700 1 --2- 172.21.15.150:0/2732328098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff008107970 0x7ff00819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:08.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.901+0000 7ff00d4f9700 1 --2- 172.21.15.150:0/2732328098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0081026b0 0x7ff008197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:08.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.901+0000 7ff00d4f9700 1 --2- 172.21.15.150:0/2732328098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff008105010 0x7ff008198530 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:08.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.901+0000 7ff00d4f9700 1 -- 172.21.15.150:0/2732328098 >> 172.21.15.150:0/2732328098 conn(0x7ff0080fc0d0 msgr2=0x7ff008104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:08.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.901+0000 7ff00d4f9700 1 -- 172.21.15.150:0/2732328098 shutdown_connections 2024-05-10T04:02:08.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:08.901+0000 7ff00d4f9700 1 -- 172.21.15.150:0/2732328098 wait complete. 2024-05-10T04:02:08.907 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 3 2024-05-10T04:02:09.722 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:09.722 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:09.722 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:09.722 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:09.722 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:09.722 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:09.722 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:09 smithi150 ceph-mon[42477]: Deploying daemon mds.j on smithi150 2024-05-10T04:02:09.722 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:09 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.161:6832/3107886390,v1:172.21.15.161:6833/3107886390] up:boot 2024-05-10T04:02:09.723 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:09 smithi150 ceph-mon[42477]: fsmap 2 up:standby 2024-05-10T04:02:09.723 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:09 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T04:02:09.723 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:09 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2732328098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:09.723 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:09 smithi150 ceph-mon[42477]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:09.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:09.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:09.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:09.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:09.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:09.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:09.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:09 smithi181 ceph-mon[46602]: Deploying daemon mds.j on smithi150 2024-05-10T04:02:09.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:09 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.161:6832/3107886390,v1:172.21.15.161:6833/3107886390] up:boot 2024-05-10T04:02:09.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:09 smithi181 ceph-mon[46602]: fsmap 2 up:standby 2024-05-10T04:02:09.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:09 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T04:02:09.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:09 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2732328098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:09.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:09 smithi181 ceph-mon[46602]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:09.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:09.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:09.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:09.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:09.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:09.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:09.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:09 smithi161 ceph-mon[46918]: Deploying daemon mds.j on smithi150 2024-05-10T04:02:09.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:09 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.161:6832/3107886390,v1:172.21.15.161:6833/3107886390] up:boot 2024-05-10T04:02:09.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:09 smithi161 ceph-mon[46918]: fsmap 2 up:standby 2024-05-10T04:02:09.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:09 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T04:02:09.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:09 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2732328098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:09.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:09 smithi161 ceph-mon[46918]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:09.994 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:02:09 smithi150 systemd[1]: Starting Ceph mds.j for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:02:10.463 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:02:10 smithi150 bash[91786]: 5167a669b4ceade656e5a6dc96da1047e9bd4e55e398eb6c67fa24d1a6f830e1 2024-05-10T04:02:10.464 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:02:10 smithi150 systemd[1]: Started Ceph mds.j for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T04:02:10.464 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:02:10 smithi150 ceph-mds[91824]: -- 172.21.15.150:0/1300247680 <== mon.0 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cf718061a0 con 0x55cf71866880 2024-05-10T04:02:10.464 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:02:10 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-j[91801]: starting mds.j at 2024-05-10T04:02:10.923 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:02:11.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.061+0000 7f2fec61e700 1 -- 172.21.15.150:0/2557616961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe4102030 msgr2=0x7f2fe4102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:11.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.061+0000 7f2fec61e700 1 --2- 172.21.15.150:0/2557616961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe4102030 0x7f2fe4102460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009990 tx=0x7f2fe0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.061+0000 7f2fec61e700 1 -- 172.21.15.150:0/2557616961 shutdown_connections 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.061+0000 7f2fec61e700 1 --2- 172.21.15.150:0/2557616961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe4101680 0x7f2fe4108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.061+0000 7f2fec61e700 1 --2- 172.21.15.150:0/2557616961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe4102030 0x7f2fe4102460 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.061+0000 7f2fec61e700 1 --2- 172.21.15.150:0/2557616961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fe4100d30 0x7f2fe4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.061+0000 7f2fec61e700 1 -- 172.21.15.150:0/2557616961 >> 172.21.15.150:0/2557616961 conn(0x7f2fe40fb8f0 msgr2=0x7f2fe40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.061+0000 7f2fec61e700 1 -- 172.21.15.150:0/2557616961 shutdown_connections 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.061+0000 7f2fec61e700 1 -- 172.21.15.150:0/2557616961 wait complete. 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.062+0000 7f2fec61e700 1 Processor -- start 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.062+0000 7f2fec61e700 1 -- start start 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.062+0000 7f2fec61e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe4100d30 0x7f2fe4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.062+0000 7f2fec61e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fe4101680 0x7f2fe41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.062+0000 7f2fec61e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe4102030 0x7f2fe419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.062+0000 7f2fec61e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe419fa80 con 0x7f2fe4100d30 2024-05-10T04:02:11.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.062+0000 7f2fec61e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe419fbf0 con 0x7f2fe4101680 2024-05-10T04:02:11.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.062+0000 7f2fec61e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe419fdb0 con 0x7f2fe4102030 2024-05-10T04:02:11.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.062+0000 7f2fea3ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe4100d30 0x7f2fe4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:11.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.062+0000 7f2fea3ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe4100d30 0x7f2fe4197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59144/0 (socket says 172.21.15.150:59144) 2024-05-10T04:02:11.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.062+0000 7f2fea3ba700 1 -- 172.21.15.150:0/3293825235 learned_addr learned my addr 172.21.15.150:0/3293825235 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:11.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.062+0000 7f2feabbb700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe4102030 0x7f2fe419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:11.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.063+0000 7f2fe9bb9700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fe4101680 0x7f2fe41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:11.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.063+0000 7f2fea3ba700 1 -- 172.21.15.150:0/3293825235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe4102030 msgr2=0x7f2fe419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:11.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.063+0000 7f2fea3ba700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe4102030 0x7f2fe419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:11.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.063+0000 7f2fea3ba700 1 -- 172.21.15.150:0/3293825235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fe4101680 msgr2=0x7f2fe41984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:11.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.063+0000 7f2fea3ba700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fe4101680 0x7f2fe41984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:11.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.063+0000 7f2fea3ba700 1 -- 172.21.15.150:0/3293825235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe00095f0 con 0x7f2fe4100d30 2024-05-10T04:02:11.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.063+0000 7f2fea3ba700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe4100d30 0x7f2fe4197fa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0006010 tx=0x7f2fe000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:11.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.063+0000 7f2fdb7fe700 1 -- 172.21.15.150:0/3293825235 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe001d070 con 0x7f2fe4100d30 2024-05-10T04:02:11.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.063+0000 7f2fdb7fe700 1 -- 172.21.15.150:0/3293825235 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fe0022470 con 0x7f2fe4100d30 2024-05-10T04:02:11.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.063+0000 7f2fec61e700 1 -- 172.21.15.150:0/3293825235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe41a0030 con 0x7f2fe4100d30 2024-05-10T04:02:11.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.063+0000 7f2fdb7fe700 1 -- 172.21.15.150:0/3293825235 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0017650 con 0x7f2fe4100d30 2024-05-10T04:02:11.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.063+0000 7f2fec61e700 1 -- 172.21.15.150:0/3293825235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe41a04b0 con 0x7f2fe4100d30 2024-05-10T04:02:11.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.064+0000 7f2fec61e700 1 -- 172.21.15.150:0/3293825235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe40678b0 con 0x7f2fe4100d30 2024-05-10T04:02:11.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.065+0000 7f2fdb7fe700 1 -- 172.21.15.150:0/3293825235 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2fe00225e0 con 0x7f2fe4100d30 2024-05-10T04:02:11.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.066+0000 7f2fdb7fe700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2fd00a11a0 0x7f2fd00a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:11.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.066+0000 7f2fe9bb9700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2fd00a11a0 0x7f2fd00a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:11.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.066+0000 7f2fdb7fe700 1 -- 172.21.15.150:0/3293825235 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2fe00cd9d0 con 0x7f2fe4100d30 2024-05-10T04:02:11.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.067+0000 7f2fe9bb9700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2fd00a11a0 0x7f2fd00a36b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4199040 tx=0x7f2fd400b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:11.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.070+0000 7f2fdb7fe700 1 -- 172.21.15.150:0/3293825235 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2fe00976c0 con 0x7f2fe4100d30 2024-05-10T04:02:11.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.331+0000 7f2fec61e700 1 -- 172.21.15.150:0/3293825235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fe410c2a0 con 0x7f2fe4100d30 2024-05-10T04:02:11.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.332+0000 7f2fdb7fe700 1 -- 172.21.15.150:0/3293825235 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2780 (secure 0 0 0) 0x7f2fe009ace0 con 0x7f2fe4100d30 2024-05-10T04:02:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:11.334 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-10T04:02:11.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.334+0000 7f2fec61e700 1 -- 172.21.15.150:0/3293825235 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2fd00a11a0 msgr2=0x7f2fd00a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:11.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.334+0000 7f2fec61e700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2fd00a11a0 0x7f2fd00a36b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4199040 tx=0x7f2fd400b5d0 comp rx=0 tx=0).stop 2024-05-10T04:02:11.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.334+0000 7f2fec61e700 1 -- 172.21.15.150:0/3293825235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe4100d30 msgr2=0x7f2fe4197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:11.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.334+0000 7f2fec61e700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe4100d30 0x7f2fe4197fa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0006010 tx=0x7f2fe000f690 comp rx=0 tx=0).stop 2024-05-10T04:02:11.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.334+0000 7f2fec61e700 1 -- 172.21.15.150:0/3293825235 shutdown_connections 2024-05-10T04:02:11.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.334+0000 7f2fec61e700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2fd00a11a0 0x7f2fd00a36b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:11.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.334+0000 7f2fec61e700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe4102030 0x7f2fe419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:11.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.334+0000 7f2fec61e700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe4100d30 0x7f2fe4197fa0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:11.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.334+0000 7f2fec61e700 1 --2- 172.21.15.150:0/3293825235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fe4101680 0x7f2fe41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:11.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.334+0000 7f2fec61e700 1 -- 172.21.15.150:0/3293825235 >> 172.21.15.150:0/3293825235 conn(0x7f2fe40fb8f0 msgr2=0x7f2fe40fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:11.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.334+0000 7f2fec61e700 1 -- 172.21.15.150:0/3293825235 shutdown_connections 2024-05-10T04:02:11.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:11.334+0000 7f2fec61e700 1 -- 172.21.15.150:0/3293825235 wait complete. 2024-05-10T04:02:11.338 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 4 2024-05-10T04:02:11.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:11 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:11.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:11 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:11.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:11 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:11.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:11 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:11.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:11 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:11.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:11 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:11.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:11 smithi150 ceph-mon[42477]: Deploying daemon mds.d on smithi150 2024-05-10T04:02:11.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:11 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.150:6834/1387516089,v1:172.21.15.150:6835/1387516089] up:boot 2024-05-10T04:02:11.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:11 smithi150 ceph-mon[42477]: fsmap 3 up:standby 2024-05-10T04:02:11.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:11 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T04:02:11.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:11 smithi150 ceph-mon[42477]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:11 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:11 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:11 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:11 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:11 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:11 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:11 smithi181 ceph-mon[46602]: Deploying daemon mds.d on smithi150 2024-05-10T04:02:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:11 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.150:6834/1387516089,v1:172.21.15.150:6835/1387516089] up:boot 2024-05-10T04:02:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:11 smithi181 ceph-mon[46602]: fsmap 3 up:standby 2024-05-10T04:02:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:11 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T04:02:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:11 smithi181 ceph-mon[46602]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:11 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:11 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:11 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:11 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:11 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:11 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:11 smithi161 ceph-mon[46918]: Deploying daemon mds.d on smithi150 2024-05-10T04:02:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:11 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.150:6834/1387516089,v1:172.21.15.150:6835/1387516089] up:boot 2024-05-10T04:02:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:11 smithi161 ceph-mon[46918]: fsmap 3 up:standby 2024-05-10T04:02:11.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:11 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T04:02:11.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:11 smithi161 ceph-mon[46918]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:11.907 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:02:11 smithi150 systemd[1]: Starting Ceph mds.d for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:02:12.190 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:02:11 smithi150 bash[92174]: 4bc2c32833e0b4d2bdcb9fbe4b6770cbdc7e78f38bd416f1e2453458b4b1843e 2024-05-10T04:02:12.190 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:02:11 smithi150 systemd[1]: Started Ceph mds.d for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T04:02:12.190 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:02:11 smithi150 ceph-mds[92213]: -- 172.21.15.150:0/944252167 <== mon.0 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559f0c6361a0 con 0x559f0c696880 2024-05-10T04:02:12.190 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:02:11 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-d[92189]: starting mds.d at 2024-05-10T04:02:12.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:12 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3293825235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:12.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:12 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:12.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:12 smithi150 ceph-mon[42477]: Deploying daemon mds.e on smithi161 2024-05-10T04:02:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:12 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3293825235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:12 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:12 smithi181 ceph-mon[46602]: Deploying daemon mds.e on smithi161 2024-05-10T04:02:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:12 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3293825235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:12 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:12.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:12 smithi161 ceph-mon[46918]: Deploying daemon mds.e on smithi161 2024-05-10T04:02:13.351 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:02:13.493 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:02:13 smithi161 systemd[1]: Starting Ceph mds.e for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:02:13.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.502+0000 7f3c02c13700 1 -- 172.21.15.150:0/3115540582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bfc101c90 msgr2=0x7f3bfc1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:13.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.502+0000 7f3c02c13700 1 --2- 172.21.15.150:0/3115540582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bfc101c90 0x7f3bfc1020a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3be8009990 tx=0x7f3be8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:13.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.502+0000 7f3c02c13700 1 -- 172.21.15.150:0/3115540582 shutdown_connections 2024-05-10T04:02:13.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.502+0000 7f3c02c13700 1 --2- 172.21.15.150:0/3115540582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bfc102630 0x7f3bfc108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:13.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.502+0000 7f3c02c13700 1 --2- 172.21.15.150:0/3115540582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bfc100b70 0x7f3bfc100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:13.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.502+0000 7f3c02c13700 1 --2- 172.21.15.150:0/3115540582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bfc101c90 0x7f3bfc1020a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:13.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.502+0000 7f3c02c13700 1 -- 172.21.15.150:0/3115540582 >> 172.21.15.150:0/3115540582 conn(0x7f3bfc0fc180 msgr2=0x7f3bfc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:13.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.502+0000 7f3c02c13700 1 -- 172.21.15.150:0/3115540582 shutdown_connections 2024-05-10T04:02:13.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.502+0000 7f3c02c13700 1 -- 172.21.15.150:0/3115540582 wait complete. 2024-05-10T04:02:13.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.503+0000 7f3c02c13700 1 Processor -- start 2024-05-10T04:02:13.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.503+0000 7f3c02c13700 1 -- start start 2024-05-10T04:02:13.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.503+0000 7f3c02c13700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bfc100b70 0x7f3bfc196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:13.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.503+0000 7f3c02c13700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bfc101c90 0x7f3bfc197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:13.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c02c13700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bfc102630 0x7f3bfc19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:13.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c02c13700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc19e9e0 con 0x7f3bfc100b70 2024-05-10T04:02:13.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c02c13700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc19eb50 con 0x7f3bfc102630 2024-05-10T04:02:13.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c02c13700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc19ed10 con 0x7f3bfc101c90 2024-05-10T04:02:13.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c009af700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bfc100b70 0x7f3bfc196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:13.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c009af700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bfc100b70 0x7f3bfc196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59170/0 (socket says 172.21.15.150:59170) 2024-05-10T04:02:13.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c009af700 1 -- 172.21.15.150:0/685777774 learned_addr learned my addr 172.21.15.150:0/685777774 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:13.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c011b0700 1 --2- 172.21.15.150:0/685777774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bfc102630 0x7f3bfc19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:13.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c009af700 1 -- 172.21.15.150:0/685777774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bfc101c90 msgr2=0x7f3bfc197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:13.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c009af700 1 --2- 172.21.15.150:0/685777774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bfc101c90 0x7f3bfc197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:13.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c009af700 1 -- 172.21.15.150:0/685777774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bfc102630 msgr2=0x7f3bfc19e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:13.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c009af700 1 --2- 172.21.15.150:0/685777774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bfc102630 0x7f3bfc19e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:13.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c009af700 1 -- 172.21.15.150:0/685777774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be80095f0 con 0x7f3bfc100b70 2024-05-10T04:02:13.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c009af700 1 --2- 172.21.15.150:0/685777774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bfc100b70 0x7f3bfc196f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000d950 tx=0x7f3bf000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:13.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3bf9ffb700 1 -- 172.21.15.150:0/685777774 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0004220 con 0x7f3bfc100b70 2024-05-10T04:02:13.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3bf9ffb700 1 -- 172.21.15.150:0/685777774 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bf0004380 con 0x7f3bfc100b70 2024-05-10T04:02:13.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c02c13700 1 -- 172.21.15.150:0/685777774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bfc19eff0 con 0x7f3bfc100b70 2024-05-10T04:02:13.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3bf9ffb700 1 -- 172.21.15.150:0/685777774 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0003bf0 con 0x7f3bfc100b70 2024-05-10T04:02:13.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.504+0000 7f3c02c13700 1 -- 172.21.15.150:0/685777774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bfc19f470 con 0x7f3bfc100b70 2024-05-10T04:02:13.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.506+0000 7f3c02c13700 1 -- 172.21.15.150:0/685777774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bfc107d80 con 0x7f3bfc100b70 2024-05-10T04:02:13.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.507+0000 7f3bf9ffb700 1 -- 172.21.15.150:0/685777774 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3bf0003d50 con 0x7f3bfc100b70 2024-05-10T04:02:13.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.507+0000 7f3bf9ffb700 1 --2- 172.21.15.150:0/685777774 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3bec0a1120 0x7f3bec0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:13.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.508+0000 7f3bfbfff700 1 --2- 172.21.15.150:0/685777774 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3bec0a1120 0x7f3bec0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:13.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.508+0000 7f3bf9ffb700 1 -- 172.21.15.150:0/685777774 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3bf00cbf30 con 0x7f3bfc100b70 2024-05-10T04:02:13.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.508+0000 7f3bfbfff700 1 --2- 172.21.15.150:0/685777774 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3bec0a1120 0x7f3bec0a3630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3be80098e0 tx=0x7f3be8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:13.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.512+0000 7f3bf9ffb700 1 -- 172.21.15.150:0/685777774 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3bf0095b60 con 0x7f3bfc100b70 2024-05-10T04:02:13.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.763+0000 7f3c02c13700 1 -- 172.21.15.150:0/685777774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bfc0678b0 con 0x7f3bfc100b70 2024-05-10T04:02:13.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.763+0000 7f3bf9ffb700 1 -- 172.21.15.150:0/685777774 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3540 (secure 0 0 0) 0x7f3bf0017df0 con 0x7f3bfc100b70 2024-05-10T04:02:13.766 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:13.766 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-10T04:02:13.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.766+0000 7f3c02c13700 1 -- 172.21.15.150:0/685777774 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3bec0a1120 msgr2=0x7f3bec0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:13.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.766+0000 7f3c02c13700 1 --2- 172.21.15.150:0/685777774 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3bec0a1120 0x7f3bec0a3630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3be80098e0 tx=0x7f3be8005fb0 comp rx=0 tx=0).stop 2024-05-10T04:02:13.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.766+0000 7f3c02c13700 1 -- 172.21.15.150:0/685777774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bfc100b70 msgr2=0x7f3bfc196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:13.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.766+0000 7f3c02c13700 1 --2- 172.21.15.150:0/685777774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bfc100b70 0x7f3bfc196f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000d950 tx=0x7f3bf000dc60 comp rx=0 tx=0).stop 2024-05-10T04:02:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.766+0000 7f3c02c13700 1 -- 172.21.15.150:0/685777774 shutdown_connections 2024-05-10T04:02:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.766+0000 7f3c02c13700 1 --2- 172.21.15.150:0/685777774 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3bec0a1120 0x7f3bec0a3630 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.766+0000 7f3c02c13700 1 --2- 172.21.15.150:0/685777774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bfc102630 0x7f3bfc19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.766+0000 7f3c02c13700 1 --2- 172.21.15.150:0/685777774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bfc100b70 0x7f3bfc196f00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.766+0000 7f3c02c13700 1 --2- 172.21.15.150:0/685777774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bfc101c90 0x7f3bfc197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.766+0000 7f3c02c13700 1 -- 172.21.15.150:0/685777774 >> 172.21.15.150:0/685777774 conn(0x7f3bfc0fc180 msgr2=0x7f3bfc104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.766+0000 7f3c02c13700 1 -- 172.21.15.150:0/685777774 shutdown_connections 2024-05-10T04:02:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:13.766+0000 7f3c02c13700 1 -- 172.21.15.150:0/685777774 wait complete. 2024-05-10T04:02:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 5 2024-05-10T04:02:13.992 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:02:13 smithi161 ceph-mds[82847]: -- 172.21.15.161:0/2557108079 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ed336dc1a0 con 0x55ed3373c880 2024-05-10T04:02:13.992 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:02:13 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-e[82824]: starting mds.e at 2024-05-10T04:02:13.992 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:02:13 smithi161 bash[82809]: 53cb2b8009a2c442c8ebf5b21ab367253aea525ffae01f08a80694bbada9cc91 2024-05-10T04:02:13.992 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:02:13 smithi161 systemd[1]: Started Ceph mds.e for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T04:02:13.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:13 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.150:6836/3462983502,v1:172.21.15.150:6837/3462983502] up:boot 2024-05-10T04:02:13.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:13 smithi161 ceph-mon[46918]: fsmap 4 up:standby 2024-05-10T04:02:13.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T04:02:13.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:13 smithi161 ceph-mon[46918]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:13.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:13.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:13.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:13.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:13.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:13.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:13 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:13.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:13 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/685777774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:14.147 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:13 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.150:6836/3462983502,v1:172.21.15.150:6837/3462983502] up:boot 2024-05-10T04:02:14.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:13 smithi181 ceph-mon[46602]: fsmap 4 up:standby 2024-05-10T04:02:14.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T04:02:14.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:13 smithi181 ceph-mon[46602]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:14.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:14.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:14.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:14.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:14.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:14.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:13 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:14.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:13 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/685777774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:14.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:13 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.150:6836/3462983502,v1:172.21.15.150:6837/3462983502] up:boot 2024-05-10T04:02:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:13 smithi150 ceph-mon[42477]: fsmap 4 up:standby 2024-05-10T04:02:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T04:02:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:13 smithi150 ceph-mon[42477]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:13 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:14.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:13 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/685777774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:14.982 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:14 smithi181 ceph-mon[46602]: Deploying daemon mds.f on smithi181 2024-05-10T04:02:14.982 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:14 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.161:6834/1403383813,v1:172.21.15.161:6835/1403383813] up:boot 2024-05-10T04:02:14.983 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:14 smithi181 ceph-mon[46602]: fsmap 5 up:standby 2024-05-10T04:02:14.983 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:14 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T04:02:15.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:14 smithi150 ceph-mon[42477]: Deploying daemon mds.f on smithi181 2024-05-10T04:02:15.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:14 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.161:6834/1403383813,v1:172.21.15.161:6835/1403383813] up:boot 2024-05-10T04:02:15.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:14 smithi150 ceph-mon[42477]: fsmap 5 up:standby 2024-05-10T04:02:15.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:14 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T04:02:15.237 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:02:14 smithi181 systemd[1]: Starting Ceph mds.f for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:02:15.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:14 smithi161 ceph-mon[46918]: Deploying daemon mds.f on smithi181 2024-05-10T04:02:15.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:14 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.161:6834/1403383813,v1:172.21.15.161:6835/1403383813] up:boot 2024-05-10T04:02:15.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:14 smithi161 ceph-mon[46918]: fsmap 5 up:standby 2024-05-10T04:02:15.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:14 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T04:02:15.737 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:02:15 smithi181 bash[83666]: 07ad2d26b3d69ac44908196563d00aa890d8c855a8922599ea229edb3996a1fd 2024-05-10T04:02:15.738 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:02:15 smithi181 systemd[1]: Started Ceph mds.f for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T04:02:15.738 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:02:15 smithi181 ceph-mds[83703]: -- 172.21.15.181:0/3785664731 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e4931c61a0 con 0x55e493226880 2024-05-10T04:02:15.738 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:02:15 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-f[83681]: starting mds.f at 2024-05-10T04:02:15.784 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:02:15.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.931+0000 7fd2cb049700 1 -- 172.21.15.150:0/1191705499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c4100ac0 msgr2=0x7fd2c4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:15.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.931+0000 7fd2cb049700 1 --2- 172.21.15.150:0/1191705499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c4100ac0 0x7fd2c4100ed0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009990 tx=0x7fd2b8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:15.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.931+0000 7fd2cb049700 1 -- 172.21.15.150:0/1191705499 shutdown_connections 2024-05-10T04:02:15.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.931+0000 7fd2cb049700 1 --2- 172.21.15.150:0/1191705499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c4102580 0x7fd2c4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:15.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.931+0000 7fd2cb049700 1 --2- 172.21.15.150:0/1191705499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c4100ac0 0x7fd2c4100ed0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:15.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.931+0000 7fd2cb049700 1 --2- 172.21.15.150:0/1191705499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2c4101be0 0x7fd2c4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:15.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.931+0000 7fd2cb049700 1 -- 172.21.15.150:0/1191705499 >> 172.21.15.150:0/1191705499 conn(0x7fd2c40fc0d0 msgr2=0x7fd2c40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:15.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.931+0000 7fd2cb049700 1 -- 172.21.15.150:0/1191705499 shutdown_connections 2024-05-10T04:02:15.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.931+0000 7fd2cb049700 1 -- 172.21.15.150:0/1191705499 wait complete. 2024-05-10T04:02:15.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.931+0000 7fd2cb049700 1 Processor -- start 2024-05-10T04:02:15.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.931+0000 7fd2cb049700 1 -- start start 2024-05-10T04:02:15.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2cb049700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c4100ac0 0x7fd2c4068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:15.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2cb049700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2c4101be0 0x7fd2c419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:15.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2cb049700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c4102580 0x7fd2c419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:15.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2cb049700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c419fa00 con 0x7fd2c4100ac0 2024-05-10T04:02:15.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2cb049700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c419fb70 con 0x7fd2c4101be0 2024-05-10T04:02:15.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2cb049700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c419fd30 con 0x7fd2c4102580 2024-05-10T04:02:15.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c8de5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c4100ac0 0x7fd2c4068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:15.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c8de5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c4100ac0 0x7fd2c4068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59202/0 (socket says 172.21.15.150:59202) 2024-05-10T04:02:15.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c8de5700 1 -- 172.21.15.150:0/1574191302 learned_addr learned my addr 172.21.15.150:0/1574191302 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:15.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c95e6700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c4102580 0x7fd2c419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:15.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c3fff700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2c4101be0 0x7fd2c419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:15.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c8de5700 1 -- 172.21.15.150:0/1574191302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c4102580 msgr2=0x7fd2c419f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:15.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c8de5700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c4102580 0x7fd2c419f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:15.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c8de5700 1 -- 172.21.15.150:0/1574191302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2c4101be0 msgr2=0x7fd2c419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:15.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c8de5700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2c4101be0 0x7fd2c419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:15.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c8de5700 1 -- 172.21.15.150:0/1574191302 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b80095f0 con 0x7fd2c4100ac0 2024-05-10T04:02:15.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c8de5700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c4100ac0 0x7fd2c4068e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80038c0 tx=0x7fd2b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:15.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c1ffb700 1 -- 172.21.15.150:0/1574191302 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b801d070 con 0x7fd2c4100ac0 2024-05-10T04:02:15.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c1ffb700 1 -- 172.21.15.150:0/1574191302 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b8003960 con 0x7fd2c4100ac0 2024-05-10T04:02:15.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2cb049700 1 -- 172.21.15.150:0/1574191302 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c419ffb0 con 0x7fd2c4100ac0 2024-05-10T04:02:15.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.932+0000 7fd2c1ffb700 1 -- 172.21.15.150:0/1574191302 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8005560 con 0x7fd2c4100ac0 2024-05-10T04:02:15.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.933+0000 7fd2cb049700 1 -- 172.21.15.150:0/1574191302 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c41a0320 con 0x7fd2c4100ac0 2024-05-10T04:02:15.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.934+0000 7fd2cb049700 1 -- 172.21.15.150:0/1574191302 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c4107cd0 con 0x7fd2c4100ac0 2024-05-10T04:02:15.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.935+0000 7fd2c1ffb700 1 -- 172.21.15.150:0/1574191302 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd2b8003ad0 con 0x7fd2c4100ac0 2024-05-10T04:02:15.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.936+0000 7fd2c1ffb700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd2b40a1120 0x7fd2b40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:15.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.936+0000 7fd2c3fff700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd2b40a1120 0x7fd2b40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:15.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.936+0000 7fd2c1ffb700 1 -- 172.21.15.150:0/1574191302 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd2b80cd6e0 con 0x7fd2c4100ac0 2024-05-10T04:02:15.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.936+0000 7fd2c3fff700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd2b40a1120 0x7fd2b40a3630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0005ea0 tx=0x7fd2b0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:15.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:15.940+0000 7fd2c1ffb700 1 -- 172.21.15.150:0/1574191302 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd2b80973a0 con 0x7fd2c4100ac0 2024-05-10T04:02:16.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:15 smithi181 ceph-mon[46602]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:16.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:16.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:16.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:16.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:16.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:16.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:15 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:16.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.181+0000 7fd2cb049700 1 -- 172.21.15.150:0/1574191302 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2c40694d0 con 0x7fd2c4100ac0 2024-05-10T04:02:16.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.181+0000 7fd2c1ffb700 1 -- 172.21.15.150:0/1574191302 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5057 (secure 0 0 0) 0x7fd2b809a9c0 con 0x7fd2c4100ac0 2024-05-10T04:02:16.184 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:16.184 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2024-05-10T04:02:16.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.184+0000 7fd2cb049700 1 -- 172.21.15.150:0/1574191302 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd2b40a1120 msgr2=0x7fd2b40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:16.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.184+0000 7fd2cb049700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd2b40a1120 0x7fd2b40a3630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0005ea0 tx=0x7fd2b0005e30 comp rx=0 tx=0).stop 2024-05-10T04:02:16.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.184+0000 7fd2cb049700 1 -- 172.21.15.150:0/1574191302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c4100ac0 msgr2=0x7fd2c4068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:16.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.184+0000 7fd2cb049700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c4100ac0 0x7fd2c4068e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80038c0 tx=0x7fd2b800f690 comp rx=0 tx=0).stop 2024-05-10T04:02:16.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.184+0000 7fd2cb049700 1 -- 172.21.15.150:0/1574191302 shutdown_connections 2024-05-10T04:02:16.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.184+0000 7fd2cb049700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd2b40a1120 0x7fd2b40a3630 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:16.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.184+0000 7fd2cb049700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c4102580 0x7fd2c419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:16.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.184+0000 7fd2cb049700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c4100ac0 0x7fd2c4068e50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:16.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.184+0000 7fd2cb049700 1 --2- 172.21.15.150:0/1574191302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2c4101be0 0x7fd2c419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:16.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.184+0000 7fd2cb049700 1 -- 172.21.15.150:0/1574191302 >> 172.21.15.150:0/1574191302 conn(0x7fd2c40fc0d0 msgr2=0x7fd2c4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:16.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.184+0000 7fd2cb049700 1 -- 172.21.15.150:0/1574191302 shutdown_connections 2024-05-10T04:02:16.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:16.184+0000 7fd2cb049700 1 -- 172.21.15.150:0/1574191302 wait complete. 2024-05-10T04:02:16.187 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 7 2024-05-10T04:02:16.198 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:15 smithi150 ceph-mon[42477]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:16.198 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:16.198 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:16.198 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:16.198 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:16.198 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:16.198 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:15 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:16.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:15 smithi161 ceph-mon[46918]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:16.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:16.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:16.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:16.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:16.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:15 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:16.739 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:02:16 smithi181 systemd[1]: Starting Ceph mds.c for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:02:17.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:16 smithi181 ceph-mon[46602]: Deploying daemon mds.c on smithi181 2024-05-10T04:02:17.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:16 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.181:6834/501599730,v1:172.21.15.181:6835/501599730] up:boot 2024-05-10T04:02:17.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:16 smithi181 ceph-mon[46602]: fsmap 6 up:standby 2024-05-10T04:02:17.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:16 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T04:02:17.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:16 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1574191302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:17.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:16 smithi150 ceph-mon[42477]: Deploying daemon mds.c on smithi181 2024-05-10T04:02:17.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:16 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.181:6834/501599730,v1:172.21.15.181:6835/501599730] up:boot 2024-05-10T04:02:17.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:16 smithi150 ceph-mon[42477]: fsmap 6 up:standby 2024-05-10T04:02:17.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:16 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T04:02:17.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:16 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1574191302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:17.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:16 smithi161 ceph-mon[46918]: Deploying daemon mds.c on smithi181 2024-05-10T04:02:17.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:16 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.181:6834/501599730,v1:172.21.15.181:6835/501599730] up:boot 2024-05-10T04:02:17.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:16 smithi161 ceph-mon[46918]: fsmap 6 up:standby 2024-05-10T04:02:17.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:16 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T04:02:17.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:16 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1574191302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:17.488 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:02:17 smithi181 bash[84008]: 80bce216878631a10c4498ef5205cf39b3d07049d73ef0d85856aeda1da8f523 2024-05-10T04:02:17.488 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:02:17 smithi181 systemd[1]: Started Ceph mds.c for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T04:02:17.488 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:02:17 smithi181 ceph-mds[84046]: -- 172.21.15.181:0/1622703307 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e375f481a0 con 0x55e375fa8d00 2024-05-10T04:02:17.488 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:02:17 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-c[84023]: starting mds.c at 2024-05-10T04:02:17.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:17 smithi161 ceph-mon[46918]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:17.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:17 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:17.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:17 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:17.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:17 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:17.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:17 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:17.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:17 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:17.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:17 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:18.200 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:02:18.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:17 smithi150 ceph-mon[42477]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:18.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:17 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:18.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:17 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:18.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:17 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:18.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:17 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:18.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:17 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:18.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:17 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:18.237 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:17 smithi181 ceph-mon[46602]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:18.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:17 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:18.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:17 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:18.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:17 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:18.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:17 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:18.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:17 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:18.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:17 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:18.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.344+0000 7fbd1416e700 1 -- 172.21.15.150:0/863853850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0c101c90 msgr2=0x7fbd0c1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:18.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.344+0000 7fbd1416e700 1 --2- 172.21.15.150:0/863853850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0c101c90 0x7fbd0c1020a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009990 tx=0x7fbd08009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:18.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.344+0000 7fbd1416e700 1 -- 172.21.15.150:0/863853850 shutdown_connections 2024-05-10T04:02:18.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.344+0000 7fbd1416e700 1 --2- 172.21.15.150:0/863853850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd0c102630 0x7fbd0c108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:18.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.344+0000 7fbd1416e700 1 --2- 172.21.15.150:0/863853850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c100b70 0x7fbd0c100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:18.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.344+0000 7fbd1416e700 1 --2- 172.21.15.150:0/863853850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0c101c90 0x7fbd0c1020a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:18.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.344+0000 7fbd1416e700 1 -- 172.21.15.150:0/863853850 >> 172.21.15.150:0/863853850 conn(0x7fbd0c0fc180 msgr2=0x7fbd0c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:18.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.344+0000 7fbd1416e700 1 -- 172.21.15.150:0/863853850 shutdown_connections 2024-05-10T04:02:18.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.344+0000 7fbd1416e700 1 -- 172.21.15.150:0/863853850 wait complete. 2024-05-10T04:02:18.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.345+0000 7fbd1416e700 1 Processor -- start 2024-05-10T04:02:18.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.345+0000 7fbd1416e700 1 -- start start 2024-05-10T04:02:18.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.345+0000 7fbd1416e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd0c100b70 0x7fbd0c06a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:18.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.345+0000 7fbd1416e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0c101c90 0x7fbd0c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:18.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.345+0000 7fbd1416e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c102630 0x7fbd0c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:18.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.345+0000 7fbd1416e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c19fab0 con 0x7fbd0c101c90 2024-05-10T04:02:18.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.345+0000 7fbd1416e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c19fc20 con 0x7fbd0c100b70 2024-05-10T04:02:18.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.345+0000 7fbd1416e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c19fde0 con 0x7fbd0c102630 2024-05-10T04:02:18.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.345+0000 7fbd11709700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0c101c90 0x7fbd0c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:18.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.345+0000 7fbd11709700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0c101c90 0x7fbd0c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59214/0 (socket says 172.21.15.150:59214) 2024-05-10T04:02:18.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.345+0000 7fbd11709700 1 -- 172.21.15.150:0/1843017734 learned_addr learned my addr 172.21.15.150:0/1843017734 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:18.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.345+0000 7fbd11f0a700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd0c100b70 0x7fbd0c06a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:18.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.345+0000 7fbd1270b700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c102630 0x7fbd0c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:18.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.346+0000 7fbd11709700 1 -- 172.21.15.150:0/1843017734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c102630 msgr2=0x7fbd0c19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:18.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.346+0000 7fbd11709700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c102630 0x7fbd0c19f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:18.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.346+0000 7fbd11709700 1 -- 172.21.15.150:0/1843017734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd0c100b70 msgr2=0x7fbd0c06a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:18.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.346+0000 7fbd11709700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd0c100b70 0x7fbd0c06a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:18.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.346+0000 7fbd11709700 1 -- 172.21.15.150:0/1843017734 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd080095f0 con 0x7fbd0c101c90 2024-05-10T04:02:18.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.346+0000 7fbd1270b700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c102630 0x7fbd0c19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:18.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.346+0000 7fbd11709700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0c101c90 0x7fbd0c19cda0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbd080098e0 tx=0x7fbd08009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:18.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.346+0000 7fbd02ffd700 1 -- 172.21.15.150:0/1843017734 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0801d070 con 0x7fbd0c101c90 2024-05-10T04:02:18.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.346+0000 7fbd02ffd700 1 -- 172.21.15.150:0/1843017734 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd08021410 con 0x7fbd0c101c90 2024-05-10T04:02:18.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.346+0000 7fbd1416e700 1 -- 172.21.15.150:0/1843017734 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c1a0060 con 0x7fbd0c101c90 2024-05-10T04:02:18.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.346+0000 7fbd02ffd700 1 -- 172.21.15.150:0/1843017734 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd08017650 con 0x7fbd0c101c90 2024-05-10T04:02:18.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.346+0000 7fbd1416e700 1 -- 172.21.15.150:0/1843017734 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c1a0500 con 0x7fbd0c101c90 2024-05-10T04:02:18.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.347+0000 7fbd1416e700 1 -- 172.21.15.150:0/1843017734 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0c107d80 con 0x7fbd0c101c90 2024-05-10T04:02:18.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.348+0000 7fbd02ffd700 1 -- 172.21.15.150:0/1843017734 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbd08021580 con 0x7fbd0c101c90 2024-05-10T04:02:18.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.349+0000 7fbd02ffd700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbcf80a1120 0x7fbcf80a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:18.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.349+0000 7fbd11f0a700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbcf80a1120 0x7fbcf80a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:18.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.349+0000 7fbd02ffd700 1 -- 172.21.15.150:0/1843017734 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbd080cdf80 con 0x7fbd0c101c90 2024-05-10T04:02:18.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.349+0000 7fbd11f0a700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbcf80a1120 0x7fbcf80a3630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c101b90 tx=0x7fbcfc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:18.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.353+0000 7fbd02ffd700 1 -- 172.21.15.150:0/1843017734 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbd08097c40 con 0x7fbd0c101c90 2024-05-10T04:02:18.580 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:02:18 smithi161 systemd[1]: Starting Ceph mds.h for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:02:18.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.598+0000 7fbd1416e700 1 -- 172.21.15.150:0/1843017734 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd0c0678b0 con 0x7fbd0c101c90 2024-05-10T04:02:18.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.599+0000 7fbd02ffd700 1 -- 172.21.15.150:0/1843017734 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5817 (secure 0 0 0) 0x7fbd080b9e60 con 0x7fbd0c101c90 2024-05-10T04:02:18.601 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:18.601 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-05-10T04:02:18.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.601+0000 7fbd1416e700 1 -- 172.21.15.150:0/1843017734 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbcf80a1120 msgr2=0x7fbcf80a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:18.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.601+0000 7fbd1416e700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbcf80a1120 0x7fbcf80a3630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c101b90 tx=0x7fbcfc009260 comp rx=0 tx=0).stop 2024-05-10T04:02:18.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.601+0000 7fbd1416e700 1 -- 172.21.15.150:0/1843017734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0c101c90 msgr2=0x7fbd0c19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:18.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.601+0000 7fbd1416e700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0c101c90 0x7fbd0c19cda0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbd080098e0 tx=0x7fbd08009580 comp rx=0 tx=0).stop 2024-05-10T04:02:18.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.601+0000 7fbd1416e700 1 -- 172.21.15.150:0/1843017734 shutdown_connections 2024-05-10T04:02:18.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.601+0000 7fbd1416e700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbcf80a1120 0x7fbcf80a3630 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:18.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.601+0000 7fbd1416e700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c102630 0x7fbd0c19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:18.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.601+0000 7fbd1416e700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd0c100b70 0x7fbd0c06a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:18.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.601+0000 7fbd1416e700 1 --2- 172.21.15.150:0/1843017734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0c101c90 0x7fbd0c19cda0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:18.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.601+0000 7fbd1416e700 1 -- 172.21.15.150:0/1843017734 >> 172.21.15.150:0/1843017734 conn(0x7fbd0c0fc180 msgr2=0x7fbd0c104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:18.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.601+0000 7fbd1416e700 1 -- 172.21.15.150:0/1843017734 shutdown_connections 2024-05-10T04:02:18.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:18.601+0000 7fbd1416e700 1 -- 172.21.15.150:0/1843017734 wait complete. 2024-05-10T04:02:18.605 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 8 2024-05-10T04:02:18.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:18 smithi150 ceph-mon[42477]: Deploying daemon mds.h on smithi161 2024-05-10T04:02:18.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:18 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] up:boot 2024-05-10T04:02:18.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:18 smithi150 ceph-mon[42477]: fsmap 7 up:standby 2024-05-10T04:02:18.965 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:18 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T04:02:18.965 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:18 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1843017734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:18.967 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:18 smithi161 ceph-mon[46918]: Deploying daemon mds.h on smithi161 2024-05-10T04:02:18.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:18 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] up:boot 2024-05-10T04:02:18.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:18 smithi161 ceph-mon[46918]: fsmap 7 up:standby 2024-05-10T04:02:18.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:18 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T04:02:18.968 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:18 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1843017734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:19.237 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:18 smithi181 ceph-mon[46602]: Deploying daemon mds.h on smithi161 2024-05-10T04:02:19.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:18 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] up:boot 2024-05-10T04:02:19.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:18 smithi181 ceph-mon[46602]: fsmap 7 up:standby 2024-05-10T04:02:19.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:18 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T04:02:19.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:18 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1843017734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:19.242 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:02:18 smithi161 bash[83151]: 2b7f5cfd783d15d404d3e3cc5d9b858ea3f96f3d698579cefdfa2f8aa98720af 2024-05-10T04:02:19.242 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:02:18 smithi161 systemd[1]: Started Ceph mds.h for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T04:02:19.242 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:02:18 smithi161 ceph-mds[83189]: -- 172.21.15.161:0/3822922800 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563b12b7e1a0 con 0x563b12bded00 2024-05-10T04:02:19.242 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:02:18 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-h[83166]: starting mds.h at 2024-05-10T04:02:20.123 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:20.123 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:20.123 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:20.123 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:20.123 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:20.123 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:20.123 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:20 smithi181 ceph-mon[46602]: Deploying daemon mds.i on smithi181 2024-05-10T04:02:20.124 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:20 smithi181 ceph-mon[46602]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:20.124 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:20 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.161:6836/1619036473,v1:172.21.15.161:6837/1619036473] up:boot 2024-05-10T04:02:20.124 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:20 smithi181 ceph-mon[46602]: fsmap 8 up:standby 2024-05-10T04:02:20.124 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:20 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T04:02:20.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:20.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:20.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:20.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:20.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:20.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:20.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:20 smithi150 ceph-mon[42477]: Deploying daemon mds.i on smithi181 2024-05-10T04:02:20.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:20 smithi150 ceph-mon[42477]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:20.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:20 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.161:6836/1619036473,v1:172.21.15.161:6837/1619036473] up:boot 2024-05-10T04:02:20.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:20 smithi150 ceph-mon[42477]: fsmap 8 up:standby 2024-05-10T04:02:20.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:20 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T04:02:20.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:20.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:20.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:20.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:20.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:20.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:20.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:20 smithi161 ceph-mon[46918]: Deploying daemon mds.i on smithi181 2024-05-10T04:02:20.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:20 smithi161 ceph-mon[46918]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:20.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:20 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.161:6836/1619036473,v1:172.21.15.161:6837/1619036473] up:boot 2024-05-10T04:02:20.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:20 smithi161 ceph-mon[46918]: fsmap 8 up:standby 2024-05-10T04:02:20.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:20 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T04:02:20.617 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:02:20.652 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:02:20 smithi181 systemd[1]: Starting Ceph mds.i for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:02:20.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.767+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2258797279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d70100ac0 msgr2=0x7f0d70100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:20.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.767+0000 7f0d75cf8700 1 --2- 172.21.15.150:0/2258797279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d70100ac0 0x7f0d70100ed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009990 tx=0x7f0d64009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:20.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.767+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2258797279 shutdown_connections 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.767+0000 7f0d75cf8700 1 --2- 172.21.15.150:0/2258797279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d70102580 0x7f0d70108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.767+0000 7f0d75cf8700 1 --2- 172.21.15.150:0/2258797279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d70100ac0 0x7f0d70100ed0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.767+0000 7f0d75cf8700 1 --2- 172.21.15.150:0/2258797279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d70101be0 0x7f0d70101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.767+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2258797279 >> 172.21.15.150:0/2258797279 conn(0x7f0d700fc0d0 msgr2=0x7f0d700fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.767+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2258797279 shutdown_connections 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.767+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2258797279 wait complete. 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.768+0000 7f0d75cf8700 1 Processor -- start 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.768+0000 7f0d75cf8700 1 -- start start 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.768+0000 7f0d75cf8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d70100ac0 0x7f0d70068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.768+0000 7f0d75cf8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d70101be0 0x7f0d7019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.768+0000 7f0d75cf8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d70102580 0x7f0d7019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.768+0000 7f0d75cf8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7019fa00 con 0x7f0d70101be0 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.768+0000 7f0d75cf8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7019fb70 con 0x7f0d70100ac0 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.768+0000 7f0d75cf8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7019fd30 con 0x7f0d70102580 2024-05-10T04:02:20.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.768+0000 7f0d6effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d70101be0 0x7f0d7019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:20.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.768+0000 7f0d6effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d70101be0 0x7f0d7019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34398/0 (socket says 172.21.15.150:34398) 2024-05-10T04:02:20.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.768+0000 7f0d6effd700 1 -- 172.21.15.150:0/2563203954 learned_addr learned my addr 172.21.15.150:0/2563203954 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:20.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.768+0000 7f0d6ffff700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d70102580 0x7f0d7019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:20.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.769+0000 7f0d6f7fe700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d70100ac0 0x7f0d70068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:20.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.769+0000 7f0d6effd700 1 -- 172.21.15.150:0/2563203954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d70102580 msgr2=0x7f0d7019f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:20.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.769+0000 7f0d6effd700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d70102580 0x7f0d7019f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:20.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.769+0000 7f0d6effd700 1 -- 172.21.15.150:0/2563203954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d70100ac0 msgr2=0x7f0d70068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:20.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.769+0000 7f0d6effd700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d70100ac0 0x7f0d70068e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:20.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.769+0000 7f0d6effd700 1 -- 172.21.15.150:0/2563203954 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d640095f0 con 0x7f0d70101be0 2024-05-10T04:02:20.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.769+0000 7f0d6ffff700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d70102580 0x7f0d7019f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:20.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.769+0000 7f0d6effd700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d70101be0 0x7f0d7019cd70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000c920 tx=0x7f0d6000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:20.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.769+0000 7f0d6cff9700 1 -- 172.21.15.150:0/2563203954 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60007b00 con 0x7f0d70101be0 2024-05-10T04:02:20.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.769+0000 7f0d6cff9700 1 -- 172.21.15.150:0/2563203954 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d60004cf0 con 0x7f0d70101be0 2024-05-10T04:02:20.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.769+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2563203954 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7019ffb0 con 0x7f0d70101be0 2024-05-10T04:02:20.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.769+0000 7f0d6cff9700 1 -- 172.21.15.150:0/2563203954 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60005610 con 0x7f0d70101be0 2024-05-10T04:02:20.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.769+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2563203954 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d701a0490 con 0x7f0d70101be0 2024-05-10T04:02:20.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.770+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2563203954 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d70107cd0 con 0x7f0d70101be0 2024-05-10T04:02:20.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.771+0000 7f0d6cff9700 1 -- 172.21.15.150:0/2563203954 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0d60004750 con 0x7f0d70101be0 2024-05-10T04:02:20.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.772+0000 7f0d6cff9700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0d580a1120 0x7f0d580a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:20.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.772+0000 7f0d6f7fe700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0d580a1120 0x7f0d580a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:20.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.773+0000 7f0d6cff9700 1 -- 172.21.15.150:0/2563203954 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0d600cc400 con 0x7f0d70101be0 2024-05-10T04:02:20.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.773+0000 7f0d6f7fe700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0d580a1120 0x7f0d580a3630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0d640038c0 tx=0x7f0d64005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:20.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:20.776+0000 7f0d6cff9700 1 -- 172.21.15.150:0/2563203954 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0d60096030 con 0x7f0d70101be0 2024-05-10T04:02:21.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.025+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2563203954 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d7010c2a0 con 0x7f0d70101be0 2024-05-10T04:02:21.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.026+0000 7f0d6cff9700 1 -- 172.21.15.150:0/2563203954 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6577 (secure 0 0 0) 0x7f0d60099650 con 0x7f0d70101be0 2024-05-10T04:02:21.028 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:21.029 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-05-10T04:02:21.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.028+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2563203954 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0d580a1120 msgr2=0x7f0d580a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:21.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.028+0000 7f0d75cf8700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0d580a1120 0x7f0d580a3630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0d640038c0 tx=0x7f0d64005040 comp rx=0 tx=0).stop 2024-05-10T04:02:21.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.028+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2563203954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d70101be0 msgr2=0x7f0d7019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:21.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.028+0000 7f0d75cf8700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d70101be0 0x7f0d7019cd70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000c920 tx=0x7f0d6000cce0 comp rx=0 tx=0).stop 2024-05-10T04:02:21.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.028+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2563203954 shutdown_connections 2024-05-10T04:02:21.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.028+0000 7f0d75cf8700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0d580a1120 0x7f0d580a3630 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:21.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.028+0000 7f0d75cf8700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d70102580 0x7f0d7019f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:21.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.028+0000 7f0d75cf8700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d70100ac0 0x7f0d70068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:21.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.028+0000 7f0d75cf8700 1 --2- 172.21.15.150:0/2563203954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d70101be0 0x7f0d7019cd70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:21.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.028+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2563203954 >> 172.21.15.150:0/2563203954 conn(0x7f0d700fc0d0 msgr2=0x7f0d70104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:21.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.028+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2563203954 shutdown_connections 2024-05-10T04:02:21.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:21.028+0000 7f0d75cf8700 1 -- 172.21.15.150:0/2563203954 wait complete. 2024-05-10T04:02:21.035 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 9 2024-05-10T04:02:21.238 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:02:20 smithi181 bash[84354]: 38f035aad37da4d2b7739d7338c6b4f8c337720f3809c7b55768a7e09f00378c 2024-05-10T04:02:21.238 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:02:20 smithi181 systemd[1]: Started Ceph mds.i for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T04:02:21.238 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:02:20 smithi181 ceph-mds[84390]: -- 172.21.15.181:0/1125586090 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557cc7b401a0 con 0x557cc7ba0400 2024-05-10T04:02:21.238 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:02:20 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-i[84368]: starting mds.i at 2024-05-10T04:02:22.034 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:22.034 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:22.034 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:22.034 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:22.034 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:22.034 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:22.034 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:21 smithi150 ceph-mon[42477]: Deploying daemon mds.g on smithi150 2024-05-10T04:02:22.035 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:21 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2563203954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:22.035 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:21 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] up:boot 2024-05-10T04:02:22.035 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:21 smithi150 ceph-mon[42477]: fsmap 9 up:standby 2024-05-10T04:02:22.035 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:21 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T04:02:22.035 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:21 smithi150 ceph-mon[42477]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:21 smithi181 ceph-mon[46602]: Deploying daemon mds.g on smithi150 2024-05-10T04:02:22.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:21 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2563203954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:22.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:21 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] up:boot 2024-05-10T04:02:22.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:21 smithi181 ceph-mon[46602]: fsmap 9 up:standby 2024-05-10T04:02:22.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:21 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T04:02:22.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:21 smithi181 ceph-mon[46602]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:22.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:22.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:22.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:22.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:22.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:22.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:21 smithi161 ceph-mon[46918]: Deploying daemon mds.g on smithi150 2024-05-10T04:02:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:21 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2563203954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:21 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] up:boot 2024-05-10T04:02:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:21 smithi161 ceph-mon[46918]: fsmap 9 up:standby 2024-05-10T04:02:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:21 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T04:02:22.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:21 smithi161 ceph-mon[46918]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:22.386 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:02:22 smithi150 systemd[1]: Starting Ceph mds.g for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:02:22.963 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:02:22 smithi150 ceph-mds[92738]: -- 172.21.15.150:0/1721244376 <== mon.0 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621acfd21a0 con 0x5621ad032400 2024-05-10T04:02:22.964 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:02:22 smithi150 bash[92700]: 9d9b74151083186742dc585320aa189f73d795fb26de30387324d3e5c2a02d84 2024-05-10T04:02:22.964 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:02:22 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-g[92715]: starting mds.g at 2024-05-10T04:02:22.964 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:02:22 smithi150 systemd[1]: Started Ceph mds.g for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T04:02:23.045 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:02:23.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.186+0000 7eff4914f700 1 -- 172.21.15.150:0/358701087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff44100530 msgr2=0x7eff4410a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:23.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.186+0000 7eff4914f700 1 --2- 172.21.15.150:0/358701087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff44100530 0x7eff4410a150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00b0a0 tx=0x7eff3c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:23.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.186+0000 7eff4914f700 1 -- 172.21.15.150:0/358701087 shutdown_connections 2024-05-10T04:02:23.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.186+0000 7eff4914f700 1 --2- 172.21.15.150:0/358701087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff44100530 0x7eff4410a150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:23.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.186+0000 7eff4914f700 1 --2- 172.21.15.150:0/358701087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff441031a0 0x7eff441035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:23.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.186+0000 7eff4914f700 1 --2- 172.21.15.150:0/358701087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff440ffbe0 0x7eff440ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:23.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.186+0000 7eff4914f700 1 -- 172.21.15.150:0/358701087 >> 172.21.15.150:0/358701087 conn(0x7eff440fb8b0 msgr2=0x7eff440fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:23.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.186+0000 7eff4914f700 1 -- 172.21.15.150:0/358701087 shutdown_connections 2024-05-10T04:02:23.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.186+0000 7eff4914f700 1 -- 172.21.15.150:0/358701087 wait complete. 2024-05-10T04:02:23.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.187+0000 7eff4914f700 1 Processor -- start 2024-05-10T04:02:23.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4914f700 1 -- start start 2024-05-10T04:02:23.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4914f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff440ffbe0 0x7eff441980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:23.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4914f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff44100530 0x7eff441985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:23.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4914f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff441031a0 0x7eff4419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:23.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4914f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4419fac0 con 0x7eff44100530 2024-05-10T04:02:23.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4914f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4419fc00 con 0x7eff441031a0 2024-05-10T04:02:23.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4914f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4419fd90 con 0x7eff440ffbe0 2024-05-10T04:02:23.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff44100530 0x7eff441985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:23.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff44100530 0x7eff441985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34444/0 (socket says 172.21.15.150:34444) 2024-05-10T04:02:23.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4259c700 1 -- 172.21.15.150:0/4038797660 learned_addr learned my addr 172.21.15.150:0/4038797660 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:23.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff42d9d700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff440ffbe0 0x7eff441980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:23.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4359e700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff441031a0 0x7eff4419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:23.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4259c700 1 -- 172.21.15.150:0/4038797660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff440ffbe0 msgr2=0x7eff441980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:23.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4259c700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff440ffbe0 0x7eff441980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:23.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4259c700 1 -- 172.21.15.150:0/4038797660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff441031a0 msgr2=0x7eff4419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:23.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4259c700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff441031a0 0x7eff4419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:23.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.188+0000 7eff4259c700 1 -- 172.21.15.150:0/4038797660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c009d00 con 0x7eff44100530 2024-05-10T04:02:23.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.189+0000 7eff4259c700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff44100530 0x7eff441985f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7eff3400c9e0 tx=0x7eff3400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:23.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.189+0000 7eff2bfff700 1 -- 172.21.15.150:0/4038797660 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34007ac0 con 0x7eff44100530 2024-05-10T04:02:23.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.189+0000 7eff4914f700 1 -- 172.21.15.150:0/4038797660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff441a0070 con 0x7eff44100530 2024-05-10T04:02:23.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.189+0000 7eff2bfff700 1 -- 172.21.15.150:0/4038797660 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff3400ce80 con 0x7eff44100530 2024-05-10T04:02:23.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.189+0000 7eff2bfff700 1 -- 172.21.15.150:0/4038797660 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34018660 con 0x7eff44100530 2024-05-10T04:02:23.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.189+0000 7eff4914f700 1 -- 172.21.15.150:0/4038797660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff441a05d0 con 0x7eff44100530 2024-05-10T04:02:23.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.190+0000 7eff4914f700 1 -- 172.21.15.150:0/4038797660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff44108ff0 con 0x7eff44100530 2024-05-10T04:02:23.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.191+0000 7eff2bfff700 1 -- 172.21.15.150:0/4038797660 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7eff3400f450 con 0x7eff44100530 2024-05-10T04:02:23.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.192+0000 7eff2bfff700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7eff2c0a1120 0x7eff2c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:23.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.192+0000 7eff42d9d700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7eff2c0a1120 0x7eff2c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:23.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.192+0000 7eff2bfff700 1 -- 172.21.15.150:0/4038797660 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7eff340cc650 con 0x7eff44100530 2024-05-10T04:02:23.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.192+0000 7eff42d9d700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7eff2c0a1120 0x7eff2c0a3630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7eff44069880 tx=0x7eff3800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:23.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.195+0000 7eff2bfff700 1 -- 172.21.15.150:0/4038797660 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7eff34096310 con 0x7eff44100530 2024-05-10T04:02:23.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.454+0000 7eff4914f700 1 -- 172.21.15.150:0/4038797660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff44068730 con 0x7eff44100530 2024-05-10T04:02:23.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.455+0000 7eff2bfff700 1 -- 172.21.15.150:0/4038797660 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8100 (secure 0 0 0) 0x7eff3400fe60 con 0x7eff44100530 2024-05-10T04:02:23.458 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:23.458 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24556,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-10T04:02:23.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.457+0000 7eff4914f700 1 -- 172.21.15.150:0/4038797660 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7eff2c0a1120 msgr2=0x7eff2c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:23.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.457+0000 7eff4914f700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7eff2c0a1120 0x7eff2c0a3630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7eff44069880 tx=0x7eff3800a3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:23.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.457+0000 7eff4914f700 1 -- 172.21.15.150:0/4038797660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff44100530 msgr2=0x7eff441985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:23.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.457+0000 7eff4914f700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff44100530 0x7eff441985f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7eff3400c9e0 tx=0x7eff3400ccf0 comp rx=0 tx=0).stop 2024-05-10T04:02:23.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.457+0000 7eff4914f700 1 -- 172.21.15.150:0/4038797660 shutdown_connections 2024-05-10T04:02:23.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.457+0000 7eff4914f700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7eff2c0a1120 0x7eff2c0a3630 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:23.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.457+0000 7eff4914f700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff441031a0 0x7eff4419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:23.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.458+0000 7eff4914f700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff440ffbe0 0x7eff441980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:23.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.458+0000 7eff4914f700 1 --2- 172.21.15.150:0/4038797660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff44100530 0x7eff441985f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:23.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.458+0000 7eff4914f700 1 -- 172.21.15.150:0/4038797660 >> 172.21.15.150:0/4038797660 conn(0x7eff440fb8b0 msgr2=0x7eff44106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:23.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.458+0000 7eff4914f700 1 -- 172.21.15.150:0/4038797660 shutdown_connections 2024-05-10T04:02:23.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:23.458+0000 7eff4914f700 1 -- 172.21.15.150:0/4038797660 wait complete. 2024-05-10T04:02:23.462 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 11 2024-05-10T04:02:23.718 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:23.718 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:23.719 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:23.719 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:23.719 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:23.719 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:23.719 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:23 smithi150 ceph-mon[42477]: Deploying daemon mds.a on smithi150 2024-05-10T04:02:23.719 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:23 smithi150 ceph-mon[42477]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:23.719 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:23 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.150:6838/2082606207,v1:172.21.15.150:6839/2082606207] up:boot 2024-05-10T04:02:23.719 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:23 smithi150 ceph-mon[42477]: fsmap 10 up:standby 2024-05-10T04:02:23.719 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:23 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T04:02:23.719 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:23 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4038797660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:23.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:23.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:23.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:23.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:23.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:23.990 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:23.991 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:23 smithi181 ceph-mon[46602]: Deploying daemon mds.a on smithi150 2024-05-10T04:02:23.991 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:23 smithi181 ceph-mon[46602]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:23.991 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:23 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.150:6838/2082606207,v1:172.21.15.150:6839/2082606207] up:boot 2024-05-10T04:02:23.991 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:23 smithi181 ceph-mon[46602]: fsmap 10 up:standby 2024-05-10T04:02:23.991 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:23 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T04:02:23.991 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:23 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4038797660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:23.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:23.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:23.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:23.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:23.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:23.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:23.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:23 smithi161 ceph-mon[46918]: Deploying daemon mds.a on smithi150 2024-05-10T04:02:23.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:23 smithi161 ceph-mon[46918]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:23.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:23 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.150:6838/2082606207,v1:172.21.15.150:6839/2082606207] up:boot 2024-05-10T04:02:23.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:23 smithi161 ceph-mon[46918]: fsmap 10 up:standby 2024-05-10T04:02:23.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:23 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T04:02:23.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:23 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4038797660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:24.188 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:02:23 smithi150 systemd[1]: Starting Ceph mds.a for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:02:24.713 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:02:24 smithi150 bash[93087]: bff7fcf467e7b81da99496eb5b6b88eb2ce2af211dba83d7bbae2527b85cdba1 2024-05-10T04:02:24.714 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:02:24 smithi150 systemd[1]: Started Ceph mds.a for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T04:02:24.714 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:02:24 smithi150 ceph-mds[93125]: -- 172.21.15.150:0/4230028844 <== mon.0 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559bca7f21a0 con 0x559bca852400 2024-05-10T04:02:24.714 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:02:24 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-a[93102]: starting mds.a at 2024-05-10T04:02:25.474 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:02:25.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:25.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:25.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:25.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:25.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:25.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:25.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:25 smithi161 ceph-mon[46918]: Deploying daemon mds.b on smithi161 2024-05-10T04:02:25.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:25 smithi161 ceph-mon[46918]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:25.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:25 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.150:6840/591989720,v1:172.21.15.150:6841/591989720] up:boot 2024-05-10T04:02:25.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:25 smithi161 ceph-mon[46918]: fsmap 11 up:standby 2024-05-10T04:02:25.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:25 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T04:02:25.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.622+0000 7f1b1069b700 1 -- 172.21.15.150:0/2638460751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b08101bd0 msgr2=0x7f1b08101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:25.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.622+0000 7f1b1069b700 1 --2- 172.21.15.150:0/2638460751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b08101bd0 0x7f1b08101fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009990 tx=0x7f1b04009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:25.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.622+0000 7f1b1069b700 1 -- 172.21.15.150:0/2638460751 shutdown_connections 2024-05-10T04:02:25.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.622+0000 7f1b1069b700 1 --2- 172.21.15.150:0/2638460751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b08103690 0x7f1b08109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.622+0000 7f1b1069b700 1 --2- 172.21.15.150:0/2638460751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b08101bd0 0x7f1b08101fe0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.622+0000 7f1b1069b700 1 --2- 172.21.15.150:0/2638460751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b08102cf0 0x7f1b08103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.622+0000 7f1b1069b700 1 -- 172.21.15.150:0/2638460751 >> 172.21.15.150:0/2638460751 conn(0x7f1b080fd1e0 msgr2=0x7f1b080ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.622+0000 7f1b1069b700 1 -- 172.21.15.150:0/2638460751 shutdown_connections 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.622+0000 7f1b1069b700 1 -- 172.21.15.150:0/2638460751 wait complete. 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.623+0000 7f1b1069b700 1 Processor -- start 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.623+0000 7f1b1069b700 1 -- start start 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.623+0000 7f1b1069b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b08101bd0 0x7f1b08198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.623+0000 7f1b1069b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b08102cf0 0x7f1b08198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.623+0000 7f1b1069b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b08103690 0x7f1b0819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.623+0000 7f1b1069b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0819fbf0 con 0x7f1b08102cf0 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.623+0000 7f1b1069b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0819fd60 con 0x7f1b08103690 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.623+0000 7f1b1069b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0819ff20 con 0x7f1b08101bd0 2024-05-10T04:02:25.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.623+0000 7f1b0dc36700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b08102cf0 0x7f1b08198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.623+0000 7f1b0dc36700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b08102cf0 0x7f1b08198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34484/0 (socket says 172.21.15.150:34484) 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.623+0000 7f1b0dc36700 1 -- 172.21.15.150:0/3369545914 learned_addr learned my addr 172.21.15.150:0/3369545914 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.623+0000 7f1b0ec38700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b08103690 0x7f1b0819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.624+0000 7f1b0e437700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b08101bd0 0x7f1b08198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.624+0000 7f1b0dc36700 1 -- 172.21.15.150:0/3369545914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b08101bd0 msgr2=0x7f1b08198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.624+0000 7f1b0dc36700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b08101bd0 0x7f1b08198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.624+0000 7f1b0dc36700 1 -- 172.21.15.150:0/3369545914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b08103690 msgr2=0x7f1b0819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.624+0000 7f1b0dc36700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b08103690 0x7f1b0819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.624+0000 7f1b0dc36700 1 -- 172.21.15.150:0/3369545914 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b040095f0 con 0x7f1b08102cf0 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.624+0000 7f1b0e437700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b08101bd0 0x7f1b08198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.624+0000 7f1b0dc36700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b08102cf0 0x7f1b08198650 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1af800b530 tx=0x7f1af800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.624+0000 7f1aff7fe700 1 -- 172.21.15.150:0/3369545914 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8011830 con 0x7f1b08102cf0 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.624+0000 7f1aff7fe700 1 -- 172.21.15.150:0/3369545914 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1af8011e50 con 0x7f1b08102cf0 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.624+0000 7f1b1069b700 1 -- 172.21.15.150:0/3369545914 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b081a0200 con 0x7f1b08102cf0 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.624+0000 7f1aff7fe700 1 -- 172.21.15.150:0/3369545914 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af800f2e0 con 0x7f1b08102cf0 2024-05-10T04:02:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.624+0000 7f1b1069b700 1 -- 172.21.15.150:0/3369545914 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b081a0790 con 0x7f1b08102cf0 2024-05-10T04:02:25.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.625+0000 7f1b1069b700 1 -- 172.21.15.150:0/3369545914 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b08108e50 con 0x7f1b08102cf0 2024-05-10T04:02:25.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.626+0000 7f1aff7fe700 1 -- 172.21.15.150:0/3369545914 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1af8010380 con 0x7f1b08102cf0 2024-05-10T04:02:25.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.627+0000 7f1aff7fe700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1af40a1120 0x7f1af40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:25.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.627+0000 7f1b0e437700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1af40a1120 0x7f1af40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:25.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.627+0000 7f1aff7fe700 1 -- 172.21.15.150:0/3369545914 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1af80cc570 con 0x7f1b08102cf0 2024-05-10T04:02:25.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.627+0000 7f1b0e437700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1af40a1120 0x7f1af40a3630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400b5c0 tx=0x7f1b04005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:25.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.631+0000 7f1aff7fe700 1 -- 172.21.15.150:0/3369545914 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1af80961a0 con 0x7f1b08102cf0 2024-05-10T04:02:25.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:25.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:25.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:25.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:25.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:25.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:25.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:25 smithi150 ceph-mon[42477]: Deploying daemon mds.b on smithi161 2024-05-10T04:02:25.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:25 smithi150 ceph-mon[42477]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:25.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:25 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.150:6840/591989720,v1:172.21.15.150:6841/591989720] up:boot 2024-05-10T04:02:25.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:25 smithi150 ceph-mon[42477]: fsmap 11 up:standby 2024-05-10T04:02:25.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:25 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T04:02:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:02:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:02:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:25 smithi181 ceph-mon[46602]: Deploying daemon mds.b on smithi161 2024-05-10T04:02:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:25 smithi181 ceph-mon[46602]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:25 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.150:6840/591989720,v1:172.21.15.150:6841/591989720] up:boot 2024-05-10T04:02:25.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:25 smithi181 ceph-mon[46602]: fsmap 11 up:standby 2024-05-10T04:02:25.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:25 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T04:02:25.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.874+0000 7f1b1069b700 1 -- 172.21.15.150:0/3369545914 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b080678b0 con 0x7f1b08102cf0 2024-05-10T04:02:25.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.874+0000 7f1aff7fe700 1 -- 172.21.15.150:0/3369545914 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8858 (secure 0 0 0) 0x7f1af80997c0 con 0x7f1b08102cf0 2024-05-10T04:02:25.877 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:25.877 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24556,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-10T04:02:25.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.877+0000 7f1b1069b700 1 -- 172.21.15.150:0/3369545914 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1af40a1120 msgr2=0x7f1af40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:25.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.877+0000 7f1b1069b700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1af40a1120 0x7f1af40a3630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400b5c0 tx=0x7f1b04005fb0 comp rx=0 tx=0).stop 2024-05-10T04:02:25.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.877+0000 7f1b1069b700 1 -- 172.21.15.150:0/3369545914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b08102cf0 msgr2=0x7f1b08198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:25.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.877+0000 7f1b1069b700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b08102cf0 0x7f1b08198650 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1af800b530 tx=0x7f1af800b840 comp rx=0 tx=0).stop 2024-05-10T04:02:25.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.877+0000 7f1b1069b700 1 -- 172.21.15.150:0/3369545914 shutdown_connections 2024-05-10T04:02:25.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.877+0000 7f1b1069b700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1af40a1120 0x7f1af40a3630 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:25.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.877+0000 7f1b1069b700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b08103690 0x7f1b0819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:25.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.877+0000 7f1b1069b700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b08101bd0 0x7f1b08198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:25.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.877+0000 7f1b1069b700 1 --2- 172.21.15.150:0/3369545914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b08102cf0 0x7f1b08198650 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:25.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.877+0000 7f1b1069b700 1 -- 172.21.15.150:0/3369545914 >> 172.21.15.150:0/3369545914 conn(0x7f1b080fd1e0 msgr2=0x7f1b08105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:25.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.877+0000 7f1b1069b700 1 -- 172.21.15.150:0/3369545914 shutdown_connections 2024-05-10T04:02:25.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:25.877+0000 7f1b1069b700 1 -- 172.21.15.150:0/3369545914 wait complete. 2024-05-10T04:02:25.880 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 12 2024-05-10T04:02:25.920 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:02:25 smithi161 systemd[1]: Starting Ceph mds.b for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:02:26.242 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:02:26 smithi161 bash[83503]: d5fddd4ab17d143e5b105e7887621df9d7f5598ebb4e5fbe11e62a50339969e2 2024-05-10T04:02:26.242 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:02:26 smithi161 systemd[1]: Started Ceph mds.b for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T04:02:26.242 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:02:26 smithi161 ceph-mds[83541]: -- 172.21.15.161:0/3454996002 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556ad18061a0 con 0x556ad1866880 2024-05-10T04:02:26.242 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:02:26 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-b[83518]: starting mds.b at 2024-05-10T04:02:26.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:26 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3369545914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:26.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:26 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:02:26.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:26 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3369545914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:26.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:26 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:02:26.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:26 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3369545914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:26.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:26.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:26 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:02:27.895 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:02:28.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.032+0000 7f8b206e5700 1 -- 172.21.15.150:0/3043186407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b181050f0 msgr2=0x7f8b18107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:28.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.032+0000 7f8b206e5700 1 --2- 172.21.15.150:0/3043186407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b181050f0 0x7f8b18107510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009990 tx=0x7f8b08009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.032+0000 7f8b206e5700 1 -- 172.21.15.150:0/3043186407 shutdown_connections 2024-05-10T04:02:28.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.032+0000 7f8b206e5700 1 --2- 172.21.15.150:0/3043186407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b18107a50 0x7f8b18109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.032+0000 7f8b206e5700 1 --2- 172.21.15.150:0/3043186407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b180fff80 0x7f8b18100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.032+0000 7f8b206e5700 1 --2- 172.21.15.150:0/3043186407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b181050f0 0x7f8b18107510 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.032+0000 7f8b206e5700 1 -- 172.21.15.150:0/3043186407 >> 172.21.15.150:0/3043186407 conn(0x7f8b180fb8f0 msgr2=0x7f8b180fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:28.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.032+0000 7f8b206e5700 1 -- 172.21.15.150:0/3043186407 shutdown_connections 2024-05-10T04:02:28.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.032+0000 7f8b206e5700 1 -- 172.21.15.150:0/3043186407 wait complete. 2024-05-10T04:02:28.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b206e5700 1 Processor -- start 2024-05-10T04:02:28.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b206e5700 1 -- start start 2024-05-10T04:02:28.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b206e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b180fff80 0x7f8b18197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:28.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b206e5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b181050f0 0x7f8b18198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:28.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b206e5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b18107a50 0x7f8b1819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:28.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b206e5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1819f9d0 con 0x7f8b181050f0 2024-05-10T04:02:28.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b206e5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1819fb40 con 0x7f8b18107a50 2024-05-10T04:02:28.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b206e5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1819fd00 con 0x7f8b180fff80 2024-05-10T04:02:28.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b1dc80700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b181050f0 0x7f8b18198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:28.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b1dc80700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b181050f0 0x7f8b18198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34498/0 (socket says 172.21.15.150:34498) 2024-05-10T04:02:28.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b1dc80700 1 -- 172.21.15.150:0/2634319672 learned_addr learned my addr 172.21.15.150:0/2634319672 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:28.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b1e481700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b180fff80 0x7f8b18197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:28.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b1ec82700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b18107a50 0x7f8b1819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:28.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b1dc80700 1 -- 172.21.15.150:0/2634319672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b180fff80 msgr2=0x7f8b18197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:28.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b1dc80700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b180fff80 0x7f8b18197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b1dc80700 1 -- 172.21.15.150:0/2634319672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b18107a50 msgr2=0x7f8b1819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:28.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b1dc80700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b18107a50 0x7f8b1819f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b1dc80700 1 -- 172.21.15.150:0/2634319672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b14009520 con 0x7f8b181050f0 2024-05-10T04:02:28.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.033+0000 7f8b1dc80700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b181050f0 0x7f8b18198430 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b080098e0 tx=0x7f8b08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:28.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.034+0000 7f8b0f7fe700 1 -- 172.21.15.150:0/2634319672 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0801d070 con 0x7f8b181050f0 2024-05-10T04:02:28.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.034+0000 7f8b0f7fe700 1 -- 172.21.15.150:0/2634319672 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b08021410 con 0x7f8b181050f0 2024-05-10T04:02:28.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.034+0000 7f8b0f7fe700 1 -- 172.21.15.150:0/2634319672 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b08017650 con 0x7f8b181050f0 2024-05-10T04:02:28.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.034+0000 7f8b206e5700 1 -- 172.21.15.150:0/2634319672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b080095f0 con 0x7f8b181050f0 2024-05-10T04:02:28.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.034+0000 7f8b206e5700 1 -- 172.21.15.150:0/2634319672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b181a01c0 con 0x7f8b181050f0 2024-05-10T04:02:28.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.035+0000 7f8b206e5700 1 -- 172.21.15.150:0/2634319672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b180fd4a0 con 0x7f8b181050f0 2024-05-10T04:02:28.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.037+0000 7f8b0f7fe700 1 -- 172.21.15.150:0/2634319672 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8b0800fad0 con 0x7f8b181050f0 2024-05-10T04:02:28.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.037+0000 7f8b0f7fe700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8b040a1120 0x7f8b040a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:28.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.037+0000 7f8b1e481700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8b040a1120 0x7f8b040a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:28.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.037+0000 7f8b0f7fe700 1 -- 172.21.15.150:0/2634319672 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8b080cda70 con 0x7f8b181050f0 2024-05-10T04:02:28.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.038+0000 7f8b1e481700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8b040a1120 0x7f8b040a3630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8b18103070 tx=0x7f8b14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:28.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.041+0000 7f8b0f7fe700 1 -- 172.21.15.150:0/2634319672 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8b08097730 con 0x7f8b181050f0 2024-05-10T04:02:28.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:28.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] up:boot 2024-05-10T04:02:28.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: fsmap 12 up:standby 2024-05-10T04:02:28.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T04:02:28.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:28.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:02:28.214 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.215 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:28 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:02:28.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.305+0000 7f8b206e5700 1 -- 172.21.15.150:0/2634319672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b180678b0 con 0x7f8b181050f0 2024-05-10T04:02:28.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.306+0000 7f8b0f7fe700 1 -- 172.21.15.150:0/2634319672 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9616 (secure 0 0 0) 0x7f8b0809ad50 con 0x7f8b181050f0 2024-05-10T04:02:28.308 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24556,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-05-10T04:02:28.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.308+0000 7f8b206e5700 1 -- 172.21.15.150:0/2634319672 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8b040a1120 msgr2=0x7f8b040a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:28.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.308+0000 7f8b206e5700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8b040a1120 0x7f8b040a3630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8b18103070 tx=0x7f8b14009260 comp rx=0 tx=0).stop 2024-05-10T04:02:28.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.308+0000 7f8b206e5700 1 -- 172.21.15.150:0/2634319672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b181050f0 msgr2=0x7f8b18198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:28.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.308+0000 7f8b206e5700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b181050f0 0x7f8b18198430 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b080098e0 tx=0x7f8b08004730 comp rx=0 tx=0).stop 2024-05-10T04:02:28.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.308+0000 7f8b206e5700 1 -- 172.21.15.150:0/2634319672 shutdown_connections 2024-05-10T04:02:28.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.308+0000 7f8b206e5700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8b040a1120 0x7f8b040a3630 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.308+0000 7f8b206e5700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b18107a50 0x7f8b1819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.308+0000 7f8b206e5700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b180fff80 0x7f8b18197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.308+0000 7f8b206e5700 1 --2- 172.21.15.150:0/2634319672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b181050f0 0x7f8b18198430 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.308+0000 7f8b206e5700 1 -- 172.21.15.150:0/2634319672 >> 172.21.15.150:0/2634319672 conn(0x7f8b180fb8f0 msgr2=0x7f8b18105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:28.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.308+0000 7f8b206e5700 1 -- 172.21.15.150:0/2634319672 shutdown_connections 2024-05-10T04:02:28.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.308+0000 7f8b206e5700 1 -- 172.21.15.150:0/2634319672 wait complete. 2024-05-10T04:02:28.312 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 13 2024-05-10T04:02:28.326 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:02:28.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.464+0000 7fc61f388700 1 -- 172.21.15.150:0/2131525885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6181036c0 msgr2=0x7fc618109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:28.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.464+0000 7fc61f388700 1 --2- 172.21.15.150:0/2131525885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6181036c0 0x7fc618109f70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc61000b0a0 tx=0x7fc61000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.464+0000 7fc61f388700 1 -- 172.21.15.150:0/2131525885 shutdown_connections 2024-05-10T04:02:28.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.464+0000 7fc61f388700 1 --2- 172.21.15.150:0/2131525885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6181036c0 0x7fc618109f70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.464+0000 7fc61f388700 1 --2- 172.21.15.150:0/2131525885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc618101c00 0x7fc618102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.464+0000 7fc61f388700 1 --2- 172.21.15.150:0/2131525885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618102d20 0x7fc618103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.464+0000 7fc61f388700 1 -- 172.21.15.150:0/2131525885 >> 172.21.15.150:0/2131525885 conn(0x7fc6180fd210 msgr2=0x7fc6180ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:28.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.464+0000 7fc61f388700 1 -- 172.21.15.150:0/2131525885 shutdown_connections 2024-05-10T04:02:28.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.464+0000 7fc61f388700 1 -- 172.21.15.150:0/2131525885 wait complete. 2024-05-10T04:02:28.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61f388700 1 Processor -- start 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61f388700 1 -- start start 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61f388700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618101c00 0x7fc6181980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61f388700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc618102d20 0x7fc618198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61f388700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6181036c0 0x7fc61819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61f388700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61819fba0 con 0x7fc618102d20 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61f388700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61819fd10 con 0x7fc618101c00 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61f388700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61819fed0 con 0x7fc6181036c0 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61c923700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc618102d20 0x7fc618198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61c923700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc618102d20 0x7fc618198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34520/0 (socket says 172.21.15.150:34520) 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61c923700 1 -- 172.21.15.150:0/1698272595 learned_addr learned my addr 172.21.15.150:0/1698272595 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61d124700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618101c00 0x7fc6181980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61d925700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6181036c0 0x7fc61819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61c923700 1 -- 172.21.15.150:0/1698272595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6181036c0 msgr2=0x7fc61819f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:28.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61c923700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6181036c0 0x7fc61819f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61c923700 1 -- 172.21.15.150:0/1698272595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618101c00 msgr2=0x7fc6181980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:28.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61c923700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618101c00 0x7fc6181980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61c923700 1 -- 172.21.15.150:0/1698272595 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc608009520 con 0x7fc618102d20 2024-05-10T04:02:28.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.465+0000 7fc61d124700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618101c00 0x7fc6181980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:02:28.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.466+0000 7fc61c923700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc618102d20 0x7fc618198600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc60800eb30 tx=0x7fc60800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:28.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.466+0000 7fc60e7fc700 1 -- 172.21.15.150:0/1698272595 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc608009880 con 0x7fc618102d20 2024-05-10T04:02:28.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.466+0000 7fc61f388700 1 -- 172.21.15.150:0/1698272595 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc610009d00 con 0x7fc618102d20 2024-05-10T04:02:28.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.466+0000 7fc60e7fc700 1 -- 172.21.15.150:0/1698272595 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6080044e0 con 0x7fc618102d20 2024-05-10T04:02:28.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.466+0000 7fc60e7fc700 1 -- 172.21.15.150:0/1698272595 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc608003d80 con 0x7fc618102d20 2024-05-10T04:02:28.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.466+0000 7fc61f388700 1 -- 172.21.15.150:0/1698272595 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc618074730 con 0x7fc618102d20 2024-05-10T04:02:28.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.467+0000 7fc61f388700 1 -- 172.21.15.150:0/1698272595 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc618108e10 con 0x7fc618102d20 2024-05-10T04:02:28.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.468+0000 7fc60e7fc700 1 -- 172.21.15.150:0/1698272595 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc608003f20 con 0x7fc618102d20 2024-05-10T04:02:28.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.469+0000 7fc60e7fc700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc6040a5530 0x7fc6040a7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:28.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.469+0000 7fc61d124700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc6040a5530 0x7fc6040a7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:28.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.469+0000 7fc60e7fc700 1 -- 172.21.15.150:0/1698272595 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc608014070 con 0x7fc618102d20 2024-05-10T04:02:28.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.469+0000 7fc61d124700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc6040a5530 0x7fc6040a7a40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc618102c20 tx=0x7fc614009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:28.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.474+0000 7fc60e7fc700 1 -- 172.21.15.150:0/1698272595 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc6080965f0 con 0x7fc618102d20 2024-05-10T04:02:28.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] up:boot 2024-05-10T04:02:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: fsmap 12 up:standby 2024-05-10T04:02:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T04:02:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:02:28.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:28 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:02:28.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:28.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] up:boot 2024-05-10T04:02:28.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: fsmap 12 up:standby 2024-05-10T04:02:28.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T04:02:28.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:28.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:02:28.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:28.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:28 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:02:28.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.687+0000 7fc61f388700 1 -- 172.21.15.150:0/1698272595 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc61810d460 con 0x7fc618102d20 2024-05-10T04:02:28.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.688+0000 7fc60e7fc700 1 -- 172.21.15.150:0/1698272595 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18489 (secure 0 0 0) 0x7fc6080b2e60 con 0x7fc618102d20 2024-05-10T04:02:28.691 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:28.691 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":77,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:01:58.242469+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:02:28.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.692+0000 7fc61f388700 1 -- 172.21.15.150:0/1698272595 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc6040a5530 msgr2=0x7fc6040a7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:28.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.692+0000 7fc61f388700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc6040a5530 0x7fc6040a7a40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc618102c20 tx=0x7fc614009260 comp rx=0 tx=0).stop 2024-05-10T04:02:28.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.692+0000 7fc61f388700 1 -- 172.21.15.150:0/1698272595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc618102d20 msgr2=0x7fc618198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:28.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.692+0000 7fc61f388700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc618102d20 0x7fc618198600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc60800eb30 tx=0x7fc60800ee40 comp rx=0 tx=0).stop 2024-05-10T04:02:28.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.692+0000 7fc61f388700 1 -- 172.21.15.150:0/1698272595 shutdown_connections 2024-05-10T04:02:28.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.692+0000 7fc61f388700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc6040a5530 0x7fc6040a7a40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.692+0000 7fc61f388700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6181036c0 0x7fc61819f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.692+0000 7fc61f388700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618101c00 0x7fc6181980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.692+0000 7fc61f388700 1 --2- 172.21.15.150:0/1698272595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc618102d20 0x7fc618198600 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.692+0000 7fc61f388700 1 -- 172.21.15.150:0/1698272595 >> 172.21.15.150:0/1698272595 conn(0x7fc6180fd210 msgr2=0x7fc618105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:28.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.692+0000 7fc61f388700 1 -- 172.21.15.150:0/1698272595 shutdown_connections 2024-05-10T04:02:28.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.692+0000 7fc61f388700 1 -- 172.21.15.150:0/1698272595 wait complete. 2024-05-10T04:02:28.710 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T04:02:28.710 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:02:28.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.875+0000 7f7bb1978700 1 -- 172.21.15.150:0/4060671943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bac1031e0 msgr2=0x7f7bac1035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:28.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.875+0000 7f7bb1978700 1 --2- 172.21.15.150:0/4060671943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bac1031e0 0x7f7bac1035f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009990 tx=0x7f7b9c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.875+0000 7f7bb1978700 1 -- 172.21.15.150:0/4060671943 shutdown_connections 2024-05-10T04:02:28.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.875+0000 7f7bb1978700 1 --2- 172.21.15.150:0/4060671943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bac100570 0x7f7bac10a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.875+0000 7f7bb1978700 1 --2- 172.21.15.150:0/4060671943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bac1031e0 0x7f7bac1035f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.875+0000 7f7bb1978700 1 --2- 172.21.15.150:0/4060671943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bac0ffc20 0x7f7bac100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.875+0000 7f7bb1978700 1 -- 172.21.15.150:0/4060671943 >> 172.21.15.150:0/4060671943 conn(0x7f7bac0fb8f0 msgr2=0x7f7bac0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.875+0000 7f7bb1978700 1 -- 172.21.15.150:0/4060671943 shutdown_connections 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.875+0000 7f7bb1978700 1 -- 172.21.15.150:0/4060671943 wait complete. 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.875+0000 7f7bb1978700 1 Processor -- start 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bb1978700 1 -- start start 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bb1978700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bac0ffc20 0x7f7bac197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bb1978700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bac100570 0x7f7bac198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bb1978700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bac1031e0 0x7f7bac19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bb1978700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac19fa10 con 0x7f7bac1031e0 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bb1978700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac19fb80 con 0x7f7bac100570 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bb1978700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac19fd40 con 0x7f7bac0ffc20 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bab7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bac1031e0 0x7f7bac19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:28.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7baaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bac0ffc20 0x7f7bac197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bab7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bac1031e0 0x7f7bac19f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34534/0 (socket says 172.21.15.150:34534) 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bab7fe700 1 -- 172.21.15.150:0/3284811060 learned_addr learned my addr 172.21.15.150:0/3284811060 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7baa7fc700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bac100570 0x7f7bac198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bab7fe700 1 -- 172.21.15.150:0/3284811060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bac0ffc20 msgr2=0x7f7bac197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bab7fe700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bac0ffc20 0x7f7bac197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bab7fe700 1 -- 172.21.15.150:0/3284811060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bac100570 msgr2=0x7f7bac198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bab7fe700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bac100570 0x7f7bac198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bab7fe700 1 -- 172.21.15.150:0/3284811060 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c0095f0 con 0x7f7bac1031e0 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7baa7fc700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bac100570 0x7f7bac198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.876+0000 7f7bab7fe700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bac1031e0 0x7f7bac19f400 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400ef30 tx=0x7f7ba400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.877+0000 7f7bb0976700 1 -- 172.21.15.150:0/3284811060 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4018070 con 0x7f7bac1031e0 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.877+0000 7f7bb0976700 1 -- 172.21.15.150:0/3284811060 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba400f040 con 0x7f7bac1031e0 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.877+0000 7f7bb1978700 1 -- 172.21.15.150:0/3284811060 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bac19ffc0 con 0x7f7bac1031e0 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.877+0000 7f7bb0976700 1 -- 172.21.15.150:0/3284811060 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400cd00 con 0x7f7bac1031e0 2024-05-10T04:02:28.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.877+0000 7f7bb1978700 1 -- 172.21.15.150:0/3284811060 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bac1a0390 con 0x7f7bac1031e0 2024-05-10T04:02:28.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.878+0000 7f7bb1978700 1 -- 172.21.15.150:0/3284811060 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bac109030 con 0x7f7bac1031e0 2024-05-10T04:02:28.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.880+0000 7f7bb0976700 1 -- 172.21.15.150:0/3284811060 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7ba4003a00 con 0x7f7bac1031e0 2024-05-10T04:02:28.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.880+0000 7f7bb0976700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b940a11c0 0x7f7b940a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:28.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.880+0000 7f7baaffd700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b940a11c0 0x7f7b940a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:28.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.880+0000 7f7bb0976700 1 -- 172.21.15.150:0/3284811060 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7ba40cc930 con 0x7f7bac1031e0 2024-05-10T04:02:28.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.881+0000 7f7baaffd700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b940a11c0 0x7f7b940a36d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00b7b0 tx=0x7f7b9c011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:28.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:28.884+0000 7f7bb0976700 1 -- 172.21.15.150:0/3284811060 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7ba40965f0 con 0x7f7bac1031e0 2024-05-10T04:02:29.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.112+0000 7f7bb1978700 1 -- 172.21.15.150:0/3284811060 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bac10d3f0 con 0x7f7bac1031e0 2024-05-10T04:02:29.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.113+0000 7f7bb0976700 1 -- 172.21.15.150:0/3284811060 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ba4099c10 con 0x7f7bac1031e0 2024-05-10T04:02:29.115 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:02:29.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.115+0000 7f7bb1978700 1 -- 172.21.15.150:0/3284811060 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b940a11c0 msgr2=0x7f7b940a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:29.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.115+0000 7f7bb1978700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b940a11c0 0x7f7b940a36d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00b7b0 tx=0x7f7b9c011040 comp rx=0 tx=0).stop 2024-05-10T04:02:29.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.115+0000 7f7bb1978700 1 -- 172.21.15.150:0/3284811060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bac1031e0 msgr2=0x7f7bac19f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:29.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.115+0000 7f7bb1978700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bac1031e0 0x7f7bac19f400 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400ef30 tx=0x7f7ba400c3a0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.115+0000 7f7bb1978700 1 -- 172.21.15.150:0/3284811060 shutdown_connections 2024-05-10T04:02:29.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.115+0000 7f7bb1978700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b940a11c0 0x7f7b940a36d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.115+0000 7f7bb1978700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bac1031e0 0x7f7bac19f400 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.115+0000 7f7bb1978700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bac0ffc20 0x7f7bac197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.115+0000 7f7bb1978700 1 --2- 172.21.15.150:0/3284811060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bac100570 0x7f7bac198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.115+0000 7f7bb1978700 1 -- 172.21.15.150:0/3284811060 >> 172.21.15.150:0/3284811060 conn(0x7f7bac0fb8f0 msgr2=0x7f7bac106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:29.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.115+0000 7f7bb1978700 1 -- 172.21.15.150:0/3284811060 shutdown_connections 2024-05-10T04:02:29.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.115+0000 7f7bb1978700 1 -- 172.21.15.150:0/3284811060 wait complete. 2024-05-10T04:02:29.131 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T04:02:29.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.266+0000 7f8d22205700 1 -- 172.21.15.150:0/1630890605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c101bd0 msgr2=0x7f8d1c101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:29.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.266+0000 7f8d22205700 1 --2- 172.21.15.150:0/1630890605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c101bd0 0x7f8d1c101fe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009990 tx=0x7f8d0c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.266+0000 7f8d22205700 1 -- 172.21.15.150:0/1630890605 shutdown_connections 2024-05-10T04:02:29.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.266+0000 7f8d22205700 1 --2- 172.21.15.150:0/1630890605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d1c103690 0x7f8d1c109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.266+0000 7f8d22205700 1 --2- 172.21.15.150:0/1630890605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c101bd0 0x7f8d1c101fe0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.266+0000 7f8d22205700 1 --2- 172.21.15.150:0/1630890605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d1c102cf0 0x7f8d1c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.266+0000 7f8d22205700 1 -- 172.21.15.150:0/1630890605 >> 172.21.15.150:0/1630890605 conn(0x7f8d1c0fd1e0 msgr2=0x7f8d1c0ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:29.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.267+0000 7f8d22205700 1 -- 172.21.15.150:0/1630890605 shutdown_connections 2024-05-10T04:02:29.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.267+0000 7f8d22205700 1 -- 172.21.15.150:0/1630890605 wait complete. 2024-05-10T04:02:29.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.267+0000 7f8d22205700 1 Processor -- start 2024-05-10T04:02:29.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.267+0000 7f8d22205700 1 -- start start 2024-05-10T04:02:29.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.267+0000 7f8d22205700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c101bd0 0x7f8d1c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:29.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.267+0000 7f8d22205700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d1c102cf0 0x7f8d1c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:29.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.267+0000 7f8d22205700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d1c103690 0x7f8d1c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:29.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d22205700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c19fb30 con 0x7f8d1c101bd0 2024-05-10T04:02:29.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d22205700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c19fca0 con 0x7f8d1c103690 2024-05-10T04:02:29.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d22205700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c19fe60 con 0x7f8d1c102cf0 2024-05-10T04:02:29.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d1b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c101bd0 0x7f8d1c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:29.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d1b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c101bd0 0x7f8d1c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36888/0 (socket says 172.21.15.150:36888) 2024-05-10T04:02:29.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d1b7fe700 1 -- 172.21.15.150:0/3841980754 learned_addr learned my addr 172.21.15.150:0/3841980754 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:29.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d1bfff700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d1c103690 0x7f8d1c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:29.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d1affd700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d1c102cf0 0x7f8d1c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:29.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d1b7fe700 1 -- 172.21.15.150:0/3841980754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d1c102cf0 msgr2=0x7f8d1c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:29.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d1b7fe700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d1c102cf0 0x7f8d1c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d1b7fe700 1 -- 172.21.15.150:0/3841980754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d1c103690 msgr2=0x7f8d1c19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:29.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d1b7fe700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d1c103690 0x7f8d1c19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d1b7fe700 1 -- 172.21.15.150:0/3841980754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c0095f0 con 0x7f8d1c101bd0 2024-05-10T04:02:29.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d1b7fe700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c101bd0 0x7f8d1c198050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c006010 tx=0x7f8d0c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:29.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d18ff9700 1 -- 172.21.15.150:0/3841980754 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c01d070 con 0x7f8d1c101bd0 2024-05-10T04:02:29.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d18ff9700 1 -- 172.21.15.150:0/3841980754 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d0c005580 con 0x7f8d1c101bd0 2024-05-10T04:02:29.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d22205700 1 -- 172.21.15.150:0/3841980754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1c1a00e0 con 0x7f8d1c101bd0 2024-05-10T04:02:29.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d18ff9700 1 -- 172.21.15.150:0/3841980754 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c00fd50 con 0x7f8d1c101bd0 2024-05-10T04:02:29.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.268+0000 7f8d22205700 1 -- 172.21.15.150:0/3841980754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c1a0560 con 0x7f8d1c101bd0 2024-05-10T04:02:29.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.269+0000 7f8d22205700 1 -- 172.21.15.150:0/3841980754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c108e50 con 0x7f8d1c101bd0 2024-05-10T04:02:29.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.271+0000 7f8d18ff9700 1 -- 172.21.15.150:0/3841980754 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8d0c00f460 con 0x7f8d1c101bd0 2024-05-10T04:02:29.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.271+0000 7f8d18ff9700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8d040a11a0 0x7f8d040a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:29.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.272+0000 7f8d1affd700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8d040a11a0 0x7f8d040a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:29.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.272+0000 7f8d18ff9700 1 -- 172.21.15.150:0/3841980754 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8d0c0cd6c0 con 0x7f8d1c101bd0 2024-05-10T04:02:29.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.272+0000 7f8d1affd700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8d040a11a0 0x7f8d040a36b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c1990f0 tx=0x7f8d100073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:29.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.275+0000 7f8d18ff9700 1 -- 172.21.15.150:0/3841980754 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8d0c097380 con 0x7f8d1c101bd0 2024-05-10T04:02:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:29 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2634319672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:29 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1698272595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:29.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.481+0000 7f8d22205700 1 -- 172.21.15.150:0/3841980754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8d1c10d3b0 con 0x7f8d1c101bd0 2024-05-10T04:02:29.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.481+0000 7f8d18ff9700 1 -- 172.21.15.150:0/3841980754 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8d0c09a9a0 con 0x7f8d1c101bd0 2024-05-10T04:02:29.484 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:29.484 INFO:teuthology.orchestra.run.smithi150.stdout:[ 2024-05-10T04:02:29.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:02:29.484 INFO:teuthology.orchestra.run.smithi150.stdout: "poolnum": 1, 2024-05-10T04:02:29.484 INFO:teuthology.orchestra.run.smithi150.stdout: "poolname": ".mgr" 2024-05-10T04:02:29.484 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:02:29.485 INFO:teuthology.orchestra.run.smithi150.stdout:] 2024-05-10T04:02:29.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.483+0000 7f8d22205700 1 -- 172.21.15.150:0/3841980754 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8d040a11a0 msgr2=0x7f8d040a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:29.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.483+0000 7f8d22205700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8d040a11a0 0x7f8d040a36b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c1990f0 tx=0x7f8d100073d0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.484+0000 7f8d22205700 1 -- 172.21.15.150:0/3841980754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c101bd0 msgr2=0x7f8d1c198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:29.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.484+0000 7f8d22205700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c101bd0 0x7f8d1c198050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c006010 tx=0x7f8d0c003b90 comp rx=0 tx=0).stop 2024-05-10T04:02:29.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.484+0000 7f8d22205700 1 -- 172.21.15.150:0/3841980754 shutdown_connections 2024-05-10T04:02:29.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.484+0000 7f8d22205700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8d040a11a0 0x7f8d040a36b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.484+0000 7f8d22205700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d1c103690 0x7f8d1c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.484+0000 7f8d22205700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c101bd0 0x7f8d1c198050 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.484+0000 7f8d22205700 1 --2- 172.21.15.150:0/3841980754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d1c102cf0 0x7f8d1c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.484+0000 7f8d22205700 1 -- 172.21.15.150:0/3841980754 >> 172.21.15.150:0/3841980754 conn(0x7f8d1c0fd1e0 msgr2=0x7f8d1c105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:29.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.484+0000 7f8d22205700 1 -- 172.21.15.150:0/3841980754 shutdown_connections 2024-05-10T04:02:29.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.484+0000 7f8d22205700 1 -- 172.21.15.150:0/3841980754 wait complete. 2024-05-10T04:02:29.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:29 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2634319672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:29.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:29 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1698272595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:29 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2634319672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:29 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1698272595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:29.500 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T04:02:29.500 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T04:02:29.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.635+0000 7f863b3e2700 1 -- 172.21.15.150:0/850358054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8634107a50 msgr2=0x7f8634109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:29.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.635+0000 7f863b3e2700 1 --2- 172.21.15.150:0/850358054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8634107a50 0x7f8634109e70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f862c00b0a0 tx=0x7f862c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.635+0000 7f863b3e2700 1 -- 172.21.15.150:0/850358054 shutdown_connections 2024-05-10T04:02:29.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.635+0000 7f863b3e2700 1 --2- 172.21.15.150:0/850358054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8634107a50 0x7f8634109e70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.635+0000 7f863b3e2700 1 --2- 172.21.15.150:0/850358054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86340fff80 0x7f8634100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.635+0000 7f863b3e2700 1 --2- 172.21.15.150:0/850358054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86341050f0 0x7f8634107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.635+0000 7f863b3e2700 1 -- 172.21.15.150:0/850358054 >> 172.21.15.150:0/850358054 conn(0x7f86340fb8f0 msgr2=0x7f86340fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:29.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.635+0000 7f863b3e2700 1 -- 172.21.15.150:0/850358054 shutdown_connections 2024-05-10T04:02:29.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.635+0000 7f863b3e2700 1 -- 172.21.15.150:0/850358054 wait complete. 2024-05-10T04:02:29.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863b3e2700 1 Processor -- start 2024-05-10T04:02:29.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863b3e2700 1 -- start start 2024-05-10T04:02:29.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863b3e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86340fff80 0x7f8634197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:29.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863b3e2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86341050f0 0x7f8634198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:29.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863b3e2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8634107a50 0x7f863419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:29.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863b3e2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863419fad0 con 0x7f86341050f0 2024-05-10T04:02:29.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863b3e2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863419fc40 con 0x7f8634107a50 2024-05-10T04:02:29.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863b3e2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863419fe00 con 0x7f86340fff80 2024-05-10T04:02:29.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863897d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86341050f0 0x7f8634198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:29.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863897d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86341050f0 0x7f8634198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36900/0 (socket says 172.21.15.150:36900) 2024-05-10T04:02:29.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863897d700 1 -- 172.21.15.150:0/1756122453 learned_addr learned my addr 172.21.15.150:0/1756122453 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:29.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863997f700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8634107a50 0x7f863419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:29.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863917e700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86340fff80 0x7f8634197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:29.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863897d700 1 -- 172.21.15.150:0/1756122453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86340fff80 msgr2=0x7f8634197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:29.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863897d700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86340fff80 0x7f8634197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863897d700 1 -- 172.21.15.150:0/1756122453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8634107a50 msgr2=0x7f863419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:29.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863897d700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8634107a50 0x7f863419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:29.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863897d700 1 -- 172.21.15.150:0/1756122453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c009d00 con 0x7f86341050f0 2024-05-10T04:02:29.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.636+0000 7f863897d700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86341050f0 0x7f8634198530 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f862400e9a0 tx=0x7f862400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:29.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.637+0000 7f862a7fc700 1 -- 172.21.15.150:0/1756122453 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8624009790 con 0x7f86341050f0 2024-05-10T04:02:29.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.637+0000 7f862a7fc700 1 -- 172.21.15.150:0/1756122453 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8624010430 con 0x7f86341050f0 2024-05-10T04:02:29.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.637+0000 7f863b3e2700 1 -- 172.21.15.150:0/1756122453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86341a00e0 con 0x7f86341050f0 2024-05-10T04:02:29.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.637+0000 7f862a7fc700 1 -- 172.21.15.150:0/1756122453 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8624018680 con 0x7f86341050f0 2024-05-10T04:02:29.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.637+0000 7f863b3e2700 1 -- 172.21.15.150:0/1756122453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86341a05f0 con 0x7f86341050f0 2024-05-10T04:02:29.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.638+0000 7f863b3e2700 1 -- 172.21.15.150:0/1756122453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86340fd4a0 con 0x7f86341050f0 2024-05-10T04:02:29.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.639+0000 7f862a7fc700 1 -- 172.21.15.150:0/1756122453 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f86240105a0 con 0x7f86341050f0 2024-05-10T04:02:29.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.640+0000 7f862a7fc700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f86200a11c0 0x7f86200a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:29.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.640+0000 7f863917e700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f86200a11c0 0x7f86200a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:29.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.640+0000 7f862a7fc700 1 -- 172.21.15.150:0/1756122453 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8624014070 con 0x7f86341050f0 2024-05-10T04:02:29.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.640+0000 7f863917e700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f86200a11c0 0x7f86200a36d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8634103070 tx=0x7f86300073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:29.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.644+0000 7f862a7fc700 1 -- 172.21.15.150:0/1756122453 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f86240963f0 con 0x7f86341050f0 2024-05-10T04:02:29.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:29.844+0000 7f863b3e2700 1 -- 172.21.15.150:0/1756122453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f86340678b0 con 0x7f86341050f0 2024-05-10T04:02:30.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.163+0000 7f862a7fc700 1 -- 172.21.15.150:0/1756122453 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8624099a10 con 0x7f86341050f0 2024-05-10T04:02:30.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.166+0000 7f863b3e2700 1 -- 172.21.15.150:0/1756122453 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f86200a11c0 msgr2=0x7f86200a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:30.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.166+0000 7f863b3e2700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f86200a11c0 0x7f86200a36d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8634103070 tx=0x7f86300073d0 comp rx=0 tx=0).stop 2024-05-10T04:02:30.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.166+0000 7f863b3e2700 1 -- 172.21.15.150:0/1756122453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86341050f0 msgr2=0x7f8634198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:30.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.166+0000 7f863b3e2700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86341050f0 0x7f8634198530 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f862400e9a0 tx=0x7f862400ecb0 comp rx=0 tx=0).stop 2024-05-10T04:02:30.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.166+0000 7f863b3e2700 1 -- 172.21.15.150:0/1756122453 shutdown_connections 2024-05-10T04:02:30.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.166+0000 7f863b3e2700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f86200a11c0 0x7f86200a36d0 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8634103070 tx=0x7f86300073d0 comp rx=0 tx=0).stop 2024-05-10T04:02:30.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.166+0000 7f863b3e2700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8634107a50 0x7f863419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:30.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.166+0000 7f863b3e2700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86340fff80 0x7f8634197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:30.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.166+0000 7f863b3e2700 1 --2- 172.21.15.150:0/1756122453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86341050f0 0x7f8634198530 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:30.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.166+0000 7f863b3e2700 1 -- 172.21.15.150:0/1756122453 >> 172.21.15.150:0/1756122453 conn(0x7f86340fb8f0 msgr2=0x7f8634105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:30.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.167+0000 7f863b3e2700 1 -- 172.21.15.150:0/1756122453 shutdown_connections 2024-05-10T04:02:30.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.167+0000 7f863b3e2700 1 -- 172.21.15.150:0/1756122453 wait complete. 2024-05-10T04:02:30.176 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_metadata' created 2024-05-10T04:02:30.184 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-10T04:02:30.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.323+0000 7f66c8a31700 1 -- 172.21.15.150:0/2460571607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c40ffc20 msgr2=0x7f66c4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:30.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.323+0000 7f66c8a31700 1 --2- 172.21.15.150:0/2460571607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c40ffc20 0x7f66c4100030 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009990 tx=0x7f66b8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:30.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.323+0000 7f66c8a31700 1 -- 172.21.15.150:0/2460571607 shutdown_connections 2024-05-10T04:02:30.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.323+0000 7f66c8a31700 1 --2- 172.21.15.150:0/2460571607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66c4100570 0x7f66c410a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:30.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.323+0000 7f66c8a31700 1 --2- 172.21.15.150:0/2460571607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c41031e0 0x7f66c41035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:30.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.323+0000 7f66c8a31700 1 --2- 172.21.15.150:0/2460571607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c40ffc20 0x7f66c4100030 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:30.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.323+0000 7f66c8a31700 1 -- 172.21.15.150:0/2460571607 >> 172.21.15.150:0/2460571607 conn(0x7f66c40fb8f0 msgr2=0x7f66c40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:30.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.323+0000 7f66c8a31700 1 -- 172.21.15.150:0/2460571607 shutdown_connections 2024-05-10T04:02:30.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.323+0000 7f66c8a31700 1 -- 172.21.15.150:0/2460571607 wait complete. 2024-05-10T04:02:30.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c8a31700 1 Processor -- start 2024-05-10T04:02:30.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c8a31700 1 -- start start 2024-05-10T04:02:30.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c8a31700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c40ffc20 0x7f66c41027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:30.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c8a31700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66c4100570 0x7f66c4100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:30.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c8a31700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c41031e0 0x7f66c4101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:30.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c8a31700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c4101a70 con 0x7f66c41031e0 2024-05-10T04:02:30.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c8a31700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c4101be0 con 0x7f66c4100570 2024-05-10T04:02:30.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c8a31700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c41a1f80 con 0x7f66c40ffc20 2024-05-10T04:02:30.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c2d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c41031e0 0x7f66c4101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:30.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c2d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c41031e0 0x7f66c4101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36920/0 (socket says 172.21.15.150:36920) 2024-05-10T04:02:30.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c2d9d700 1 -- 172.21.15.150:0/3392008687 learned_addr learned my addr 172.21.15.150:0/3392008687 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:30.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c1d9b700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66c4100570 0x7f66c4100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:30.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c259c700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c40ffc20 0x7f66c41027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:30.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c2d9d700 1 -- 172.21.15.150:0/3392008687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c40ffc20 msgr2=0x7f66c41027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:30.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c2d9d700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c40ffc20 0x7f66c41027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:30.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c2d9d700 1 -- 172.21.15.150:0/3392008687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66c4100570 msgr2=0x7f66c4100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:30.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c2d9d700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66c4100570 0x7f66c4100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:30.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c2d9d700 1 -- 172.21.15.150:0/3392008687 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b80095f0 con 0x7f66c41031e0 2024-05-10T04:02:30.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.324+0000 7f66c259c700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c40ffc20 0x7f66c41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:30.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.325+0000 7f66c2d9d700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c41031e0 0x7f66c4101390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f66bc007ee0 tx=0x7f66bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:30.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.325+0000 7f66b37fe700 1 -- 172.21.15.150:0/3392008687 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc017070 con 0x7f66c41031e0 2024-05-10T04:02:30.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.325+0000 7f66b37fe700 1 -- 172.21.15.150:0/3392008687 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66bc00f040 con 0x7f66c41031e0 2024-05-10T04:02:30.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.325+0000 7f66c8a31700 1 -- 172.21.15.150:0/3392008687 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c41a21e0 con 0x7f66c41031e0 2024-05-10T04:02:30.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.325+0000 7f66b37fe700 1 -- 172.21.15.150:0/3392008687 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc005740 con 0x7f66c41031e0 2024-05-10T04:02:30.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.325+0000 7f66c8a31700 1 -- 172.21.15.150:0/3392008687 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c41a26c0 con 0x7f66c41031e0 2024-05-10T04:02:30.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.326+0000 7f66c8a31700 1 -- 172.21.15.150:0/3392008687 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c4109030 con 0x7f66c41031e0 2024-05-10T04:02:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.327+0000 7f66b37fe700 1 -- 172.21.15.150:0/3392008687 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f66bc00dc60 con 0x7f66c41031e0 2024-05-10T04:02:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.328+0000 7f66b37fe700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f66ac0a1240 0x7f66ac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.328+0000 7f66c259c700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f66ac0a1240 0x7f66ac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.328+0000 7f66b37fe700 1 -- 172.21.15.150:0/3392008687 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f66bc0cc420 con 0x7f66c41031e0 2024-05-10T04:02:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.328+0000 7f66c259c700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f66ac0a1240 0x7f66ac0a3750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f66b4005eb0 tx=0x7f66b4005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.332+0000 7f66b37fe700 1 -- 172.21.15.150:0/3392008687 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f66bc095e40 con 0x7f66c41031e0 2024-05-10T04:02:30.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:30 smithi150 ceph-mon[42477]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:30 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3284811060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:02:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:30 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3841980754' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T04:02:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:30 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1756122453' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T04:02:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:30 smithi181 ceph-mon[46602]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:30 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3284811060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:30.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:30.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:02:30.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:30 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3841980754' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T04:02:30.490 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:30 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1756122453' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T04:02:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:30 smithi161 ceph-mon[46918]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:30 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3284811060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:02:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:02:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:02:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:30 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3841980754' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T04:02:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:30 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1756122453' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T04:02:30.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:30.531+0000 7f66c8a31700 1 -- 172.21.15.150:0/3392008687 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f66c40678b0 con 0x7f66c41031e0 2024-05-10T04:02:31.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.177+0000 7f66b37fe700 1 -- 172.21.15.150:0/3392008687 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v79) v1 ==== 171+0+0 (secure 0 0 0) 0x7f66bc099460 con 0x7f66c41031e0 2024-05-10T04:02:31.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.185+0000 7f66c8a31700 1 -- 172.21.15.150:0/3392008687 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f66ac0a1240 msgr2=0x7f66ac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:31.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.185+0000 7f66c8a31700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f66ac0a1240 0x7f66ac0a3750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f66b4005eb0 tx=0x7f66b4005e40 comp rx=0 tx=0).stop 2024-05-10T04:02:31.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.185+0000 7f66c8a31700 1 -- 172.21.15.150:0/3392008687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c41031e0 msgr2=0x7f66c4101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:31.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.185+0000 7f66c8a31700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c41031e0 0x7f66c4101390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f66bc007ee0 tx=0x7f66bc00d3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.185+0000 7f66c8a31700 1 -- 172.21.15.150:0/3392008687 shutdown_connections 2024-05-10T04:02:31.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.185+0000 7f66c8a31700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f66ac0a1240 0x7f66ac0a3750 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f66b4005eb0 tx=0x7f66b4005e40 comp rx=0 tx=0).stop 2024-05-10T04:02:31.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.185+0000 7f66c8a31700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c41031e0 0x7f66c4101390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.185+0000 7f66c8a31700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c40ffc20 0x7f66c41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.185+0000 7f66c8a31700 1 --2- 172.21.15.150:0/3392008687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66c4100570 0x7f66c4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.185+0000 7f66c8a31700 1 -- 172.21.15.150:0/3392008687 >> 172.21.15.150:0/3392008687 conn(0x7f66c40fb8f0 msgr2=0x7f66c4106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:31.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.186+0000 7f66c8a31700 1 -- 172.21.15.150:0/3392008687 shutdown_connections 2024-05-10T04:02:31.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.187+0000 7f66c8a31700 1 -- 172.21.15.150:0/3392008687 wait complete. 2024-05-10T04:02:31.195 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_data' created 2024-05-10T04:02:31.208 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T04:02:31.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.350+0000 7ff84bc27700 1 -- 172.21.15.150:0/1847990727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff844107060 msgr2=0x7ff844109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:31.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.350+0000 7ff84bc27700 1 --2- 172.21.15.150:0/1847990727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff844107060 0x7ff844109480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff840009990 tx=0x7ff840009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.350+0000 7ff84bc27700 1 -- 172.21.15.150:0/1847990727 shutdown_connections 2024-05-10T04:02:31.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.350+0000 7ff84bc27700 1 --2- 172.21.15.150:0/1847990727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff844109aa0 0x7ff84410bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.350+0000 7ff84bc27700 1 --2- 172.21.15.150:0/1847990727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff844069090 0x7ff844106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.350+0000 7ff84bc27700 1 --2- 172.21.15.150:0/1847990727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff844107060 0x7ff844109480 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.350+0000 7ff84bc27700 1 -- 172.21.15.150:0/1847990727 >> 172.21.15.150:0/1847990727 conn(0x7ff8440fc0d0 msgr2=0x7ff8440fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:31.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.350+0000 7ff84bc27700 1 -- 172.21.15.150:0/1847990727 shutdown_connections 2024-05-10T04:02:31.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.350+0000 7ff84bc27700 1 -- 172.21.15.150:0/1847990727 wait complete. 2024-05-10T04:02:31.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.351+0000 7ff84bc27700 1 Processor -- start 2024-05-10T04:02:31.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.351+0000 7ff84bc27700 1 -- start start 2024-05-10T04:02:31.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.351+0000 7ff84bc27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff844069090 0x7ff84419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:31.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.351+0000 7ff84bc27700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff844107060 0x7ff84419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:31.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.351+0000 7ff84bc27700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff844109aa0 0x7ff8441a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:31.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.351+0000 7ff84bc27700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8441a1c20 con 0x7ff844109aa0 2024-05-10T04:02:31.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.351+0000 7ff84bc27700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8441a1d60 con 0x7ff844107060 2024-05-10T04:02:31.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.351+0000 7ff84bc27700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8441a1ef0 con 0x7ff844069090 2024-05-10T04:02:31.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.351+0000 7ff84a1c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff844109aa0 0x7ff8441a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:31.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.351+0000 7ff84a1c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff844109aa0 0x7ff8441a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36942/0 (socket says 172.21.15.150:36942) 2024-05-10T04:02:31.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.351+0000 7ff8491c2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff844107060 0x7ff84419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:31.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.351+0000 7ff84a1c4700 1 -- 172.21.15.150:0/235781209 learned_addr learned my addr 172.21.15.150:0/235781209 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:31.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.351+0000 7ff8499c3700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff844069090 0x7ff84419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:31.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.352+0000 7ff84a1c4700 1 -- 172.21.15.150:0/235781209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff844069090 msgr2=0x7ff84419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:31.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.352+0000 7ff84a1c4700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff844069090 0x7ff84419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.352+0000 7ff84a1c4700 1 -- 172.21.15.150:0/235781209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff844107060 msgr2=0x7ff84419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:31.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.352+0000 7ff84a1c4700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff844107060 0x7ff84419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.352+0000 7ff84a1c4700 1 -- 172.21.15.150:0/235781209 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8400095f0 con 0x7ff844109aa0 2024-05-10T04:02:31.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.352+0000 7ff8491c2700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff844107060 0x7ff84419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:02:31.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.352+0000 7ff8499c3700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff844069090 0x7ff84419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:31.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.352+0000 7ff84a1c4700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff844109aa0 0x7ff8441a1650 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff83c007da0 tx=0x7ff83c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:31.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.352+0000 7ff83affd700 1 -- 172.21.15.150:0/235781209 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c00ad00 con 0x7ff844109aa0 2024-05-10T04:02:31.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.352+0000 7ff84bc27700 1 -- 172.21.15.150:0/235781209 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8441a21d0 con 0x7ff844109aa0 2024-05-10T04:02:31.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.352+0000 7ff83affd700 1 -- 172.21.15.150:0/235781209 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff83c00ae60 con 0x7ff844109aa0 2024-05-10T04:02:31.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.352+0000 7ff83affd700 1 -- 172.21.15.150:0/235781209 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c011660 con 0x7ff844109aa0 2024-05-10T04:02:31.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.352+0000 7ff84bc27700 1 -- 172.21.15.150:0/235781209 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8441a2690 con 0x7ff844109aa0 2024-05-10T04:02:31.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.353+0000 7ff84bc27700 1 -- 172.21.15.150:0/235781209 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8440fdc80 con 0x7ff844109aa0 2024-05-10T04:02:31.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.355+0000 7ff83affd700 1 -- 172.21.15.150:0/235781209 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff83c004ad0 con 0x7ff844109aa0 2024-05-10T04:02:31.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.355+0000 7ff83affd700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff8300a11c0 0x7ff8300a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:31.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.355+0000 7ff8499c3700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff8300a11c0 0x7ff8300a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:31.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.355+0000 7ff83affd700 1 -- 172.21.15.150:0/235781209 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7879+0+0 (secure 0 0 0) 0x7ff83c0ccd10 con 0x7ff844109aa0 2024-05-10T04:02:31.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.356+0000 7ff8499c3700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff8300a11c0 0x7ff8300a36d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8440694a0 tx=0x7ff834009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:31.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.360+0000 7ff83affd700 1 -- 172.21.15.150:0/235781209 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff83c096650 con 0x7ff844109aa0 2024-05-10T04:02:31.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:31 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1756122453' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T04:02:31.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:31 smithi150 ceph-mon[42477]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T04:02:31.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:31 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3392008687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-10T04:02:31.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:31 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1756122453' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T04:02:31.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:31 smithi181 ceph-mon[46602]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T04:02:31.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:31 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3392008687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-10T04:02:31.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:31 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1756122453' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T04:02:31.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:31 smithi161 ceph-mon[46918]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T04:02:31.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:31 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3392008687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-10T04:02:31.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.603+0000 7ff84bc27700 1 -- 172.21.15.150:0/235781209 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff84410f610 con 0x7ff844109aa0 2024-05-10T04:02:31.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.610+0000 7ff83affd700 1 -- 172.21.15.150:0/235781209 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff83c099c70 con 0x7ff844109aa0 2024-05-10T04:02:31.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.614+0000 7ff84bc27700 1 -- 172.21.15.150:0/235781209 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff8300a11c0 msgr2=0x7ff8300a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:31.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.614+0000 7ff84bc27700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff8300a11c0 0x7ff8300a36d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8440694a0 tx=0x7ff834009260 comp rx=0 tx=0).stop 2024-05-10T04:02:31.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.614+0000 7ff84bc27700 1 -- 172.21.15.150:0/235781209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff844109aa0 msgr2=0x7ff8441a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:31.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.614+0000 7ff84bc27700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff844109aa0 0x7ff8441a1650 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff83c007da0 tx=0x7ff83c00a340 comp rx=0 tx=0).stop 2024-05-10T04:02:31.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.616+0000 7ff84bc27700 1 -- 172.21.15.150:0/235781209 shutdown_connections 2024-05-10T04:02:31.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.616+0000 7ff84bc27700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff8300a11c0 0x7ff8300a36d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.616+0000 7ff84bc27700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff844109aa0 0x7ff8441a1650 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.616+0000 7ff84bc27700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff844069090 0x7ff84419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.616+0000 7ff84bc27700 1 --2- 172.21.15.150:0/235781209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff844107060 0x7ff84419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.616+0000 7ff84bc27700 1 -- 172.21.15.150:0/235781209 >> 172.21.15.150:0/235781209 conn(0x7ff8440fc0d0 msgr2=0x7ff844107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:31.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.621+0000 7ff84bc27700 1 -- 172.21.15.150:0/235781209 shutdown_connections 2024-05-10T04:02:31.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.622+0000 7ff84bc27700 1 -- 172.21.15.150:0/235781209 wait complete. 2024-05-10T04:02:31.627 INFO:teuthology.orchestra.run.smithi150.stderr:new fs with metadata pool 2 and data pool 3 2024-05-10T04:02:31.649 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-10T04:02:31.650 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-10T04:02:31.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.785+0000 7f8fd6e85700 1 -- 172.21.15.150:0/947949863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fd0100ac0 msgr2=0x7f8fd0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:31.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.785+0000 7f8fd6e85700 1 --2- 172.21.15.150:0/947949863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fd0100ac0 0x7f8fd0100ed0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009990 tx=0x7f8fc4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.785+0000 7f8fd6e85700 1 -- 172.21.15.150:0/947949863 shutdown_connections 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.785+0000 7f8fd6e85700 1 --2- 172.21.15.150:0/947949863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0102580 0x7f8fd0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.785+0000 7f8fd6e85700 1 --2- 172.21.15.150:0/947949863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fd0100ac0 0x7f8fd0100ed0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.785+0000 7f8fd6e85700 1 --2- 172.21.15.150:0/947949863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd0101be0 0x7f8fd0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.785+0000 7f8fd6e85700 1 -- 172.21.15.150:0/947949863 >> 172.21.15.150:0/947949863 conn(0x7f8fd00fc0d0 msgr2=0x7f8fd00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.785+0000 7f8fd6e85700 1 -- 172.21.15.150:0/947949863 shutdown_connections 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.785+0000 7f8fd6e85700 1 -- 172.21.15.150:0/947949863 wait complete. 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fd6e85700 1 Processor -- start 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fd6e85700 1 -- start start 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fd6e85700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0100ac0 0x7f8fd0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fd6e85700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fd0101be0 0x7f8fd0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fd6e85700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd0102580 0x7f8fd019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fd6e85700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd019f8e0 con 0x7f8fd0101be0 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fd6e85700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd019fa20 con 0x7f8fd0102580 2024-05-10T04:02:31.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fd6e85700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd019fbb0 con 0x7f8fd0100ac0 2024-05-10T04:02:31.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fcffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fd0101be0 0x7f8fd0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:31.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fcffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fd0101be0 0x7f8fd0198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36956/0 (socket says 172.21.15.150:36956) 2024-05-10T04:02:31.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fcffff700 1 -- 172.21.15.150:0/2706230166 learned_addr learned my addr 172.21.15.150:0/2706230166 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:31.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fd5422700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd0102580 0x7f8fd019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:31.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fd4c21700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0100ac0 0x7f8fd0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:31.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fcffff700 1 -- 172.21.15.150:0/2706230166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0100ac0 msgr2=0x7f8fd0197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:31.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fcffff700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0100ac0 0x7f8fd0197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fcffff700 1 -- 172.21.15.150:0/2706230166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd0102580 msgr2=0x7f8fd019f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:31.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.786+0000 7f8fcffff700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd0102580 0x7f8fd019f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:31.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.787+0000 7f8fcffff700 1 -- 172.21.15.150:0/2706230166 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc40095f0 con 0x7f8fd0101be0 2024-05-10T04:02:31.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.787+0000 7f8fcffff700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fd0101be0 0x7f8fd0198410 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00c9e0 tx=0x7f8fbc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:31.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.787+0000 7f8fcdffb700 1 -- 172.21.15.150:0/2706230166 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc007ac0 con 0x7f8fd0101be0 2024-05-10T04:02:31.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.787+0000 7f8fcdffb700 1 -- 172.21.15.150:0/2706230166 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fbc007c20 con 0x7f8fd0101be0 2024-05-10T04:02:31.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.787+0000 7f8fd6e85700 1 -- 172.21.15.150:0/2706230166 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd019fe90 con 0x7f8fd0101be0 2024-05-10T04:02:31.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.787+0000 7f8fcdffb700 1 -- 172.21.15.150:0/2706230166 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc018660 con 0x7f8fd0101be0 2024-05-10T04:02:31.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.787+0000 7f8fd6e85700 1 -- 172.21.15.150:0/2706230166 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd01a0310 con 0x7f8fd0101be0 2024-05-10T04:02:31.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.788+0000 7f8fd6e85700 1 -- 172.21.15.150:0/2706230166 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd00694d0 con 0x7f8fd0101be0 2024-05-10T04:02:31.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.789+0000 7f8fcdffb700 1 -- 172.21.15.150:0/2706230166 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8fbc00f450 con 0x7f8fd0101be0 2024-05-10T04:02:31.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.790+0000 7f8fcdffb700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8fc00a1240 0x7f8fc00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:31.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.790+0000 7f8fd4c21700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8fc00a1240 0x7f8fc00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:31.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.790+0000 7f8fcdffb700 1 -- 172.21.15.150:0/2706230166 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8fbc0cca10 con 0x7f8fd0101be0 2024-05-10T04:02:31.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.790+0000 7f8fd4c21700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8fc00a1240 0x7f8fc00a3750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40038c0 tx=0x7f8fc4005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:31.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.793+0000 7f8fcdffb700 1 -- 172.21.15.150:0/2706230166 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8fbc096360 con 0x7f8fd0101be0 2024-05-10T04:02:32.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:31.999+0000 7f8fd6e85700 1 -- 172.21.15.150:0/2706230166 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f8fd010c4f0 con 0x7f8fd0101be0 2024-05-10T04:02:32.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:32 smithi150 ceph-mon[42477]: pgmap v219: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:32 smithi150 ceph-mon[42477]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T04:02:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:32 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3392008687' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-10T04:02:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:32 smithi150 ceph-mon[42477]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T04:02:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:32 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/235781209' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T04:02:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:32 smithi150 ceph-mon[42477]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-10T04:02:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:32 smithi150 ceph-mon[42477]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T04:02:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:32 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/235781209' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T04:02:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:32 smithi150 ceph-mon[42477]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-10T04:02:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:32 smithi150 ceph-mon[42477]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-10T04:02:32.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:32 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2706230166' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-10T04:02:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:32 smithi181 ceph-mon[46602]: pgmap v219: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:32 smithi181 ceph-mon[46602]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T04:02:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:32 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3392008687' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-10T04:02:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:32 smithi181 ceph-mon[46602]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T04:02:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:32 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/235781209' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T04:02:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:32 smithi181 ceph-mon[46602]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-10T04:02:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:32 smithi181 ceph-mon[46602]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T04:02:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:32 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/235781209' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T04:02:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:32 smithi181 ceph-mon[46602]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-10T04:02:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:32 smithi181 ceph-mon[46602]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-10T04:02:32.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:32 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2706230166' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-10T04:02:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:32 smithi161 ceph-mon[46918]: pgmap v219: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:02:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:32 smithi161 ceph-mon[46918]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T04:02:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:32 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3392008687' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-10T04:02:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:32 smithi161 ceph-mon[46918]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T04:02:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:32 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/235781209' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T04:02:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:32 smithi161 ceph-mon[46918]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-10T04:02:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:32 smithi161 ceph-mon[46918]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T04:02:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:32 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/235781209' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T04:02:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:32 smithi161 ceph-mon[46918]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-10T04:02:32.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:32 smithi161 ceph-mon[46918]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-10T04:02:32.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:32 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2706230166' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-10T04:02:32.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.616+0000 7f8fcdffb700 1 -- 172.21.15.150:0/2706230166 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) v1 ==== 157+0+0 (secure 0 0 0) 0x7f8fbc099980 con 0x7f8fd0101be0 2024-05-10T04:02:32.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.618+0000 7f8fd6e85700 1 -- 172.21.15.150:0/2706230166 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8fc00a1240 msgr2=0x7f8fc00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:32.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.619+0000 7f8fd6e85700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8fc00a1240 0x7f8fc00a3750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40038c0 tx=0x7f8fc4005040 comp rx=0 tx=0).stop 2024-05-10T04:02:32.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.619+0000 7f8fd6e85700 1 -- 172.21.15.150:0/2706230166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fd0101be0 msgr2=0x7f8fd0198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:32.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.619+0000 7f8fd6e85700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fd0101be0 0x7f8fd0198410 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00c9e0 tx=0x7f8fbc00ccf0 comp rx=0 tx=0).stop 2024-05-10T04:02:32.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.619+0000 7f8fd6e85700 1 -- 172.21.15.150:0/2706230166 shutdown_connections 2024-05-10T04:02:32.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.619+0000 7f8fd6e85700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8fc00a1240 0x7f8fc00a3750 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:32.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.619+0000 7f8fd6e85700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd0102580 0x7f8fd019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:32.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.619+0000 7f8fd6e85700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0100ac0 0x7f8fd0197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:32.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.619+0000 7f8fd6e85700 1 --2- 172.21.15.150:0/2706230166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fd0101be0 0x7f8fd0198410 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:32.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.619+0000 7f8fd6e85700 1 -- 172.21.15.150:0/2706230166 >> 172.21.15.150:0/2706230166 conn(0x7f8fd00fc0d0 msgr2=0x7f8fd00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:32.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.619+0000 7f8fd6e85700 1 -- 172.21.15.150:0/2706230166 shutdown_connections 2024-05-10T04:02:32.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.619+0000 7f8fd6e85700 1 -- 172.21.15.150:0/2706230166 wait complete. 2024-05-10T04:02:32.636 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T04:02:32.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.778+0000 7f627cab9700 1 -- 172.21.15.150:0/1815522571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6278102580 msgr2=0x7f6278108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:32.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.778+0000 7f627cab9700 1 --2- 172.21.15.150:0/1815522571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6278102580 0x7f6278108e30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f627000b0a0 tx=0x7f627000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:32.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.778+0000 7f627cab9700 1 -- 172.21.15.150:0/1815522571 shutdown_connections 2024-05-10T04:02:32.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.778+0000 7f627cab9700 1 --2- 172.21.15.150:0/1815522571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6278102580 0x7f6278108e30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:32.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.778+0000 7f627cab9700 1 --2- 172.21.15.150:0/1815522571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6278100ac0 0x7f6278100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:32.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.778+0000 7f627cab9700 1 --2- 172.21.15.150:0/1815522571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6278101be0 0x7f6278101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:32.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.778+0000 7f627cab9700 1 -- 172.21.15.150:0/1815522571 >> 172.21.15.150:0/1815522571 conn(0x7f62780fc0d0 msgr2=0x7f62780fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:32.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.778+0000 7f627cab9700 1 -- 172.21.15.150:0/1815522571 shutdown_connections 2024-05-10T04:02:32.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.778+0000 7f627cab9700 1 -- 172.21.15.150:0/1815522571 wait complete. 2024-05-10T04:02:32.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.779+0000 7f627cab9700 1 Processor -- start 2024-05-10T04:02:32.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.779+0000 7f627cab9700 1 -- start start 2024-05-10T04:02:32.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.779+0000 7f627cab9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6278100ac0 0x7f62781980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:32.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.779+0000 7f627cab9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6278101be0 0x7f62781985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:32.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.779+0000 7f627cab9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6278102580 0x7f627819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:32.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.779+0000 7f627cab9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627819fb80 con 0x7f6278101be0 2024-05-10T04:02:32.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.779+0000 7f627cab9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627819fcf0 con 0x7f6278100ac0 2024-05-10T04:02:32.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.779+0000 7f627cab9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627819feb0 con 0x7f6278102580 2024-05-10T04:02:32.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.779+0000 7f6275d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6278101be0 0x7f62781985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:32.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f6275d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6278101be0 0x7f62781985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36968/0 (socket says 172.21.15.150:36968) 2024-05-10T04:02:32.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f6275d9b700 1 -- 172.21.15.150:0/2031506913 learned_addr learned my addr 172.21.15.150:0/2031506913 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:32.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f627659c700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6278100ac0 0x7f62781980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:32.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f6275d9b700 1 -- 172.21.15.150:0/2031506913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6278102580 msgr2=0x7f627819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:32.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f6275d9b700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6278102580 0x7f627819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:32.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f6275d9b700 1 -- 172.21.15.150:0/2031506913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6278100ac0 msgr2=0x7f62781980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:32.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f6275d9b700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6278100ac0 0x7f62781980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:32.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f6275d9b700 1 -- 172.21.15.150:0/2031506913 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6270009d00 con 0x7f6278101be0 2024-05-10T04:02:32.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f627659c700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6278100ac0 0x7f62781980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:32.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f6275d9b700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6278101be0 0x7f62781985e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f626c00d950 tx=0x7f626c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:32.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f62677fe700 1 -- 172.21.15.150:0/2031506913 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c009b40 con 0x7f6278101be0 2024-05-10T04:02:32.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f62677fe700 1 -- 172.21.15.150:0/2031506913 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f626c009ca0 con 0x7f6278101be0 2024-05-10T04:02:32.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f627cab9700 1 -- 172.21.15.150:0/2031506913 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62781a0190 con 0x7f6278101be0 2024-05-10T04:02:32.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f62677fe700 1 -- 172.21.15.150:0/2031506913 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c00f620 con 0x7f6278101be0 2024-05-10T04:02:32.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.780+0000 7f627cab9700 1 -- 172.21.15.150:0/2031506913 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62781a06f0 con 0x7f6278101be0 2024-05-10T04:02:32.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.781+0000 7f627cab9700 1 -- 172.21.15.150:0/2031506913 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62780694d0 con 0x7f6278101be0 2024-05-10T04:02:32.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.782+0000 7f62677fe700 1 -- 172.21.15.150:0/2031506913 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f626c010460 con 0x7f6278101be0 2024-05-10T04:02:32.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.783+0000 7f62677fe700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f62600a11a0 0x7f62600a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:32.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.783+0000 7f627659c700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f62600a11a0 0x7f62600a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:32.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.783+0000 7f62677fe700 1 -- 172.21.15.150:0/2031506913 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f626c0ccbd0 con 0x7f6278101be0 2024-05-10T04:02:32.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.783+0000 7f627659c700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f62600a11a0 0x7f62600a36b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6278101ae0 tx=0x7f6268009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:32.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.787+0000 7f62677fe700 1 -- 172.21.15.150:0/2031506913 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f626c096380 con 0x7f6278101be0 2024-05-10T04:02:32.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:32.983+0000 7f627cab9700 1 -- 172.21.15.150:0/2031506913 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f627810c380 con 0x7f6278101be0 2024-05-10T04:02:33.963 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:33 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] up:active 2024-05-10T04:02:33.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:33 smithi150 ceph-mon[42477]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:33.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:33 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2706230166' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-10T04:02:33.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:33 smithi150 ceph-mon[42477]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T04:02:33.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:33 smithi150 ceph-mon[42477]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:33.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:33 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2031506913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T04:02:33.965 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:33 smithi150 ceph-mon[42477]: pgmap v223: 97 pgs: 16 active+clean, 5 creating+activating, 16 creating+peering, 60 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-10T04:02:33.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:33 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] up:active 2024-05-10T04:02:33.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:33 smithi181 ceph-mon[46602]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:33.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:33 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2706230166' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-10T04:02:33.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:33 smithi181 ceph-mon[46602]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T04:02:33.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:33 smithi181 ceph-mon[46602]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:33.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:33 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2031506913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T04:02:33.989 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:33 smithi181 ceph-mon[46602]: pgmap v223: 97 pgs: 16 active+clean, 5 creating+activating, 16 creating+peering, 60 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-10T04:02:33.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:33 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] up:active 2024-05-10T04:02:33.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:33 smithi161 ceph-mon[46918]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:33.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:33 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2706230166' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-10T04:02:33.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:33 smithi161 ceph-mon[46918]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T04:02:33.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:33 smithi161 ceph-mon[46918]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:33.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:33 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2031506913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T04:02:33.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:33 smithi161 ceph-mon[46918]: pgmap v223: 97 pgs: 16 active+clean, 5 creating+activating, 16 creating+peering, 60 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-10T04:02:34.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.628+0000 7f62677fe700 1 -- 172.21.15.150:0/2031506913 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) v1 ==== 228+0+0 (secure 0 0 0) 0x7f626c020100 con 0x7f6278101be0 2024-05-10T04:02:34.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.635+0000 7f627cab9700 1 -- 172.21.15.150:0/2031506913 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f62600a11a0 msgr2=0x7f62600a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:34.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.635+0000 7f627cab9700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f62600a11a0 0x7f62600a36b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6278101ae0 tx=0x7f6268009670 comp rx=0 tx=0).stop 2024-05-10T04:02:34.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.635+0000 7f627cab9700 1 -- 172.21.15.150:0/2031506913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6278101be0 msgr2=0x7f62781985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:34.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.635+0000 7f627cab9700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6278101be0 0x7f62781985e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f626c00d950 tx=0x7f626c00dd10 comp rx=0 tx=0).stop 2024-05-10T04:02:34.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.636+0000 7f627cab9700 1 -- 172.21.15.150:0/2031506913 shutdown_connections 2024-05-10T04:02:34.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.636+0000 7f627cab9700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f62600a11a0 0x7f62600a36b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:34.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.636+0000 7f627cab9700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6278102580 0x7f627819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:34.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.636+0000 7f627cab9700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6278100ac0 0x7f62781980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:34.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.636+0000 7f627cab9700 1 --2- 172.21.15.150:0/2031506913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6278101be0 0x7f62781985e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:34.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.636+0000 7f627cab9700 1 -- 172.21.15.150:0/2031506913 >> 172.21.15.150:0/2031506913 conn(0x7f62780fc0d0 msgr2=0x7f62780fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:34.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.636+0000 7f627cab9700 1 -- 172.21.15.150:0/2031506913 shutdown_connections 2024-05-10T04:02:34.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.636+0000 7f627cab9700 1 -- 172.21.15.150:0/2031506913 wait complete. 2024-05-10T04:02:34.639 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_data_ec' created 2024-05-10T04:02:34.655 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-10T04:02:34.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.791+0000 7fc05ad3f700 1 -- 172.21.15.150:0/429950126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0541025d0 msgr2=0x7fc054109050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:34.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.791+0000 7fc05ad3f700 1 --2- 172.21.15.150:0/429950126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0541025d0 0x7fc054109050 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00b0a0 tx=0x7fc04c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:34.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.792+0000 7fc05ad3f700 1 -- 172.21.15.150:0/429950126 shutdown_connections 2024-05-10T04:02:34.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.792+0000 7fc05ad3f700 1 --2- 172.21.15.150:0/429950126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0541025d0 0x7fc054109050 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:34.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.792+0000 7fc05ad3f700 1 --2- 172.21.15.150:0/429950126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc054101090 0x7fc0541014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:34.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.792+0000 7fc05ad3f700 1 --2- 172.21.15.150:0/429950126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc054101a70 0x7fc054101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:34.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.792+0000 7fc05ad3f700 1 -- 172.21.15.150:0/429950126 >> 172.21.15.150:0/429950126 conn(0x7fc0540fb8f0 msgr2=0x7fc0540fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:34.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.792+0000 7fc05ad3f700 1 -- 172.21.15.150:0/429950126 shutdown_connections 2024-05-10T04:02:34.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.792+0000 7fc05ad3f700 1 -- 172.21.15.150:0/429950126 wait complete. 2024-05-10T04:02:34.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.792+0000 7fc05ad3f700 1 Processor -- start 2024-05-10T04:02:34.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.792+0000 7fc05ad3f700 1 -- start start 2024-05-10T04:02:34.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc05ad3f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc054101090 0x7fc054196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:34.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc05ad3f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc054101a70 0x7fc054197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:34.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc05ad3f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0541025d0 0x7fc05419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:34.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc05ad3f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05419e9e0 con 0x7fc0541025d0 2024-05-10T04:02:34.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc05ad3f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05419eb20 con 0x7fc054101a70 2024-05-10T04:02:34.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc05ad3f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05419ecb0 con 0x7fc054101090 2024-05-10T04:02:34.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc0592dc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0541025d0 0x7fc05419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:34.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc0592dc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0541025d0 0x7fc05419e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36990/0 (socket says 172.21.15.150:36990) 2024-05-10T04:02:34.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc0592dc700 1 -- 172.21.15.150:0/4188437516 learned_addr learned my addr 172.21.15.150:0/4188437516 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:34.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc053fff700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc054101a70 0x7fc054197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:34.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc058adb700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc054101090 0x7fc054196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:34.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc0592dc700 1 -- 172.21.15.150:0/4188437516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc054101090 msgr2=0x7fc054196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:34.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc0592dc700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc054101090 0x7fc054196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:34.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc0592dc700 1 -- 172.21.15.150:0/4188437516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc054101a70 msgr2=0x7fc054197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:34.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc0592dc700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc054101a70 0x7fc054197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:34.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc0592dc700 1 -- 172.21.15.150:0/4188437516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c009d00 con 0x7fc0541025d0 2024-05-10T04:02:34.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc053fff700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc054101a70 0x7fc054197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:34.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc0592dc700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0541025d0 0x7fc05419e410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00b070 tx=0x7fc04c004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:34.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.793+0000 7fc051ffb700 1 -- 172.21.15.150:0/4188437516 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c00e040 con 0x7fc0541025d0 2024-05-10T04:02:34.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.794+0000 7fc051ffb700 1 -- 172.21.15.150:0/4188437516 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc04c00cec0 con 0x7fc0541025d0 2024-05-10T04:02:34.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.794+0000 7fc05ad3f700 1 -- 172.21.15.150:0/4188437516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05419ef30 con 0x7fc0541025d0 2024-05-10T04:02:34.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.794+0000 7fc051ffb700 1 -- 172.21.15.150:0/4188437516 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c004020 con 0x7fc0541025d0 2024-05-10T04:02:34.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.794+0000 7fc05ad3f700 1 -- 172.21.15.150:0/4188437516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05419f350 con 0x7fc0541025d0 2024-05-10T04:02:34.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.795+0000 7fc05ad3f700 1 -- 172.21.15.150:0/4188437516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc054107ef0 con 0x7fc0541025d0 2024-05-10T04:02:34.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.796+0000 7fc051ffb700 1 -- 172.21.15.150:0/4188437516 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc04c003a00 con 0x7fc0541025d0 2024-05-10T04:02:34.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.797+0000 7fc051ffb700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc0440a1240 0x7fc0440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:34.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.797+0000 7fc058adb700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc0440a1240 0x7fc0440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:34.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.797+0000 7fc051ffb700 1 -- 172.21.15.150:0/4188437516 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fc04c0ce640 con 0x7fc0541025d0 2024-05-10T04:02:34.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.797+0000 7fc058adb700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc0440a1240 0x7fc0440a3750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc048005ea0 tx=0x7fc048005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:34.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.801+0000 7fc051ffb700 1 -- 172.21.15.150:0/4188437516 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc04c097c30 con 0x7fc0541025d0 2024-05-10T04:02:34.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:34 smithi150 ceph-mon[42477]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T04:02:34.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:34 smithi150 ceph-mon[42477]: Cluster is now healthy 2024-05-10T04:02:34.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:34 smithi150 ceph-mon[42477]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T04:02:34.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:34 smithi181 ceph-mon[46602]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T04:02:34.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:34 smithi181 ceph-mon[46602]: Cluster is now healthy 2024-05-10T04:02:34.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:34 smithi181 ceph-mon[46602]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T04:02:34.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:34 smithi161 ceph-mon[46918]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T04:02:34.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:34 smithi161 ceph-mon[46918]: Cluster is now healthy 2024-05-10T04:02:34.994 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:34 smithi161 ceph-mon[46918]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T04:02:35.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:34.999+0000 7fc05ad3f700 1 -- 172.21.15.150:0/4188437516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fc05410c4f0 con 0x7fc0541025d0 2024-05-10T04:02:35.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.632+0000 7fc051ffb700 1 -- 172.21.15.150:0/4188437516 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc04c09b250 con 0x7fc0541025d0 2024-05-10T04:02:35.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.637+0000 7fc05ad3f700 1 -- 172.21.15.150:0/4188437516 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc0440a1240 msgr2=0x7fc0440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:35.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.637+0000 7fc05ad3f700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc0440a1240 0x7fc0440a3750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc048005ea0 tx=0x7fc048005e30 comp rx=0 tx=0).stop 2024-05-10T04:02:35.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.637+0000 7fc05ad3f700 1 -- 172.21.15.150:0/4188437516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0541025d0 msgr2=0x7fc05419e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:35.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.637+0000 7fc05ad3f700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0541025d0 0x7fc05419e410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00b070 tx=0x7fc04c004d30 comp rx=0 tx=0).stop 2024-05-10T04:02:35.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.646+0000 7fc05ad3f700 1 -- 172.21.15.150:0/4188437516 shutdown_connections 2024-05-10T04:02:35.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.646+0000 7fc05ad3f700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc0440a1240 0x7fc0440a3750 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:35.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.646+0000 7fc05ad3f700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0541025d0 0x7fc05419e410 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:35.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.646+0000 7fc05ad3f700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc054101090 0x7fc054196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:35.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.646+0000 7fc05ad3f700 1 --2- 172.21.15.150:0/4188437516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc054101a70 0x7fc054197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:35.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.646+0000 7fc05ad3f700 1 -- 172.21.15.150:0/4188437516 >> 172.21.15.150:0/4188437516 conn(0x7fc0540fb8f0 msgr2=0x7fc054105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:35.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.650+0000 7fc05ad3f700 1 -- 172.21.15.150:0/4188437516 shutdown_connections 2024-05-10T04:02:35.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.651+0000 7fc05ad3f700 1 -- 172.21.15.150:0/4188437516 wait complete. 2024-05-10T04:02:35.657 INFO:teuthology.orchestra.run.smithi150.stderr:set pool 4 allow_ec_overwrites to true 2024-05-10T04:02:35.671 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-10T04:02:35.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.806+0000 7f05ffde1700 1 -- 172.21.15.150:0/2089742590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f81010b0 msgr2=0x7f05f81014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:35.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.806+0000 7f05ffde1700 1 --2- 172.21.15.150:0/2089742590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f81010b0 0x7f05f81014c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009990 tx=0x7f05e8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:35.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.806+0000 7f05ffde1700 1 -- 172.21.15.150:0/2089742590 shutdown_connections 2024-05-10T04:02:35.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.806+0000 7f05ffde1700 1 --2- 172.21.15.150:0/2089742590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05f81025f0 0x7f05f8109070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:35.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.806+0000 7f05ffde1700 1 --2- 172.21.15.150:0/2089742590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f81010b0 0x7f05f81014c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:35.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.806+0000 7f05ffde1700 1 --2- 172.21.15.150:0/2089742590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05f8101a90 0x7f05f8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:35.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.806+0000 7f05ffde1700 1 -- 172.21.15.150:0/2089742590 >> 172.21.15.150:0/2089742590 conn(0x7f05f80fb8f0 msgr2=0x7f05f80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:35.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.807+0000 7f05ffde1700 1 -- 172.21.15.150:0/2089742590 shutdown_connections 2024-05-10T04:02:35.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.807+0000 7f05ffde1700 1 -- 172.21.15.150:0/2089742590 wait complete. 2024-05-10T04:02:35.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.807+0000 7f05ffde1700 1 Processor -- start 2024-05-10T04:02:35.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.807+0000 7f05ffde1700 1 -- start start 2024-05-10T04:02:35.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.807+0000 7f05ffde1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05f81010b0 0x7f05f8104ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:35.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.807+0000 7f05ffde1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05f8101a90 0x7f05f8103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:35.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.807+0000 7f05ffde1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f81025f0 0x7f05f81036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.807+0000 7f05ffde1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f8103d80 con 0x7f05f81025f0 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.807+0000 7f05ffde1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f8103ef0 con 0x7f05f81010b0 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.807+0000 7f05ffde1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f81a2010 con 0x7f05f8101a90 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.807+0000 7f05fe37e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f81025f0 0x7f05f81036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.807+0000 7f05fe37e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f81025f0 0x7f05f81036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36994/0 (socket says 172.21.15.150:36994) 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05fe37e700 1 -- 172.21.15.150:0/3147309096 learned_addr learned my addr 172.21.15.150:0/3147309096 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05fdb7d700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05f81010b0 0x7f05f8104ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05fd37c700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05f8101a90 0x7f05f8103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05fe37e700 1 -- 172.21.15.150:0/3147309096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05f8101a90 msgr2=0x7f05f8103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05fe37e700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05f8101a90 0x7f05f8103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05fe37e700 1 -- 172.21.15.150:0/3147309096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05f81010b0 msgr2=0x7f05f8104ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05fe37e700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05f81010b0 0x7f05f8104ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05fe37e700 1 -- 172.21.15.150:0/3147309096 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e80095f0 con 0x7f05f81025f0 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05fd37c700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05f8101a90 0x7f05f8103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:35.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05fe37e700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f81025f0 0x7f05f81036a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f05f0007ee0 tx=0x7f05f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:35.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05eeffd700 1 -- 172.21.15.150:0/3147309096 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0017070 con 0x7f05f81025f0 2024-05-10T04:02:35.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05eeffd700 1 -- 172.21.15.150:0/3147309096 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05f000f040 con 0x7f05f81025f0 2024-05-10T04:02:35.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05eeffd700 1 -- 172.21.15.150:0/3147309096 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0005740 con 0x7f05f81025f0 2024-05-10T04:02:35.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05ffde1700 1 -- 172.21.15.150:0/3147309096 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f81a2270 con 0x7f05f81025f0 2024-05-10T04:02:35.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.808+0000 7f05ffde1700 1 -- 172.21.15.150:0/3147309096 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f81a2780 con 0x7f05f81025f0 2024-05-10T04:02:35.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.809+0000 7f05ffde1700 1 -- 172.21.15.150:0/3147309096 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f80678b0 con 0x7f05f81025f0 2024-05-10T04:02:35.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.810+0000 7f05eeffd700 1 -- 172.21.15.150:0/3147309096 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f05f000dc60 con 0x7f05f81025f0 2024-05-10T04:02:35.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.811+0000 7f05eeffd700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f05e40a1240 0x7f05e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:35.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.811+0000 7f05fdb7d700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f05e40a1240 0x7f05e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:35.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.811+0000 7f05eeffd700 1 -- 172.21.15.150:0/3147309096 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f05f00cc7e0 con 0x7f05f81025f0 2024-05-10T04:02:35.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.811+0000 7f05fdb7d700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f05e40a1240 0x7f05e40a3750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05e8006010 tx=0x7f05e801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:35.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:35.816+0000 7f05eeffd700 1 -- 172.21.15.150:0/3147309096 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f05f0095da0 con 0x7f05f81025f0 2024-05-10T04:02:35.963 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:35 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2031506913' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T04:02:35.963 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:35 smithi150 ceph-mon[42477]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T04:02:35.963 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:35 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4188437516' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-10T04:02:35.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:35 smithi150 ceph-mon[42477]: pgmap v226: 129 pgs: 62 active+clean, 5 creating+activating, 16 creating+peering, 46 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 9 op/s 2024-05-10T04:02:35.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:35 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2031506913' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T04:02:35.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:35 smithi181 ceph-mon[46602]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T04:02:35.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:35 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4188437516' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-10T04:02:35.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:35 smithi181 ceph-mon[46602]: pgmap v226: 129 pgs: 62 active+clean, 5 creating+activating, 16 creating+peering, 46 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 9 op/s 2024-05-10T04:02:35.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:35 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2031506913' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T04:02:35.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:35 smithi161 ceph-mon[46918]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T04:02:35.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:35 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4188437516' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-10T04:02:35.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:35 smithi161 ceph-mon[46918]: pgmap v226: 129 pgs: 62 active+clean, 5 creating+activating, 16 creating+peering, 46 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 9 op/s 2024-05-10T04:02:36.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.061+0000 7f05ffde1700 1 -- 172.21.15.150:0/3147309096 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f05f810c320 con 0x7f05f81025f0 2024-05-10T04:02:36.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.068+0000 7f05eeffd700 1 -- 172.21.15.150:0/3147309096 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f05f00993c0 con 0x7f05f81025f0 2024-05-10T04:02:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.071+0000 7f05ffde1700 1 -- 172.21.15.150:0/3147309096 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f05e40a1240 msgr2=0x7f05e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.071+0000 7f05ffde1700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f05e40a1240 0x7f05e40a3750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05e8006010 tx=0x7f05e801a040 comp rx=0 tx=0).stop 2024-05-10T04:02:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.071+0000 7f05ffde1700 1 -- 172.21.15.150:0/3147309096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f81025f0 msgr2=0x7f05f81036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.071+0000 7f05ffde1700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f81025f0 0x7f05f81036a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f05f0007ee0 tx=0x7f05f000d3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.071+0000 7f05ffde1700 1 -- 172.21.15.150:0/3147309096 shutdown_connections 2024-05-10T04:02:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.071+0000 7f05ffde1700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f05e40a1240 0x7f05e40a3750 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.071+0000 7f05ffde1700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f81025f0 0x7f05f81036a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.071+0000 7f05ffde1700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05f81010b0 0x7f05f8104ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.071+0000 7f05ffde1700 1 --2- 172.21.15.150:0/3147309096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05f8101a90 0x7f05f8103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.071+0000 7f05ffde1700 1 -- 172.21.15.150:0/3147309096 >> 172.21.15.150:0/3147309096 conn(0x7f05f80fb8f0 msgr2=0x7f05f80fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.071+0000 7f05ffde1700 1 -- 172.21.15.150:0/3147309096 shutdown_connections 2024-05-10T04:02:36.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.071+0000 7f05ffde1700 1 -- 172.21.15.150:0/3147309096 wait complete. 2024-05-10T04:02:36.075 INFO:teuthology.orchestra.run.smithi150.stderr:added data pool 4 to fsmap 2024-05-10T04:02:36.086 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.217+0000 7f428d4de700 1 -- 172.21.15.150:0/1946051537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4288107980 msgr2=0x7f4288109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.217+0000 7f428d4de700 1 --2- 172.21.15.150:0/1946051537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4288107980 0x7f4288109da0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f428000b0a0 tx=0x7f428000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.217+0000 7f428d4de700 1 -- 172.21.15.150:0/1946051537 shutdown_connections 2024-05-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.217+0000 7f428d4de700 1 --2- 172.21.15.150:0/1946051537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4288107980 0x7f4288109da0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.217+0000 7f428d4de700 1 --2- 172.21.15.150:0/1946051537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42881026c0 0x7f4288104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.217+0000 7f428d4de700 1 --2- 172.21.15.150:0/1946051537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4288105020 0x7f4288107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.217+0000 7f428d4de700 1 -- 172.21.15.150:0/1946051537 >> 172.21.15.150:0/1946051537 conn(0x7f42880fc0e0 msgr2=0x7f42880fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.217+0000 7f428d4de700 1 -- 172.21.15.150:0/1946051537 shutdown_connections 2024-05-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.217+0000 7f428d4de700 1 -- 172.21.15.150:0/1946051537 wait complete. 2024-05-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.217+0000 7f428d4de700 1 Processor -- start 2024-05-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f428d4de700 1 -- start start 2024-05-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f428d4de700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42881026c0 0x7f4288101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f428d4de700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4288105020 0x7f42881005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f428d4de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4288107980 0x7f4288100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f428d4de700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42881011f0 con 0x7f42881026c0 2024-05-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f428d4de700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4288101360 con 0x7f4288105020 2024-05-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f428d4de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42881a2220 con 0x7f4288107980 2024-05-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f4286ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42881026c0 0x7f4288101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f4286ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42881026c0 0x7f4288101f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37010/0 (socket says 172.21.15.150:37010) 2024-05-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f4286ffd700 1 -- 172.21.15.150:0/1766470077 learned_addr learned my addr 172.21.15.150:0/1766470077 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f42867fc700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4288105020 0x7f42881005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f42877fe700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4288107980 0x7f4288100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f4286ffd700 1 -- 172.21.15.150:0/1766470077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4288107980 msgr2=0x7f4288100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f4286ffd700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4288107980 0x7f4288100b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f4286ffd700 1 -- 172.21.15.150:0/1766470077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4288105020 msgr2=0x7f42881005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f4286ffd700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4288105020 0x7f42881005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f4286ffd700 1 -- 172.21.15.150:0/1766470077 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4280009d00 con 0x7f42881026c0 2024-05-10T04:02:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f42867fc700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4288105020 0x7f42881005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f42877fe700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4288107980 0x7f4288100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f4286ffd700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42881026c0 0x7f4288101f30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f427800e970 tx=0x7f427800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.218+0000 7f426ffff700 1 -- 172.21.15.150:0/1766470077 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4278009790 con 0x7f42881026c0 2024-05-10T04:02:36.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.219+0000 7f426ffff700 1 -- 172.21.15.150:0/1766470077 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4278004cf0 con 0x7f42881026c0 2024-05-10T04:02:36.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.219+0000 7f428d4de700 1 -- 172.21.15.150:0/1766470077 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42881a2480 con 0x7f42881026c0 2024-05-10T04:02:36.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.219+0000 7f426ffff700 1 -- 172.21.15.150:0/1766470077 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4278005680 con 0x7f42881026c0 2024-05-10T04:02:36.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.219+0000 7f428d4de700 1 -- 172.21.15.150:0/1766470077 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42881a2960 con 0x7f42881026c0 2024-05-10T04:02:36.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.219+0000 7f428d4de700 1 -- 172.21.15.150:0/1766470077 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42880fdc90 con 0x7f42881026c0 2024-05-10T04:02:36.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.221+0000 7f426ffff700 1 -- 172.21.15.150:0/1766470077 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4278004750 con 0x7f42881026c0 2024-05-10T04:02:36.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.222+0000 7f426ffff700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f42700a1120 0x7f42700a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:36.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.222+0000 7f42867fc700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f42700a1120 0x7f42700a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:36.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.222+0000 7f426ffff700 1 -- 172.21.15.150:0/1766470077 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4278014070 con 0x7f42881026c0 2024-05-10T04:02:36.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.222+0000 7f42867fc700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f42700a1120 0x7f42700a3630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f427c005950 tx=0x7f427c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:36.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.226+0000 7f426ffff700 1 -- 172.21.15.150:0/1766470077 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f42780965b0 con 0x7f42881026c0 2024-05-10T04:02:36.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.467+0000 7f428d4de700 1 -- 172.21.15.150:0/1766470077 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4288068730 con 0x7f42881026c0 2024-05-10T04:02:36.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.468+0000 7f426ffff700 1 -- 172.21.15.150:0/1766470077 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f4278099bd0 con 0x7f42881026c0 2024-05-10T04:02:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:36.471 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24556,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:02:36.063637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-10T04:02:36.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.471+0000 7f428d4de700 1 -- 172.21.15.150:0/1766470077 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f42700a1120 msgr2=0x7f42700a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.471+0000 7f428d4de700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f42700a1120 0x7f42700a3630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f427c005950 tx=0x7f427c0058e0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.471+0000 7f428d4de700 1 -- 172.21.15.150:0/1766470077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42881026c0 msgr2=0x7f4288101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.471+0000 7f428d4de700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42881026c0 0x7f4288101f30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f427800e970 tx=0x7f427800ed30 comp rx=0 tx=0).stop 2024-05-10T04:02:36.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.471+0000 7f428d4de700 1 -- 172.21.15.150:0/1766470077 shutdown_connections 2024-05-10T04:02:36.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.471+0000 7f428d4de700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f42700a1120 0x7f42700a3630 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.471+0000 7f428d4de700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4288107980 0x7f4288100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.471+0000 7f428d4de700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42881026c0 0x7f4288101f30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.471+0000 7f428d4de700 1 --2- 172.21.15.150:0/1766470077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4288105020 0x7f42881005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.471+0000 7f428d4de700 1 -- 172.21.15.150:0/1766470077 >> 172.21.15.150:0/1766470077 conn(0x7f42880fc0e0 msgr2=0x7f42881035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:36.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.471+0000 7f428d4de700 1 -- 172.21.15.150:0/1766470077 shutdown_connections 2024-05-10T04:02:36.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.471+0000 7f428d4de700 1 -- 172.21.15.150:0/1766470077 wait complete. 2024-05-10T04:02:36.474 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 17 2024-05-10T04:02:36.486 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:02:36.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.616+0000 7f22a83ab700 1 -- 172.21.15.150:0/4050935468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a0100570 msgr2=0x7f22a010a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.616+0000 7f22a83ab700 1 --2- 172.21.15.150:0/4050935468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a0100570 0x7f22a010a190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f229800b0a0 tx=0x7f229800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.616+0000 7f22a83ab700 1 -- 172.21.15.150:0/4050935468 shutdown_connections 2024-05-10T04:02:36.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.616+0000 7f22a83ab700 1 --2- 172.21.15.150:0/4050935468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a0100570 0x7f22a010a190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.616+0000 7f22a83ab700 1 --2- 172.21.15.150:0/4050935468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a01031e0 0x7f22a01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.616+0000 7f22a83ab700 1 --2- 172.21.15.150:0/4050935468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a00ffc20 0x7f22a0100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.616+0000 7f22a83ab700 1 -- 172.21.15.150:0/4050935468 >> 172.21.15.150:0/4050935468 conn(0x7f22a00fb8f0 msgr2=0x7f22a00fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:36.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.616+0000 7f22a83ab700 1 -- 172.21.15.150:0/4050935468 shutdown_connections 2024-05-10T04:02:36.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.616+0000 7f22a83ab700 1 -- 172.21.15.150:0/4050935468 wait complete. 2024-05-10T04:02:36.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.617+0000 7f22a83ab700 1 Processor -- start 2024-05-10T04:02:36.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.617+0000 7f22a83ab700 1 -- start start 2024-05-10T04:02:36.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.617+0000 7f22a83ab700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a00ffc20 0x7f22a01980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:36.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.617+0000 7f22a83ab700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a0100570 0x7f22a01985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:36.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.617+0000 7f22a83ab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a01031e0 0x7f22a019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:36.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.617+0000 7f22a83ab700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a019fb80 con 0x7f22a0100570 2024-05-10T04:02:36.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.617+0000 7f22a83ab700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a019fcf0 con 0x7f22a00ffc20 2024-05-10T04:02:36.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.617+0000 7f22a83ab700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a019feb0 con 0x7f22a01031e0 2024-05-10T04:02:36.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.617+0000 7f22a5946700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a0100570 0x7f22a01985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:36.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.617+0000 7f22a5946700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a0100570 0x7f22a01985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37024/0 (socket says 172.21.15.150:37024) 2024-05-10T04:02:36.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.617+0000 7f22a5946700 1 -- 172.21.15.150:0/478055092 learned_addr learned my addr 172.21.15.150:0/478055092 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:36.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.617+0000 7f22a6147700 1 --2- 172.21.15.150:0/478055092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a00ffc20 0x7f22a01980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:36.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.618+0000 7f22a5946700 1 -- 172.21.15.150:0/478055092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a01031e0 msgr2=0x7f22a019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.618+0000 7f22a5946700 1 --2- 172.21.15.150:0/478055092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a01031e0 0x7f22a019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.618+0000 7f22a5946700 1 -- 172.21.15.150:0/478055092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a00ffc20 msgr2=0x7f22a01980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.618+0000 7f22a5946700 1 --2- 172.21.15.150:0/478055092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a00ffc20 0x7f22a01980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.618+0000 7f22a5946700 1 -- 172.21.15.150:0/478055092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2298009d00 con 0x7f22a0100570 2024-05-10T04:02:36.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.618+0000 7f22a5946700 1 --2- 172.21.15.150:0/478055092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a0100570 0x7f22a01985e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f229c00d980 tx=0x7f229c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:36.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.618+0000 7f22977fe700 1 -- 172.21.15.150:0/478055092 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c004220 con 0x7f22a0100570 2024-05-10T04:02:36.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.618+0000 7f22977fe700 1 -- 172.21.15.150:0/478055092 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f229c007640 con 0x7f22a0100570 2024-05-10T04:02:36.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.618+0000 7f22a83ab700 1 -- 172.21.15.150:0/478055092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a01a0190 con 0x7f22a0100570 2024-05-10T04:02:36.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.618+0000 7f22977fe700 1 -- 172.21.15.150:0/478055092 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c005610 con 0x7f22a0100570 2024-05-10T04:02:36.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.618+0000 7f22a83ab700 1 -- 172.21.15.150:0/478055092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a01a06a0 con 0x7f22a0100570 2024-05-10T04:02:36.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.619+0000 7f22a83ab700 1 -- 172.21.15.150:0/478055092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a0109030 con 0x7f22a0100570 2024-05-10T04:02:36.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.622+0000 7f22977fe700 1 -- 172.21.15.150:0/478055092 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f229c004750 con 0x7f22a0100570 2024-05-10T04:02:36.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.623+0000 7f22977fe700 1 --2- 172.21.15.150:0/478055092 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f228c0a1120 0x7f228c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:36.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.623+0000 7f22977fe700 1 -- 172.21.15.150:0/478055092 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f229c0cce70 con 0x7f22a0100570 2024-05-10T04:02:36.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.623+0000 7f22a6147700 1 --2- 172.21.15.150:0/478055092 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f228c0a1120 0x7f228c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:36.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.623+0000 7f22a6147700 1 --2- 172.21.15.150:0/478055092 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f228c0a1120 0x7f228c0a3630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f22a0068120 tx=0x7f2290009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:36.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.627+0000 7f22977fe700 1 -- 172.21.15.150:0/478055092 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f229c096410 con 0x7f22a0100570 2024-05-10T04:02:36.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.826+0000 7f22a83ab700 1 -- 172.21.15.150:0/478055092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22a00678b0 con 0x7f22a0100570 2024-05-10T04:02:36.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.828+0000 7f22977fe700 1 -- 172.21.15.150:0/478055092 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24027 (secure 0 0 0) 0x7f229c016090 con 0x7f22a0100570 2024-05-10T04:02:36.829 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:36.830 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":85,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:02:36.063310+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:02:36.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.830+0000 7f22a83ab700 1 -- 172.21.15.150:0/478055092 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f228c0a1120 msgr2=0x7f228c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.830+0000 7f22a83ab700 1 --2- 172.21.15.150:0/478055092 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f228c0a1120 0x7f228c0a3630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f22a0068120 tx=0x7f2290009260 comp rx=0 tx=0).stop 2024-05-10T04:02:36.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.830+0000 7f22a83ab700 1 -- 172.21.15.150:0/478055092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a0100570 msgr2=0x7f22a01985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.830+0000 7f22a83ab700 1 --2- 172.21.15.150:0/478055092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a0100570 0x7f22a01985e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f229c00d980 tx=0x7f229c00dc90 comp rx=0 tx=0).stop 2024-05-10T04:02:36.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.830+0000 7f22a83ab700 1 -- 172.21.15.150:0/478055092 shutdown_connections 2024-05-10T04:02:36.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.830+0000 7f22a83ab700 1 --2- 172.21.15.150:0/478055092 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f228c0a1120 0x7f228c0a3630 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.830+0000 7f22a83ab700 1 --2- 172.21.15.150:0/478055092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a01031e0 0x7f22a019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.830+0000 7f22a83ab700 1 --2- 172.21.15.150:0/478055092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a00ffc20 0x7f22a01980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.830+0000 7f22a83ab700 1 --2- 172.21.15.150:0/478055092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a0100570 0x7f22a01985e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.830+0000 7f22a83ab700 1 -- 172.21.15.150:0/478055092 >> 172.21.15.150:0/478055092 conn(0x7f22a00fb8f0 msgr2=0x7f22a0106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:36.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.830+0000 7f22a83ab700 1 -- 172.21.15.150:0/478055092 shutdown_connections 2024-05-10T04:02:36.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.830+0000 7f22a83ab700 1 -- 172.21.15.150:0/478055092 wait complete. 2024-05-10T04:02:36.843 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:36 smithi150 ceph-mon[42477]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T04:02:36.843 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:36 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4188437516' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-10T04:02:36.844 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:36 smithi150 ceph-mon[42477]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T04:02:36.844 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:36 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3147309096' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-10T04:02:36.844 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:36 smithi150 ceph-mon[42477]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T04:02:36.844 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:36 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3147309096' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-10T04:02:36.844 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:36 smithi150 ceph-mon[42477]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:36.844 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:36 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1766470077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:36.846 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:02:36.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.978+0000 7f1fa0645700 1 -- 172.21.15.150:0/3819632543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98101be0 msgr2=0x7f1f98101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.978+0000 7f1fa0645700 1 --2- 172.21.15.150:0/3819632543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98101be0 0x7f1f98101ff0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009990 tx=0x7f1f8c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.978+0000 7f1fa0645700 1 -- 172.21.15.150:0/3819632543 shutdown_connections 2024-05-10T04:02:36.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.978+0000 7f1fa0645700 1 --2- 172.21.15.150:0/3819632543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f98102580 0x7f1f98108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.978+0000 7f1fa0645700 1 --2- 172.21.15.150:0/3819632543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f98100ac0 0x7f1f98100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.978+0000 7f1fa0645700 1 --2- 172.21.15.150:0/3819632543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98101be0 0x7f1f98101ff0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.978+0000 7f1fa0645700 1 -- 172.21.15.150:0/3819632543 >> 172.21.15.150:0/3819632543 conn(0x7f1f980fc0d0 msgr2=0x7f1f980fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:36.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.978+0000 7f1fa0645700 1 -- 172.21.15.150:0/3819632543 shutdown_connections 2024-05-10T04:02:36.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.978+0000 7f1fa0645700 1 -- 172.21.15.150:0/3819632543 wait complete. 2024-05-10T04:02:36.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.978+0000 7f1fa0645700 1 Processor -- start 2024-05-10T04:02:36.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.978+0000 7f1fa0645700 1 -- start start 2024-05-10T04:02:36.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1fa0645700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f98100ac0 0x7f1f98068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:36.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1fa0645700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f98101be0 0x7f1f9819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:36.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1fa0645700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98102580 0x7f1f9819f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:36.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1fa0645700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9819f9e0 con 0x7f1f98102580 2024-05-10T04:02:36.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1fa0645700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9819fb50 con 0x7f1f98100ac0 2024-05-10T04:02:36.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1fa0645700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9819fd10 con 0x7f1f98101be0 2024-05-10T04:02:36.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f9ebe2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98102580 0x7f1f9819f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:36.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f9ebe2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98102580 0x7f1f9819f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37038/0 (socket says 172.21.15.150:37038) 2024-05-10T04:02:36.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f9ebe2700 1 -- 172.21.15.150:0/288531607 learned_addr learned my addr 172.21.15.150:0/288531607 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:36.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f9e3e1700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f98100ac0 0x7f1f98068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:36.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f9dbe0700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f98101be0 0x7f1f9819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:36.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f9ebe2700 1 -- 172.21.15.150:0/288531607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f98101be0 msgr2=0x7f1f9819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f9ebe2700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f98101be0 0x7f1f9819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f9ebe2700 1 -- 172.21.15.150:0/288531607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f98100ac0 msgr2=0x7f1f98068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:36.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f9ebe2700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f98100ac0 0x7f1f98068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:36.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f9ebe2700 1 -- 172.21.15.150:0/288531607 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c0095f0 con 0x7f1f98102580 2024-05-10T04:02:36.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f9ebe2700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98102580 0x7f1f9819f2e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1f90007d70 tx=0x7f1f9000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:36.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f8b7fe700 1 -- 172.21.15.150:0/288531607 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9000ac90 con 0x7f1f98102580 2024-05-10T04:02:36.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f8b7fe700 1 -- 172.21.15.150:0/288531607 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f9000adf0 con 0x7f1f98102580 2024-05-10T04:02:36.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1fa0645700 1 -- 172.21.15.150:0/288531607 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9819fff0 con 0x7f1f98102580 2024-05-10T04:02:36.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1f8b7fe700 1 -- 172.21.15.150:0/288531607 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f900115e0 con 0x7f1f98102580 2024-05-10T04:02:36.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.979+0000 7f1fa0645700 1 -- 172.21.15.150:0/288531607 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f981a0410 con 0x7f1f98102580 2024-05-10T04:02:36.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.980+0000 7f1fa0645700 1 -- 172.21.15.150:0/288531607 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f98107cd0 con 0x7f1f98102580 2024-05-10T04:02:36.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.982+0000 7f1f8b7fe700 1 -- 172.21.15.150:0/288531607 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1f90012420 con 0x7f1f98102580 2024-05-10T04:02:36.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.982+0000 7f1f8b7fe700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1f840a1120 0x7f1f840a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:36.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.982+0000 7f1f8b7fe700 1 -- 172.21.15.150:0/288531607 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1f900cd010 con 0x7f1f98102580 2024-05-10T04:02:36.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.982+0000 7f1f9e3e1700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1f840a1120 0x7f1f840a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:36.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.983+0000 7f1f9e3e1700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1f840a1120 0x7f1f840a3630 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1f98101ae0 tx=0x7f1f94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:36.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:36.986+0000 7f1f8b7fe700 1 -- 172.21.15.150:0/288531607 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1f900965b0 con 0x7f1f98102580 2024-05-10T04:02:36.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:36 smithi181 ceph-mon[46602]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T04:02:36.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:36 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4188437516' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-10T04:02:36.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:36 smithi181 ceph-mon[46602]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T04:02:36.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:36 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3147309096' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-10T04:02:36.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:36 smithi181 ceph-mon[46602]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T04:02:36.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:36 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3147309096' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-10T04:02:36.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:36 smithi181 ceph-mon[46602]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:36.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:36 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1766470077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:36.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:36 smithi161 ceph-mon[46918]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T04:02:36.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:36 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4188437516' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-10T04:02:36.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:36 smithi161 ceph-mon[46918]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T04:02:36.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:36 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3147309096' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-10T04:02:36.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:36 smithi161 ceph-mon[46918]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T04:02:36.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:36 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3147309096' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-10T04:02:36.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:36 smithi161 ceph-mon[46918]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:36.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:36 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1766470077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:37.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.185+0000 7f1fa0645700 1 -- 172.21.15.150:0/288531607 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f9810c2c0 con 0x7f1f98102580 2024-05-10T04:02:37.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.187+0000 7f1f8b7fe700 1 -- 172.21.15.150:0/288531607 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24027 (secure 0 0 0) 0x7f1f90099bd0 con 0x7f1f98102580 2024-05-10T04:02:37.188 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:37.189 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":85,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:02:36.063310+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:02:37.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.189+0000 7f1fa0645700 1 -- 172.21.15.150:0/288531607 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1f840a1120 msgr2=0x7f1f840a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:37.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.189+0000 7f1fa0645700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1f840a1120 0x7f1f840a3630 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1f98101ae0 tx=0x7f1f94009260 comp rx=0 tx=0).stop 2024-05-10T04:02:37.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.189+0000 7f1fa0645700 1 -- 172.21.15.150:0/288531607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98102580 msgr2=0x7f1f9819f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:37.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.189+0000 7f1fa0645700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98102580 0x7f1f9819f2e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1f90007d70 tx=0x7f1f9000a340 comp rx=0 tx=0).stop 2024-05-10T04:02:37.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.189+0000 7f1fa0645700 1 -- 172.21.15.150:0/288531607 shutdown_connections 2024-05-10T04:02:37.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.189+0000 7f1fa0645700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1f840a1120 0x7f1f840a3630 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.189+0000 7f1fa0645700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98102580 0x7f1f9819f2e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.189+0000 7f1fa0645700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f98100ac0 0x7f1f98068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.189+0000 7f1fa0645700 1 --2- 172.21.15.150:0/288531607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f98101be0 0x7f1f9819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.189+0000 7f1fa0645700 1 -- 172.21.15.150:0/288531607 >> 172.21.15.150:0/288531607 conn(0x7f1f980fc0d0 msgr2=0x7f1f98104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:37.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.189+0000 7f1fa0645700 1 -- 172.21.15.150:0/288531607 shutdown_connections 2024-05-10T04:02:37.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.189+0000 7f1fa0645700 1 -- 172.21.15.150:0/288531607 wait complete. 2024-05-10T04:02:37.221 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:02:37.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.353+0000 7fd44c6a1700 1 -- 172.21.15.150:0/801882564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd444100470 msgr2=0x7fd44410a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:37.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.353+0000 7fd44c6a1700 1 --2- 172.21.15.150:0/801882564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd444100470 0x7fd44410a090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00b0a0 tx=0x7fd43c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.353+0000 7fd44c6a1700 1 -- 172.21.15.150:0/801882564 shutdown_connections 2024-05-10T04:02:37.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.353+0000 7fd44c6a1700 1 --2- 172.21.15.150:0/801882564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd444100470 0x7fd44410a090 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.353+0000 7fd44c6a1700 1 --2- 172.21.15.150:0/801882564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4441030e0 0x7fd4441034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.353+0000 7fd44c6a1700 1 --2- 172.21.15.150:0/801882564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4440ffb20 0x7fd4440fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.353+0000 7fd44c6a1700 1 -- 172.21.15.150:0/801882564 >> 172.21.15.150:0/801882564 conn(0x7fd4440fb7b0 msgr2=0x7fd4440fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:37.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.353+0000 7fd44c6a1700 1 -- 172.21.15.150:0/801882564 shutdown_connections 2024-05-10T04:02:37.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.353+0000 7fd44c6a1700 1 -- 172.21.15.150:0/801882564 wait complete. 2024-05-10T04:02:37.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44c6a1700 1 Processor -- start 2024-05-10T04:02:37.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44c6a1700 1 -- start start 2024-05-10T04:02:37.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44c6a1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4440ffb20 0x7fd444198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:37.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44c6a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd444100470 0x7fd444198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:37.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44c6a1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4441030e0 0x7fd44419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:37.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44c6a1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44419fab0 con 0x7fd4440ffb20 2024-05-10T04:02:37.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44c6a1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44419fbf0 con 0x7fd4441030e0 2024-05-10T04:02:37.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44c6a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44419fd80 con 0x7fd444100470 2024-05-10T04:02:37.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44a43d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4440ffb20 0x7fd444198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:37.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44a43d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4440ffb20 0x7fd444198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37058/0 (socket says 172.21.15.150:37058) 2024-05-10T04:02:37.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44a43d700 1 -- 172.21.15.150:0/2959649145 learned_addr learned my addr 172.21.15.150:0/2959649145 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:37.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd449c3c700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd444100470 0x7fd444198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:37.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44ac3e700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4441030e0 0x7fd44419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:37.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44a43d700 1 -- 172.21.15.150:0/2959649145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd444100470 msgr2=0x7fd444198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:37.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44a43d700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd444100470 0x7fd444198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44a43d700 1 -- 172.21.15.150:0/2959649145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4441030e0 msgr2=0x7fd44419f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:37.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44a43d700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4441030e0 0x7fd44419f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.354+0000 7fd44a43d700 1 -- 172.21.15.150:0/2959649145 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c009d00 con 0x7fd4440ffb20 2024-05-10T04:02:37.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.355+0000 7fd44a43d700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4440ffb20 0x7fd444198010 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd44000e970 tx=0x7fd44000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:37.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.355+0000 7fd43b7fe700 1 -- 172.21.15.150:0/2959649145 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440004220 con 0x7fd4440ffb20 2024-05-10T04:02:37.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.355+0000 7fd43b7fe700 1 -- 172.21.15.150:0/2959649145 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd440004380 con 0x7fd4440ffb20 2024-05-10T04:02:37.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.355+0000 7fd44c6a1700 1 -- 172.21.15.150:0/2959649145 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4441a0060 con 0x7fd4440ffb20 2024-05-10T04:02:37.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.355+0000 7fd43b7fe700 1 -- 172.21.15.150:0/2959649145 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440003c40 con 0x7fd4440ffb20 2024-05-10T04:02:37.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.355+0000 7fd44c6a1700 1 -- 172.21.15.150:0/2959649145 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4441a0570 con 0x7fd4440ffb20 2024-05-10T04:02:37.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.356+0000 7fd44c6a1700 1 -- 172.21.15.150:0/2959649145 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd444108f30 con 0x7fd4440ffb20 2024-05-10T04:02:37.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.357+0000 7fd43b7fe700 1 -- 172.21.15.150:0/2959649145 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd44000c7e0 con 0x7fd4440ffb20 2024-05-10T04:02:37.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.358+0000 7fd43b7fe700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd4300a1120 0x7fd4300a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:37.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.358+0000 7fd449c3c700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd4300a1120 0x7fd4300a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:37.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.358+0000 7fd43b7fe700 1 -- 172.21.15.150:0/2959649145 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd440014070 con 0x7fd4440ffb20 2024-05-10T04:02:37.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.359+0000 7fd449c3c700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd4300a1120 0x7fd4300a3630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd4441990b0 tx=0x7fd43400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:37.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.362+0000 7fd43b7fe700 1 -- 172.21.15.150:0/2959649145 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd440096370 con 0x7fd4440ffb20 2024-05-10T04:02:37.492 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:02:37 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-b[83518]: 2024-05-10T04:02:37.085+0000 7f6a83230700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T04:02:37.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.604+0000 7fd44c6a1700 1 -- 172.21.15.150:0/2959649145 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd444067810 con 0x7fd4440ffb20 2024-05-10T04:02:37.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.604+0000 7fd43b7fe700 1 -- 172.21.15.150:0/2959649145 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fd440099990 con 0x7fd4440ffb20 2024-05-10T04:02:37.606 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:37.606 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24556,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:02:36.063637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-10T04:02:37.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.607+0000 7fd44c6a1700 1 -- 172.21.15.150:0/2959649145 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd4300a1120 msgr2=0x7fd4300a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:37.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.607+0000 7fd44c6a1700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd4300a1120 0x7fd4300a3630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd4441990b0 tx=0x7fd43400b410 comp rx=0 tx=0).stop 2024-05-10T04:02:37.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.607+0000 7fd44c6a1700 1 -- 172.21.15.150:0/2959649145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4440ffb20 msgr2=0x7fd444198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:37.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.607+0000 7fd44c6a1700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4440ffb20 0x7fd444198010 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd44000e970 tx=0x7fd44000ec80 comp rx=0 tx=0).stop 2024-05-10T04:02:37.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.607+0000 7fd44c6a1700 1 -- 172.21.15.150:0/2959649145 shutdown_connections 2024-05-10T04:02:37.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.607+0000 7fd44c6a1700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd4300a1120 0x7fd4300a3630 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.607+0000 7fd44c6a1700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4441030e0 0x7fd44419f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.607+0000 7fd44c6a1700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4440ffb20 0x7fd444198010 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.607+0000 7fd44c6a1700 1 --2- 172.21.15.150:0/2959649145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd444100470 0x7fd444198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.607+0000 7fd44c6a1700 1 -- 172.21.15.150:0/2959649145 >> 172.21.15.150:0/2959649145 conn(0x7fd4440fb7b0 msgr2=0x7fd444106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:37.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.607+0000 7fd44c6a1700 1 -- 172.21.15.150:0/2959649145 shutdown_connections 2024-05-10T04:02:37.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.607+0000 7fd44c6a1700 1 -- 172.21.15.150:0/2959649145 wait complete. 2024-05-10T04:02:37.610 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 17 2024-05-10T04:02:37.621 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:02:37.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.755+0000 7f6805f74700 1 -- 172.21.15.150:0/2106938225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6800100ac0 msgr2=0x7f6800100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:37.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.755+0000 7f6805f74700 1 --2- 172.21.15.150:0/2106938225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6800100ac0 0x7f6800100ed0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009990 tx=0x7f67f0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.755+0000 7f6805f74700 1 -- 172.21.15.150:0/2106938225 shutdown_connections 2024-05-10T04:02:37.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.755+0000 7f6805f74700 1 --2- 172.21.15.150:0/2106938225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6800102580 0x7f6800108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.755+0000 7f6805f74700 1 --2- 172.21.15.150:0/2106938225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6800100ac0 0x7f6800100ed0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.755+0000 7f6805f74700 1 --2- 172.21.15.150:0/2106938225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6800101be0 0x7f6800101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.755+0000 7f6805f74700 1 -- 172.21.15.150:0/2106938225 >> 172.21.15.150:0/2106938225 conn(0x7f68000fc0d0 msgr2=0x7f68000fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:37.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.755+0000 7f6805f74700 1 -- 172.21.15.150:0/2106938225 shutdown_connections 2024-05-10T04:02:37.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.755+0000 7f6805f74700 1 -- 172.21.15.150:0/2106938225 wait complete. 2024-05-10T04:02:37.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.756+0000 7f6805f74700 1 Processor -- start 2024-05-10T04:02:37.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.756+0000 7f6805f74700 1 -- start start 2024-05-10T04:02:37.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.756+0000 7f6805f74700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6800100ac0 0x7f6800196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:37.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.756+0000 7f6805f74700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6800101be0 0x7f6800197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:37.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.756+0000 7f6805f74700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6800102580 0x7f680019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:37.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.756+0000 7f6805f74700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680019e9d0 con 0x7f6800101be0 2024-05-10T04:02:37.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.756+0000 7f6805f74700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680019eb40 con 0x7f6800100ac0 2024-05-10T04:02:37.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.756+0000 7f6805f74700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680019ed00 con 0x7f6800102580 2024-05-10T04:02:37.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.756+0000 7f67feffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6800101be0 0x7f6800197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:37.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.756+0000 7f67feffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6800101be0 0x7f6800197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37078/0 (socket says 172.21.15.150:37078) 2024-05-10T04:02:37.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.756+0000 7f67feffd700 1 -- 172.21.15.150:0/25934268 learned_addr learned my addr 172.21.15.150:0/25934268 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:37.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.756+0000 7f67fffff700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6800102580 0x7f680019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:37.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.756+0000 7f67ff7fe700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6800100ac0 0x7f6800196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:37.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.757+0000 7f67feffd700 1 -- 172.21.15.150:0/25934268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6800102580 msgr2=0x7f680019e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:37.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.757+0000 7f67feffd700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6800102580 0x7f680019e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.757+0000 7f67feffd700 1 -- 172.21.15.150:0/25934268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6800100ac0 msgr2=0x7f6800196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:37.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.757+0000 7f67feffd700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6800100ac0 0x7f6800196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.757+0000 7f67feffd700 1 -- 172.21.15.150:0/25934268 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f00095f0 con 0x7f6800101be0 2024-05-10T04:02:37.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.757+0000 7f67feffd700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6800101be0 0x7f6800197430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f67f400e9a0 tx=0x7f67f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:37.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.757+0000 7f67fcff9700 1 -- 172.21.15.150:0/25934268 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4009790 con 0x7f6800101be0 2024-05-10T04:02:37.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.757+0000 7f6805f74700 1 -- 172.21.15.150:0/25934268 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680019efe0 con 0x7f6800101be0 2024-05-10T04:02:37.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.757+0000 7f67fcff9700 1 -- 172.21.15.150:0/25934268 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67f4010430 con 0x7f6800101be0 2024-05-10T04:02:37.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.757+0000 7f67fcff9700 1 -- 172.21.15.150:0/25934268 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4018680 con 0x7f6800101be0 2024-05-10T04:02:37.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.757+0000 7f6805f74700 1 -- 172.21.15.150:0/25934268 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680019f460 con 0x7f6800101be0 2024-05-10T04:02:37.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.758+0000 7f6805f74700 1 -- 172.21.15.150:0/25934268 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6800107cd0 con 0x7f6800101be0 2024-05-10T04:02:37.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.759+0000 7f67fcff9700 1 -- 172.21.15.150:0/25934268 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f67f40105a0 con 0x7f6800101be0 2024-05-10T04:02:37.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.760+0000 7f67fcff9700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f67e80a1120 0x7f67e80a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:37.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.760+0000 7f67fcff9700 1 -- 172.21.15.150:0/25934268 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f67f4014070 con 0x7f6800101be0 2024-05-10T04:02:37.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.760+0000 7f67ff7fe700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f67e80a1120 0x7f67e80a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:37.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.760+0000 7f67ff7fe700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f67e80a1120 0x7f67e80a3630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67f000b7b0 tx=0x7f67f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:37.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.763+0000 7f67fcff9700 1 -- 172.21.15.150:0/25934268 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f67f4096460 con 0x7f6800101be0 2024-05-10T04:02:37.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.961+0000 7f6805f74700 1 -- 172.21.15.150:0/25934268 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f680010c2f0 con 0x7f6800101be0 2024-05-10T04:02:37.963 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:37 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/478055092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:37.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:37 smithi150 ceph-mon[42477]: pgmap v229: 129 pgs: 108 active+clean, 1 creating+peering, 20 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2024-05-10T04:02:37.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:37 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/288531607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:37.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:37 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2959649145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:37.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.963+0000 7f67fcff9700 1 -- 172.21.15.150:0/25934268 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24027 (secure 0 0 0) 0x7f67f4026090 con 0x7f6800101be0 2024-05-10T04:02:37.966 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:37.966 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":85,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:02:36.063310+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:02:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.965+0000 7f6805f74700 1 -- 172.21.15.150:0/25934268 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f67e80a1120 msgr2=0x7f67e80a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.965+0000 7f6805f74700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f67e80a1120 0x7f67e80a3630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67f000b7b0 tx=0x7f67f001a040 comp rx=0 tx=0).stop 2024-05-10T04:02:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.965+0000 7f6805f74700 1 -- 172.21.15.150:0/25934268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6800101be0 msgr2=0x7f6800197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.965+0000 7f6805f74700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6800101be0 0x7f6800197430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f67f400e9a0 tx=0x7f67f400ecb0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.965+0000 7f6805f74700 1 -- 172.21.15.150:0/25934268 shutdown_connections 2024-05-10T04:02:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.965+0000 7f6805f74700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f67e80a1120 0x7f67e80a3630 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.965+0000 7f6805f74700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6800102580 0x7f680019e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.965+0000 7f6805f74700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6800100ac0 0x7f6800196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.965+0000 7f6805f74700 1 --2- 172.21.15.150:0/25934268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6800101be0 0x7f6800197430 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.965+0000 7f6805f74700 1 -- 172.21.15.150:0/25934268 >> 172.21.15.150:0/25934268 conn(0x7f68000fc0d0 msgr2=0x7f6800104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:37.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.965+0000 7f6805f74700 1 -- 172.21.15.150:0/25934268 shutdown_connections 2024-05-10T04:02:37.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:37.965+0000 7f6805f74700 1 -- 172.21.15.150:0/25934268 wait complete. 2024-05-10T04:02:37.981 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T04:02:31.605102+0000', 'modified': '2024-05-10T04:02:36.063637+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24557}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6839/267491715', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 267491715}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 267491715}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-05-10T04:02:37.981 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T04:02:37.981 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-10T04:02:37.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:37 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/478055092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:37.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:37 smithi181 ceph-mon[46602]: pgmap v229: 129 pgs: 108 active+clean, 1 creating+peering, 20 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2024-05-10T04:02:37.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:37 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/288531607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:37.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:37 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2959649145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:37.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:37 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/478055092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:37.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:37 smithi161 ceph-mon[46918]: pgmap v229: 129 pgs: 108 active+clean, 1 creating+peering, 20 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2024-05-10T04:02:37.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:37 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/288531607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:37.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:37 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2959649145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:02:38.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.114+0000 7f716d337700 1 -- 172.21.15.150:0/1426840167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7168100d70 msgr2=0x7f7168101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:38.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.114+0000 7f716d337700 1 --2- 172.21.15.150:0/1426840167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7168100d70 0x7f7168101180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7158009990 tx=0x7f7158009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.114+0000 7f716d337700 1 -- 172.21.15.150:0/1426840167 shutdown_connections 2024-05-10T04:02:38.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.114+0000 7f716d337700 1 --2- 172.21.15.150:0/1426840167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71681016c0 0x7f7168108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.114+0000 7f716d337700 1 --2- 172.21.15.150:0/1426840167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7168102060 0x7f7168102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.114+0000 7f716d337700 1 --2- 172.21.15.150:0/1426840167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7168100d70 0x7f7168101180 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.114+0000 7f716d337700 1 -- 172.21.15.150:0/1426840167 >> 172.21.15.150:0/1426840167 conn(0x7f71680fb8b0 msgr2=0x7f71680fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:38.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.114+0000 7f716d337700 1 -- 172.21.15.150:0/1426840167 shutdown_connections 2024-05-10T04:02:38.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.114+0000 7f716d337700 1 -- 172.21.15.150:0/1426840167 wait complete. 2024-05-10T04:02:38.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.115+0000 7f716d337700 1 Processor -- start 2024-05-10T04:02:38.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.115+0000 7f716d337700 1 -- start start 2024-05-10T04:02:38.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.115+0000 7f716d337700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7168100d70 0x7f7168197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.115+0000 7f716d337700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71681016c0 0x7f71681984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.115+0000 7f716d337700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7168102060 0x7f716819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.115+0000 7f716d337700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716819f9f0 con 0x7f7168100d70 2024-05-10T04:02:38.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.115+0000 7f716d337700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716819fb60 con 0x7f7168102060 2024-05-10T04:02:38.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.115+0000 7f716d337700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716819fd20 con 0x7f71681016c0 2024-05-10T04:02:38.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f7166ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7168100d70 0x7f7168197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f7166ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7168100d70 0x7f7168197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37102/0 (socket says 172.21.15.150:37102) 2024-05-10T04:02:38.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f7166ffd700 1 -- 172.21.15.150:0/3875573959 learned_addr learned my addr 172.21.15.150:0/3875573959 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:38.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f71677fe700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7168102060 0x7f716819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f71667fc700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71681016c0 0x7f71681984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f7166ffd700 1 -- 172.21.15.150:0/3875573959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71681016c0 msgr2=0x7f71681984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:38.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f7166ffd700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71681016c0 0x7f71681984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f7166ffd700 1 -- 172.21.15.150:0/3875573959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7168102060 msgr2=0x7f716819f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:38.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f7166ffd700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7168102060 0x7f716819f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f7166ffd700 1 -- 172.21.15.150:0/3875573959 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71580095f0 con 0x7f7168100d70 2024-05-10T04:02:38.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f7166ffd700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7168100d70 0x7f7168197f60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f715c00e970 tx=0x7f715c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:38.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f714ffff700 1 -- 172.21.15.150:0/3875573959 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c009790 con 0x7f7168100d70 2024-05-10T04:02:38.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f714ffff700 1 -- 172.21.15.150:0/3875573959 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f715c00eec0 con 0x7f7168100d70 2024-05-10T04:02:38.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f716d337700 1 -- 172.21.15.150:0/3875573959 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716819ffa0 con 0x7f7168100d70 2024-05-10T04:02:38.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f714ffff700 1 -- 172.21.15.150:0/3875573959 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c005590 con 0x7f7168100d70 2024-05-10T04:02:38.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.116+0000 7f716d337700 1 -- 172.21.15.150:0/3875573959 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71681a0450 con 0x7f7168100d70 2024-05-10T04:02:38.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.119+0000 7f714ffff700 1 -- 172.21.15.150:0/3875573959 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f715c00cbd0 con 0x7f7168100d70 2024-05-10T04:02:38.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.119+0000 7f714ffff700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f71500a11c0 0x7f71500a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.119+0000 7f71667fc700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f71500a11c0 0x7f71500a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.119+0000 7f714ffff700 1 -- 172.21.15.150:0/3875573959 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f715c014070 con 0x7f7168100d70 2024-05-10T04:02:38.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.120+0000 7f71667fc700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f71500a11c0 0x7f71500a36d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7158006010 tx=0x7f715801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:38.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.120+0000 7f717d67bb80 1 Processor -- start 2024-05-10T04:02:38.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.120+0000 7f717d67bb80 1 -- start start 2024-05-10T04:02:38.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f717d67bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e7650 0x56335d7e7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f717d67bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 0x56335d7e8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f717d67bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8e70 0x56335d7ef770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f717d67bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d70e100 con 0x56335d7e8520 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f717d67bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d70bac0 con 0x56335d7e7650 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f717d67bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d7073b0 con 0x56335d7e8e70 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f71667fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 0x56335d7e8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f71667fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 0x56335d7e8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37118/0 (socket says 172.21.15.150:37118) 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f7166ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e7650 0x56335d7e7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f71667fc700 1 -- 172.21.15.150:0/3095791191 learned_addr learned my addr 172.21.15.150:0/3095791191 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f71677fe700 1 --2- 172.21.15.150:0/3095791191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8e70 0x56335d7ef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f71667fc700 1 -- 172.21.15.150:0/3095791191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8e70 msgr2=0x56335d7ef770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f71667fc700 1 --2- 172.21.15.150:0/3095791191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8e70 0x56335d7ef770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f71667fc700 1 -- 172.21.15.150:0/3095791191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e7650 msgr2=0x56335d7e7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f71667fc700 1 --2- 172.21.15.150:0/3095791191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e7650 0x56335d7e7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f71667fc700 1 -- 172.21.15.150:0/3095791191 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56335d588360 con 0x56335d7e8520 2024-05-10T04:02:38.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f71667fc700 1 --2- 172.21.15.150:0/3095791191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 0x56335d7e8930 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7158015000 tx=0x7f715801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc44929242d6c847 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:38.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f714d7fa700 1 -- 172.21.15.150:0/3095791191 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715802a070 con 0x56335d7e8520 2024-05-10T04:02:38.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f714d7fa700 1 -- 172.21.15.150:0/3095791191 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f715800fcb0 con 0x56335d7e8520 2024-05-10T04:02:38.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.121+0000 7f714d7fa700 1 -- 172.21.15.150:0/3095791191 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715801d040 con 0x56335d7e8520 2024-05-10T04:02:38.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.122+0000 7f717d67bb80 1 -- 172.21.15.150:0/3095791191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 msgr2=0x56335d7e8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:38.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.122+0000 7f717d67bb80 1 --2- 172.21.15.150:0/3095791191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 0x56335d7e8930 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7158015000 tx=0x7f715801e000 comp rx=0 tx=0).stop 2024-05-10T04:02:38.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.122+0000 7f717d67bb80 1 -- 172.21.15.150:0/3095791191 shutdown_connections 2024-05-10T04:02:38.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.122+0000 7f717d67bb80 1 --2- 172.21.15.150:0/3095791191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8e70 0x56335d7ef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.122+0000 7f717d67bb80 1 --2- 172.21.15.150:0/3095791191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e7650 0x56335d7e7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.122+0000 7f717d67bb80 1 --2- 172.21.15.150:0/3095791191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 0x56335d7e8930 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.122+0000 7f717d67bb80 1 -- 172.21.15.150:0/3095791191 >> 172.21.15.150:0/3095791191 conn(0x56335d749a10 msgr2=0x56335d7e1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:38.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.122+0000 7f717d67bb80 1 -- 172.21.15.150:0/3095791191 shutdown_connections 2024-05-10T04:02:38.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.122+0000 7f717d67bb80 1 -- 172.21.15.150:0/3095791191 wait complete. 2024-05-10T04:02:38.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f717d67bb80 1 -- start start 2024-05-10T04:02:38.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f717d67bb80 1 Processor -- start 2024-05-10T04:02:38.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f717d67bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 0x56335d7f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f717d67bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8520 0x56335d7f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f717d67bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 0x56335d7f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f717d67bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d6a3510 con 0x56335d7e7650 2024-05-10T04:02:38.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f717d67bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d6a31b0 con 0x56335d7e8e70 2024-05-10T04:02:38.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f717d67bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d58a650 con 0x56335d7e8520 2024-05-10T04:02:38.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f717d67bb80 10 client.0 _ll_register_callbacks cb 0x56335d6acae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:02:38.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f7166ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 0x56335d7f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f7166ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 0x56335d7f15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37120/0 (socket says 172.21.15.150:37120) 2024-05-10T04:02:38.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f7166ffd700 1 -- 172.21.15.150:0/3748703592 learned_addr learned my addr 172.21.15.150:0/3748703592 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:38.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f71677fe700 1 --2- 172.21.15.150:0/3748703592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 0x56335d7f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f71667fc700 1 --2- 172.21.15.150:0/3748703592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8520 0x56335d7f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f7166ffd700 1 -- 172.21.15.150:0/3748703592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8520 msgr2=0x56335d7f1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:02:38.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f7166ffd700 1 --2- 172.21.15.150:0/3748703592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8520 0x56335d7f1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f7166ffd700 1 -- 172.21.15.150:0/3748703592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 msgr2=0x56335d7f8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:02:38.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f7166ffd700 1 --2- 172.21.15.150:0/3748703592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 0x56335d7f8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.123+0000 7f7166ffd700 1 -- 172.21.15.150:0/3748703592 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71680ffc60 con 0x56335d7e7650 2024-05-10T04:02:38.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f7166ffd700 1 --2- 172.21.15.150:0/3748703592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 0x56335d7f15f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f715c018000 tx=0x7f715c09a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79e8f3c37242687 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:38.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f714aff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:02:38.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f714aff5700 1 -- 172.21.15.150:0/3748703592 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c099630 con 0x56335d7e7650 2024-05-10T04:02:38.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f714aff5700 1 -- 172.21.15.150:0/3748703592 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f715c0a23c0 con 0x56335d7e7650 2024-05-10T04:02:38.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f717d67bb80 1 -- 172.21.15.150:0/3748703592 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56335d660220 con 0x56335d7e7650 2024-05-10T04:02:38.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f714aff5700 1 -- 172.21.15.150:0/3748703592 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f715c0a1f30 con 0x56335d7e7650 2024-05-10T04:02:38.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f717d67bb80 10 client.14796 fetch_fsmap learned FSMap version 17 2024-05-10T04:02:38.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f717d67bb80 1 -- 172.21.15.150:0/3748703592 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56335d534b10 con 0x56335d7e7650 2024-05-10T04:02:38.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f714aff5700 1 -- 172.21.15.150:0/3748703592 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56335d660220 con 0x56335d7e7650 2024-05-10T04:02:38.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f714aff5700 1 -- 172.21.15.150:0/3748703592 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f715c094c70 con 0x56335d7e7650 2024-05-10T04:02:38.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f717d67bb80 10 client.14796 fetch_fsmap finished waiting for FSMap version 17 2024-05-10T04:02:38.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f717d67bb80 10 client.14796 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T04:02:38.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f717d67bb80 20 client.14796 populate_metadata read hostname 'smithi150' 2024-05-10T04:02:38.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f717d67bb80 1 --2- 172.21.15.150:0/3748703592 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56335d7fcc00 0x56335d801930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f717d67bb80 4 client.14796 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:02:38.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.124+0000 7f717d67bb80 1 -- 172.21.15.150:0/3748703592 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56335d7b6730 con 0x56335d7fcc00 2024-05-10T04:02:38.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.125+0000 7f71667fc700 1 --2- 172.21.15.150:0/3748703592 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56335d7fcc00 0x56335d801930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.125+0000 7f714aff5700 10 client.14796 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:02:38.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.125+0000 7f71667fc700 1 --2- 172.21.15.150:0/3748703592 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56335d7fcc00 0x56335d801930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95a0dd9491e56f8 server_cookie=5d0b888bcc368282 in_seq=0 out_seq=0 2024-05-10T04:02:38.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f714aff5700 1 -- 172.21.15.150:0/3748703592 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56335d7b6730 con 0x56335d7fcc00 2024-05-10T04:02:38.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f714aff5700 10 client.14796 handle_command_reply: tid=0 2024-05-10T04:02:38.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f717d67bb80 1 client.14796 shutdown 2024-05-10T04:02:38.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f717d67bb80 1 -- 172.21.15.150:0/3748703592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 msgr2=0x56335d7f15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:38.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f717d67bb80 1 --2- 172.21.15.150:0/3748703592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 0x56335d7f15f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f715c018000 tx=0x7f715c09a730 comp rx=0 tx=0).stop 2024-05-10T04:02:38.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f717d67bb80 1 -- 172.21.15.150:0/3748703592 shutdown_connections 2024-05-10T04:02:38.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f717d67bb80 1 --2- 172.21.15.150:0/3748703592 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56335d7fcc00 0x56335d801930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f717d67bb80 1 --2- 172.21.15.150:0/3748703592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 0x56335d7f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f717d67bb80 1 --2- 172.21.15.150:0/3748703592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 0x56335d7f15f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f714aff5700 0 client.14796 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:02:38.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f717d67bb80 1 --2- 172.21.15.150:0/3748703592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8520 0x56335d7f1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f717d67bb80 1 -- 172.21.15.150:0/3748703592 >> 172.21.15.150:0/3748703592 conn(0x56335d749a10 msgr2=0x56335d7e1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:38.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f717d67bb80 1 -- 172.21.15.150:0/3748703592 shutdown_connections 2024-05-10T04:02:38.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f717d67bb80 1 -- 172.21.15.150:0/3748703592 wait complete. 2024-05-10T04:02:38.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.126+0000 7f717d67bb80 20 client.14796 trim_cache size 0 max 16384 2024-05-10T04:02:38.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.144+0000 7f717d67bb80 1 Processor -- start 2024-05-10T04:02:38.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.144+0000 7f717d67bb80 1 -- start start 2024-05-10T04:02:38.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.144+0000 7f717d67bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e7650 0x56335d7f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.144+0000 7f717d67bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 0x56335d7de130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.144+0000 7f717d67bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 0x56335d7f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.144+0000 7f717d67bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d7fbdb0 con 0x56335d7e8520 2024-05-10T04:02:38.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.144+0000 7f717d67bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d6c52b0 con 0x56335d7e8e70 2024-05-10T04:02:38.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.144+0000 7f717d67bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d700150 con 0x56335d7e7650 2024-05-10T04:02:38.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.144+0000 7f71667fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 0x56335d7de130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.144+0000 7f71667fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 0x56335d7de130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37124/0 (socket says 172.21.15.150:37124) 2024-05-10T04:02:38.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.144+0000 7f71667fc700 1 -- 172.21.15.150:0/1841563612 learned_addr learned my addr 172.21.15.150:0/1841563612 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:38.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.145+0000 7f7166ffd700 1 --2- 172.21.15.150:0/1841563612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e7650 0x56335d7f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.145+0000 7f71667fc700 1 -- 172.21.15.150:0/1841563612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e7650 msgr2=0x56335d7f0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:02:38.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.145+0000 7f71667fc700 1 --2- 172.21.15.150:0/1841563612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e7650 0x56335d7f0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.145+0000 7f71667fc700 1 -- 172.21.15.150:0/1841563612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 msgr2=0x56335d7f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:02:38.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.145+0000 7f71677fe700 1 --2- 172.21.15.150:0/1841563612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 0x56335d7f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.145+0000 7f71667fc700 1 --2- 172.21.15.150:0/1841563612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 0x56335d7f6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.145+0000 7f71667fc700 1 -- 172.21.15.150:0/1841563612 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56335d6cd510 con 0x56335d7e8520 2024-05-10T04:02:38.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.145+0000 7f71667fc700 1 --2- 172.21.15.150:0/1841563612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 0x56335d7de130 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71580098e0 tx=0x7f7158021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4b3c257787f9a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:38.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.145+0000 7f714dffb700 1 -- 172.21.15.150:0/1841563612 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715802a070 con 0x56335d7e8520 2024-05-10T04:02:38.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.145+0000 7f714dffb700 1 -- 172.21.15.150:0/1841563612 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f715802b040 con 0x56335d7e8520 2024-05-10T04:02:38.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.145+0000 7f714dffb700 1 -- 172.21.15.150:0/1841563612 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715801d050 con 0x56335d7e8520 2024-05-10T04:02:38.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 -- 172.21.15.150:0/1841563612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 msgr2=0x56335d7de130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:38.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 --2- 172.21.15.150:0/1841563612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 0x56335d7de130 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71580098e0 tx=0x7f7158021000 comp rx=0 tx=0).stop 2024-05-10T04:02:38.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 -- 172.21.15.150:0/1841563612 shutdown_connections 2024-05-10T04:02:38.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 --2- 172.21.15.150:0/1841563612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 0x56335d7f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 --2- 172.21.15.150:0/1841563612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e7650 0x56335d7f0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 --2- 172.21.15.150:0/1841563612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e8520 0x56335d7de130 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 -- 172.21.15.150:0/1841563612 >> 172.21.15.150:0/1841563612 conn(0x56335d749a10 msgr2=0x56335d7e1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:38.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 -- 172.21.15.150:0/1841563612 shutdown_connections 2024-05-10T04:02:38.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 -- 172.21.15.150:0/1841563612 wait complete. 2024-05-10T04:02:38.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 -- start start 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 Processor -- start 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 0x56335d7f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8520 0x56335d7f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 0x56335d7f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d7f63b0 con 0x56335d7e7650 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d7f6520 con 0x56335d7e8e70 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d7f66e0 con 0x56335d7e8520 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f717d67bb80 10 client.0 _ll_register_callbacks cb 0x56335d6acae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f7166ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 0x56335d7f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f7166ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 0x56335d7f4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37126/0 (socket says 172.21.15.150:37126) 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.146+0000 7f7166ffd700 1 -- 172.21.15.150:0/1834546381 learned_addr learned my addr 172.21.15.150:0/1834546381 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f71677fe700 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 0x56335d7f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f71667fc700 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8520 0x56335d7f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f7166ffd700 1 -- 172.21.15.150:0/1834546381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8520 msgr2=0x56335d7f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f7166ffd700 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8520 0x56335d7f4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f7166ffd700 1 -- 172.21.15.150:0/1834546381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 msgr2=0x56335d7f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f7166ffd700 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 0x56335d7f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f7166ffd700 1 -- 172.21.15.150:0/1834546381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56335d588360 con 0x56335d7e7650 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f71677fe700 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 0x56335d7f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f7166ffd700 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 0x56335d7f4a50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f715c096440 tx=0x7f715c0984f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b945c01c3f7aa76 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f714aff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f714aff5700 1 -- 172.21.15.150:0/1834546381 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c095dd0 con 0x56335d7e7650 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f714aff5700 1 -- 172.21.15.150:0/1834546381 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f715c05fe40 con 0x56335d7e7650 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f717d67bb80 1 -- 172.21.15.150:0/1834546381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f715800fcb0 con 0x56335d7e7650 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f714aff5700 1 -- 172.21.15.150:0/1834546381 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f715c0a23c0 con 0x56335d7e7650 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f717d67bb80 10 client.14802 fetch_fsmap learned FSMap version 17 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f717d67bb80 1 -- 172.21.15.150:0/1834546381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56335d58cc80 con 0x56335d7e7650 2024-05-10T04:02:38.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f714aff5700 1 -- 172.21.15.150:0/1834546381 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f715c0a6020 con 0x56335d7e7650 2024-05-10T04:02:38.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.147+0000 7f714aff5700 1 -- 172.21.15.150:0/1834546381 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f715c081070 con 0x56335d7e7650 2024-05-10T04:02:38.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f717d67bb80 10 client.14802 fetch_fsmap finished waiting for FSMap version 17 2024-05-10T04:02:38.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f717d67bb80 10 client.14802 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T04:02:38.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f717d67bb80 20 client.14802 populate_metadata read hostname 'smithi150' 2024-05-10T04:02:38.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f717d67bb80 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56335d855490 0x56335d853a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:38.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f717d67bb80 4 client.14802 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:02:38.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f717d67bb80 1 -- 172.21.15.150:0/1834546381 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56335d6ffe70 con 0x56335d855490 2024-05-10T04:02:38.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f71667fc700 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56335d855490 0x56335d853a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:38.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f714aff5700 10 client.14802 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:02:38.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f71667fc700 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56335d855490 0x56335d853a50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7eb541d96153977 server_cookie=25652eb59627cfa in_seq=0 out_seq=0 2024-05-10T04:02:38.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f714aff5700 1 -- 172.21.15.150:0/1834546381 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x56335d6ffe70 con 0x56335d855490 2024-05-10T04:02:38.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f714aff5700 10 client.14802 handle_command_reply: tid=0 2024-05-10T04:02:38.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f717d67bb80 1 client.14802 shutdown 2024-05-10T04:02:38.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f717d67bb80 1 -- 172.21.15.150:0/1834546381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 msgr2=0x56335d7f4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:38.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.148+0000 7f717d67bb80 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 0x56335d7f4a50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f715c096440 tx=0x7f715c0984f0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f717d67bb80 1 -- 172.21.15.150:0/1834546381 shutdown_connections 2024-05-10T04:02:38.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f717d67bb80 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56335d855490 0x56335d853a50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f717d67bb80 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56335d7e8e70 0x56335d7f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f717d67bb80 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56335d7e7650 0x56335d7f4a50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f717d67bb80 1 --2- 172.21.15.150:0/1834546381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56335d7e8520 0x56335d7f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f717d67bb80 1 -- 172.21.15.150:0/1834546381 >> 172.21.15.150:0/1834546381 conn(0x56335d749a10 msgr2=0x56335d7de130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:38.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f714aff5700 0 client.14802 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:02:38.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f717d67bb80 1 -- 172.21.15.150:0/1834546381 shutdown_connections 2024-05-10T04:02:38.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f717d67bb80 1 -- 172.21.15.150:0/1834546381 wait complete. 2024-05-10T04:02:38.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f717d67bb80 20 client.14802 trim_cache size 0 max 16384 2024-05-10T04:02:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:02:38.160 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "325b4c82-0e81-11ef-bc98-c7b262605968", 2024-05-10T04:02:38.160 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 0, 2024-05-10T04:02:38.160 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24557, 2024-05-10T04:02:38.160 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:02:38.160 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:02:38.160 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:02:38.160 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 6.5404669709999999, 2024-05-10T04:02:38.160 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 17, 2024-05-10T04:02:38.160 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 85, 2024-05-10T04:02:38.160 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 84, 2024-05-10T04:02:38.160 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 12.064735426 2024-05-10T04:02:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:02:38.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f716d337700 1 -- 172.21.15.150:0/3875573959 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f71500a11c0 msgr2=0x7f71500a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:38.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f716d337700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f71500a11c0 0x7f71500a36d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7158006010 tx=0x7f715801a040 comp rx=0 tx=0).stop 2024-05-10T04:02:38.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f716d337700 1 -- 172.21.15.150:0/3875573959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7168100d70 msgr2=0x7f7168197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:38.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f716d337700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7168100d70 0x7f7168197f60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f715c00e970 tx=0x7f715c00ec80 comp rx=0 tx=0).stop 2024-05-10T04:02:38.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f716d337700 1 -- 172.21.15.150:0/3875573959 shutdown_connections 2024-05-10T04:02:38.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f716d337700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f71500a11c0 0x7f71500a36d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f716d337700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7168102060 0x7f716819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f716d337700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7168100d70 0x7f7168197f60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f716d337700 1 --2- 172.21.15.150:0/3875573959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71681016c0 0x7f71681984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:38.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f716d337700 1 -- 172.21.15.150:0/3875573959 >> 172.21.15.150:0/3875573959 conn(0x7f71680fb8b0 msgr2=0x7f7168104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:38.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f716d337700 1 -- 172.21.15.150:0/3875573959 shutdown_connections 2024-05-10T04:02:38.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:38.149+0000 7f716d337700 1 -- 172.21.15.150:0/3875573959 wait complete. 2024-05-10T04:02:38.163 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T04:02:38.163 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:02:38.164 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-10T04:02:38.382 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:02:38.855 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:38 smithi150 ceph-mon[42477]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T04:02:38.855 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:38 smithi150 ceph-mon[42477]: Cluster is now healthy 2024-05-10T04:02:38.855 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:38 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/25934268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:38.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:38 smithi181 ceph-mon[46602]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T04:02:38.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:38 smithi181 ceph-mon[46602]: Cluster is now healthy 2024-05-10T04:02:38.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:38 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/25934268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:38.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:38 smithi161 ceph-mon[46918]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T04:02:38.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:38 smithi161 ceph-mon[46918]: Cluster is now healthy 2024-05-10T04:02:38.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:38 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/25934268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:39.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.461+0000 7f47160b3700 1 -- 172.21.15.150:0/2714880904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4710100a60 msgr2=0x7f471010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:39.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.461+0000 7f47160b3700 1 --2- 172.21.15.150:0/2714880904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4710100a60 0x7f471010eaf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f470800b0a0 tx=0x7f470800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:39.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.461+0000 7f47160b3700 1 -- 172.21.15.150:0/2714880904 shutdown_connections 2024-05-10T04:02:39.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.461+0000 7f47160b3700 1 --2- 172.21.15.150:0/2714880904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4710100a60 0x7f471010eaf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:39.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.461+0000 7f47160b3700 1 --2- 172.21.15.150:0/2714880904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4710106c80 0x7f4710107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:39.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.461+0000 7f47160b3700 1 --2- 172.21.15.150:0/2714880904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4710100110 0x7f4710100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:39.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.462+0000 7f47160b3700 1 -- 172.21.15.150:0/2714880904 >> 172.21.15.150:0/2714880904 conn(0x7f47100fbee0 msgr2=0x7f47100fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:39.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.462+0000 7f47160b3700 1 -- 172.21.15.150:0/2714880904 shutdown_connections 2024-05-10T04:02:39.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.462+0000 7f47160b3700 1 -- 172.21.15.150:0/2714880904 wait complete. 2024-05-10T04:02:39.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.462+0000 7f47160b3700 1 Processor -- start 2024-05-10T04:02:39.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.462+0000 7f47160b3700 1 -- start start 2024-05-10T04:02:39.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f47160b3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4710100110 0x7f471019e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:39.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f47160b3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4710100a60 0x7f471019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:39.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f47160b3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4710106c80 0x7f47101984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:39.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f47160b3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4710198ac0 con 0x7f4710100a60 2024-05-10T04:02:39.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f47160b3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4710198c30 con 0x7f4710106c80 2024-05-10T04:02:39.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f47160b3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4710198df0 con 0x7f4710100110 2024-05-10T04:02:39.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f470effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4710100a60 0x7f471019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:39.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f470effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4710100a60 0x7f471019ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37296/0 (socket says 172.21.15.150:37296) 2024-05-10T04:02:39.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f470effd700 1 -- 172.21.15.150:0/3981398696 learned_addr learned my addr 172.21.15.150:0/3981398696 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:39.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f470ffff700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4710106c80 0x7f47101984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:39.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f470f7fe700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4710100110 0x7f471019e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:39.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f470effd700 1 -- 172.21.15.150:0/3981398696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4710100110 msgr2=0x7f471019e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:39.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f470effd700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4710100110 0x7f471019e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:39.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f470effd700 1 -- 172.21.15.150:0/3981398696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4710106c80 msgr2=0x7f47101984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:39.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f470effd700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4710106c80 0x7f47101984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:39.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f470effd700 1 -- 172.21.15.150:0/3981398696 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4708009d00 con 0x7f4710100a60 2024-05-10T04:02:39.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f470effd700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4710100a60 0x7f471019ea80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f470400d980 tx=0x7f470400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:39.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.463+0000 7f470cff9700 1 -- 172.21.15.150:0/3981398696 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704009930 con 0x7f4710100a60 2024-05-10T04:02:39.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.464+0000 7f47160b3700 1 -- 172.21.15.150:0/3981398696 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47101990d0 con 0x7f4710100a60 2024-05-10T04:02:39.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.464+0000 7f470cff9700 1 -- 172.21.15.150:0/3981398696 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4704010460 con 0x7f4710100a60 2024-05-10T04:02:39.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.464+0000 7f470cff9700 1 -- 172.21.15.150:0/3981398696 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470400f620 con 0x7f4710100a60 2024-05-10T04:02:39.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.464+0000 7f47160b3700 1 -- 172.21.15.150:0/3981398696 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47101a5510 con 0x7f4710100a60 2024-05-10T04:02:39.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.465+0000 7f47160b3700 1 -- 172.21.15.150:0/3981398696 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471010a240 con 0x7f4710100a60 2024-05-10T04:02:39.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.466+0000 7f470cff9700 1 -- 172.21.15.150:0/3981398696 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f47040108d0 con 0x7f4710100a60 2024-05-10T04:02:39.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.466+0000 7f470cff9700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f46f80a11a0 0x7f46f80a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:39.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.466+0000 7f470f7fe700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f46f80a11a0 0x7f46f80a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:39.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.466+0000 7f470cff9700 1 -- 172.21.15.150:0/3981398696 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f47040cc5c0 con 0x7f4710100a60 2024-05-10T04:02:39.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.467+0000 7f470f7fe700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f46f80a11a0 0x7f46f80a36b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4700005950 tx=0x7f47000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:39.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.470+0000 7f470cff9700 1 -- 172.21.15.150:0/3981398696 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4704095b00 con 0x7f4710100a60 2024-05-10T04:02:39.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.710+0000 7f47160b3700 1 -- 172.21.15.150:0/3981398696 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4710199bc0 con 0x7f4710100a60 2024-05-10T04:02:39.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.710+0000 7f470cff9700 1 -- 172.21.15.150:0/3981398696 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4704017070 con 0x7f4710100a60 2024-05-10T04:02:39.712 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T04:02:39.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.712+0000 7f47160b3700 1 -- 172.21.15.150:0/3981398696 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f46f80a11a0 msgr2=0x7f46f80a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:39.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.712+0000 7f47160b3700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f46f80a11a0 0x7f46f80a36b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4700005950 tx=0x7f47000058e0 comp rx=0 tx=0).stop 2024-05-10T04:02:39.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.712+0000 7f47160b3700 1 -- 172.21.15.150:0/3981398696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4710100a60 msgr2=0x7f471019ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:39.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.712+0000 7f47160b3700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4710100a60 0x7f471019ea80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f470400d980 tx=0x7f470400dc90 comp rx=0 tx=0).stop 2024-05-10T04:02:39.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.713+0000 7f47160b3700 1 -- 172.21.15.150:0/3981398696 shutdown_connections 2024-05-10T04:02:39.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.713+0000 7f47160b3700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f46f80a11a0 0x7f46f80a36b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:39.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.713+0000 7f47160b3700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4710106c80 0x7f47101984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:39.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.713+0000 7f47160b3700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4710100110 0x7f471019e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:39.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.713+0000 7f47160b3700 1 --2- 172.21.15.150:0/3981398696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4710100a60 0x7f471019ea80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:39.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.713+0000 7f47160b3700 1 -- 172.21.15.150:0/3981398696 >> 172.21.15.150:0/3981398696 conn(0x7f47100fbee0 msgr2=0x7f47101049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:39.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.713+0000 7f47160b3700 1 -- 172.21.15.150:0/3981398696 shutdown_connections 2024-05-10T04:02:39.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:39.713+0000 7f47160b3700 1 -- 172.21.15.150:0/3981398696 wait complete. 2024-05-10T04:02:39.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:39 smithi150 ceph-mon[42477]: pgmap v230: 129 pgs: 125 active+clean, 1 creating+peering, 3 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-05-10T04:02:39.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:39 smithi181 ceph-mon[46602]: pgmap v230: 129 pgs: 125 active+clean, 1 creating+peering, 3 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-05-10T04:02:39.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:39 smithi161 ceph-mon[46918]: pgmap v230: 129 pgs: 125 active+clean, 1 creating+peering, 3 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-05-10T04:02:40.093 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T04:02:40.093 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T04:02:40.093 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-05-10T04:02:40.093 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-10T04:02:40.093 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:02:40.093 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T04:02:40.093 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T04:02:40.093 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T04:02:40.093 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-10T04:02:40.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:40.093 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-05-10T04:02:40.112 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T04:02:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T04:02:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T04:02:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-05-10T04:02:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-05-10T04:02:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-05-10T04:02:40.113 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T04:02:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-05-10T04:02:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-05-10T04:02:40.113 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T04:02:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-05-10T04:02:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-05-10T04:02:40.114 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T04:02:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-05-10T04:02:40.114 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T04:02:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-05-10T04:02:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-10T04:02:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 41890sec preferred_lft 41890sec 2024-05-10T04:02:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::d613:fbd0:302a:c5e5/64 scope link noprefixroute 2024-05-10T04:02:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-05-10T04:02:40.115 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T04:02:40.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:40.115 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:02:40.115 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:02:40.115 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add name ceph-brx type bridge 2024-05-10T04:02:40.115 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr flush dev ceph-brx 2024-05-10T04:02:40.115 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set ceph-brx up 2024-05-10T04:02:40.115 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T04:02:40.115 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:02:40.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:40.279 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T04:02:40.310 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-05-10T04:02:40.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:40.310 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-05-10T04:02:40.379 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-05-10T04:02:40.379 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T04:02:40.379 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T04:02:40.380 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T04:02:40.380 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T04:02:40.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:40.380 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:02:40.380 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:02:40.380 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T04:02:40.380 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T04:02:40.380 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T04:02:40.380 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:02:40.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:40.658 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-05-10T04:02:40.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:40.677 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2024-05-10T04:02:40.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:40.734 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:02:40.734 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:02:40.734 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T04:02:40.734 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-10T04:02:40.734 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:02:40.812 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:40 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3981398696' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T04:02:40.830 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-10T04:02:40.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:40.831 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:02:40.831 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:02:40.831 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-10T04:02:40.831 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T04:02:40.831 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-10T04:02:40.831 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-10T04:02:40.832 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-10T04:02:40.832 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:02:40.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:40 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3981398696' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T04:02:40.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:40 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3981398696' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T04:02:41.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:41.089 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:02:41.089 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:02:41.089 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.0 up 2024-05-10T04:02:41.089 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T04:02:41.090 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:02:41.189 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-10T04:02:41.189 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T04:02:41.189 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:02:41.207 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-10T04:02:41.207 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T04:02:41.207 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:02:41.264 DEBUG:teuthology.orchestra.run.smithi150:> sudo modprobe fuse 2024-05-10T04:02:41.332 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-10T04:02:41.332 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-10T04:02:41.399 INFO:teuthology.orchestra.run.smithi150.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-10T04:02:41.400 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-10T04:02:41.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:41.401 DEBUG:teuthology.orchestra.run.smithi150:> ls /sys/fs/fuse/connections 2024-05-10T04:02:41.461 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-10T04:02:41.461 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-10T04:02:41.502 DEBUG:teuthology.orchestra.run.smithi150:> sudo modprobe fuse 2024-05-10T04:02:41.529 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-10T04:02:41.529 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-10T04:02:41.597 INFO:teuthology.orchestra.run.smithi150.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-10T04:02:41.598 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-10T04:02:41.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:41.599 DEBUG:teuthology.orchestra.run.smithi150:> ls /sys/fs/fuse/connections 2024-05-10T04:02:41.602 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi150.stderr:2024-05-10T04:02:41.599+0000 7fbfb1c1e5c0 -1 init, newargv = 0x7fbf980046a0 newargc=15 2024-05-10T04:02:41.602 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi150.stderr:ceph-fuse[95517]: starting ceph client 2024-05-10T04:02:41.608 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi150.stderr:ceph-fuse[95517]: starting fuse 2024-05-10T04:02:41.620 INFO:teuthology.orchestra.run.smithi150.stdout:124 2024-05-10T04:02:41.620 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> import glob 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> import re 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> import os 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> import subprocess 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> def _find_admin_socket(client_name): 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> files = glob.glob(asok_path) 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> # Given a non-glob path, it better be there 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> if "*" not in asok_path: 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> assert(len(files) == 1) 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> return files[0] 2024-05-10T04:02:41.621 DEBUG:teuthology.orchestra.run.smithi150:> 2024-05-10T04:02:41.622 DEBUG:teuthology.orchestra.run.smithi150:> for f in files: 2024-05-10T04:02:41.622 DEBUG:teuthology.orchestra.run.smithi150:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-10T04:02:41.622 DEBUG:teuthology.orchestra.run.smithi150:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-10T04:02:41.622 DEBUG:teuthology.orchestra.run.smithi150:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-10T04:02:41.622 DEBUG:teuthology.orchestra.run.smithi150:> contents = proc_f.read() 2024-05-10T04:02:41.622 DEBUG:teuthology.orchestra.run.smithi150:> if mountpoint in contents: 2024-05-10T04:02:41.622 DEBUG:teuthology.orchestra.run.smithi150:> return f 2024-05-10T04:02:41.622 DEBUG:teuthology.orchestra.run.smithi150:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-10T04:02:41.622 DEBUG:teuthology.orchestra.run.smithi150:> 2024-05-10T04:02:41.622 DEBUG:teuthology.orchestra.run.smithi150:> print(_find_admin_socket("client.admin")) 2024-05-10T04:02:41.622 DEBUG:teuthology.orchestra.run.smithi150:> ' 2024-05-10T04:02:41.745 INFO:teuthology.orchestra.run.smithi150.stdout:/var/run/ceph/ceph-client.admin.95517.asok 2024-05-10T04:02:41.925 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:41 smithi150 ceph-mon[42477]: pgmap v231: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 317 B/s wr, 0 op/s 2024-05-10T04:02:41.926 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.95517.asok 2024-05-10T04:02:41.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:41.927 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.95517.asok status 2024-05-10T04:02:41.988 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:41 smithi181 ceph-mon[46602]: pgmap v231: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 317 B/s wr, 0 op/s 2024-05-10T04:02:41.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:41 smithi161 ceph-mon[46918]: pgmap v231: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 317 B/s wr, 0 op/s 2024-05-10T04:02:42.095 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:02:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "metadata": { 2024-05-10T04:02:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_sha1": "50174015a7e79f951b988949abe236791f52d4da", 2024-05-10T04:02:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable)", 2024-05-10T04:02:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "entity_id": "admin", 2024-05-10T04:02:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "hostname": "smithi150", 2024-05-10T04:02:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-10T04:02:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "pid": "95517", 2024-05-10T04:02:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "root": "/" 2024-05-10T04:02:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:02:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "dentry_count": 0, 2024-05-10T04:02:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "dentry_pinned_count": 0, 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 14814, 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "inst": { 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "name": { 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "client", 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "num": 14814 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": { 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "192.168.0.1:0", 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1235614402 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": { 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-05-10T04:02:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "192.168.0.1:0", 2024-05-10T04:02:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1235614402 2024-05-10T04:02:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:02:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "inst_str": "client.14814 192.168.0.1:0/1235614402", 2024-05-10T04:02:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "addr_str": "192.168.0.1:0/1235614402", 2024-05-10T04:02:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "inode_count": 1, 2024-05-10T04:02:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_epoch": 17, 2024-05-10T04:02:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_epoch": 85, 2024-05-10T04:02:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_epoch_barrier": 0, 2024-05-10T04:02:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "blocklisted": false, 2024-05-10T04:02:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs" 2024-05-10T04:02:42.097 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:02:42.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:42.105 DEBUG:teuthology.orchestra.run.smithi150:> stat --file-system '--printf=%T 2024-05-10T04:02:42.105 DEBUG:teuthology.orchestra.run.smithi150:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:02:42.123 INFO:teuthology.orchestra.run.smithi150.stdout:fuseblk 2024-05-10T04:02:42.123 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:02:42.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:42.123 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:02:42.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:42.191 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-10T04:02:42.294 INFO:teuthology.orchestra.run.smithi150.stdout:# file: . 2024-05-10T04:02:42.294 INFO:teuthology.orchestra.run.smithi150.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-10T04:02:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:42.298 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:02:42.298 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T04:02:42.316 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:02:42.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.487+0000 7f11fc104700 1 -- 172.21.15.150:0/786512787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f41020a0 msgr2=0x7f11f41024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:42.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.487+0000 7f11fc104700 1 --2- 172.21.15.150:0/786512787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f41020a0 0x7f11f41024b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009990 tx=0x7f11f0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.487+0000 7f11fc104700 1 -- 172.21.15.150:0/786512787 shutdown_connections 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.487+0000 7f11fc104700 1 --2- 172.21.15.150:0/786512787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f4101700 0x7f11f4109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.487+0000 7f11fc104700 1 --2- 172.21.15.150:0/786512787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f41020a0 0x7f11f41024b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.487+0000 7f11fc104700 1 --2- 172.21.15.150:0/786512787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f4100db0 0x7f11f41011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.487+0000 7f11fc104700 1 -- 172.21.15.150:0/786512787 >> 172.21.15.150:0/786512787 conn(0x7f11f40fb8f0 msgr2=0x7f11f40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.487+0000 7f11fc104700 1 -- 172.21.15.150:0/786512787 shutdown_connections 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.487+0000 7f11fc104700 1 -- 172.21.15.150:0/786512787 wait complete. 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.487+0000 7f11fc104700 1 Processor -- start 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.487+0000 7f11fc104700 1 -- start start 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11fc104700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f4100db0 0x7f11f4196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11fc104700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f4101700 0x7f11f41973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11fc104700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f41020a0 0x7f11f419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11fc104700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f419e940 con 0x7f11f4100db0 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11fc104700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f419eab0 con 0x7f11f4101700 2024-05-10T04:02:42.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11fc104700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f419ec70 con 0x7f11f41020a0 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11f9ea0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f4100db0 0x7f11f4196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11f9ea0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f4100db0 0x7f11f4196e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37328/0 (socket says 172.21.15.150:37328) 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11f9ea0700 1 -- 172.21.15.150:0/591836447 learned_addr learned my addr 172.21.15.150:0/591836447 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11fa6a1700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f41020a0 0x7f11f419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11f969f700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f4101700 0x7f11f41973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11f9ea0700 1 -- 172.21.15.150:0/591836447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f41020a0 msgr2=0x7f11f419e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11f9ea0700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f41020a0 0x7f11f419e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11f9ea0700 1 -- 172.21.15.150:0/591836447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f4101700 msgr2=0x7f11f41973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11f9ea0700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f4101700 0x7f11f41973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11f9ea0700 1 -- 172.21.15.150:0/591836447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f00095f0 con 0x7f11f4100db0 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11f9ea0700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f4100db0 0x7f11f4196e60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f11f00098e0 tx=0x7f11f0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11eaffd700 1 -- 172.21.15.150:0/591836447 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f001d070 con 0x7f11f4100db0 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11eaffd700 1 -- 172.21.15.150:0/591836447 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11f0021410 con 0x7f11f4100db0 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11fc104700 1 -- 172.21.15.150:0/591836447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f419eef0 con 0x7f11f4100db0 2024-05-10T04:02:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.488+0000 7f11eaffd700 1 -- 172.21.15.150:0/591836447 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0017650 con 0x7f11f4100db0 2024-05-10T04:02:42.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.489+0000 7f11fc104700 1 -- 172.21.15.150:0/591836447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f419f260 con 0x7f11f4100db0 2024-05-10T04:02:42.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.490+0000 7f11fc104700 1 -- 172.21.15.150:0/591836447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f4107eb0 con 0x7f11f4100db0 2024-05-10T04:02:42.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.492+0000 7f11eaffd700 1 -- 172.21.15.150:0/591836447 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f11f0021580 con 0x7f11f4100db0 2024-05-10T04:02:42.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.492+0000 7f11eaffd700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f11e00a1120 0x7f11e00a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:42.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.492+0000 7f11f969f700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f11e00a1120 0x7f11e00a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:42.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.492+0000 7f11eaffd700 1 -- 172.21.15.150:0/591836447 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f11f00ce480 con 0x7f11f4100db0 2024-05-10T04:02:42.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.493+0000 7f11f969f700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f11e00a1120 0x7f11e00a3630 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f11f4197f00 tx=0x7f11e400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:42.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.496+0000 7f11eaffd700 1 -- 172.21.15.150:0/591836447 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f11f0097ab0 con 0x7f11f4100db0 2024-05-10T04:02:42.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.696+0000 7f11fc104700 1 -- 172.21.15.150:0/591836447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11f410c290 con 0x7f11f4100db0 2024-05-10T04:02:42.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.697+0000 7f11eaffd700 1 -- 172.21.15.150:0/591836447 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24027 (secure 0 0 0) 0x7f11f009b0d0 con 0x7f11f4100db0 2024-05-10T04:02:42.700 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:42.700 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":85,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:02:36.063310+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:02:42.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.699+0000 7f11fc104700 1 -- 172.21.15.150:0/591836447 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f11e00a1120 msgr2=0x7f11e00a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:42.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.699+0000 7f11fc104700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f11e00a1120 0x7f11e00a3630 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f11f4197f00 tx=0x7f11e400b410 comp rx=0 tx=0).stop 2024-05-10T04:02:42.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.699+0000 7f11fc104700 1 -- 172.21.15.150:0/591836447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f4100db0 msgr2=0x7f11f4196e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:42.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.699+0000 7f11fc104700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f4100db0 0x7f11f4196e60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f11f00098e0 tx=0x7f11f0009580 comp rx=0 tx=0).stop 2024-05-10T04:02:42.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.700+0000 7f11fc104700 1 -- 172.21.15.150:0/591836447 shutdown_connections 2024-05-10T04:02:42.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.700+0000 7f11fc104700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f11e00a1120 0x7f11e00a3630 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.700+0000 7f11fc104700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f41020a0 0x7f11f419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.700+0000 7f11fc104700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f4100db0 0x7f11f4196e60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.700+0000 7f11fc104700 1 --2- 172.21.15.150:0/591836447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f4101700 0x7f11f41973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.700+0000 7f11fc104700 1 -- 172.21.15.150:0/591836447 >> 172.21.15.150:0/591836447 conn(0x7f11f40fb8f0 msgr2=0x7f11f4104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:42.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.700+0000 7f11fc104700 1 -- 172.21.15.150:0/591836447 shutdown_connections 2024-05-10T04:02:42.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.700+0000 7f11fc104700 1 -- 172.21.15.150:0/591836447 wait complete. 2024-05-10T04:02:42.715 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:02:30.076694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:02:30.760710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:02:33.850714+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:02:42.715 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:02:42.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.850+0000 7f69870fd700 1 -- 172.21.15.150:0/68953866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6980101c00 msgr2=0x7f6980102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:42.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.850+0000 7f69870fd700 1 --2- 172.21.15.150:0/68953866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6980101c00 0x7f6980102010 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6974009990 tx=0x7f6974009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.850+0000 7f69870fd700 1 -- 172.21.15.150:0/68953866 shutdown_connections 2024-05-10T04:02:42.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.850+0000 7f69870fd700 1 --2- 172.21.15.150:0/68953866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69801036c0 0x7f6980109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.850+0000 7f69870fd700 1 --2- 172.21.15.150:0/68953866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6980101c00 0x7f6980102010 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.850+0000 7f69870fd700 1 --2- 172.21.15.150:0/68953866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6980102d20 0x7f6980103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.850+0000 7f69870fd700 1 -- 172.21.15.150:0/68953866 >> 172.21.15.150:0/68953866 conn(0x7f69800fd210 msgr2=0x7f69800ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:42.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.850+0000 7f69870fd700 1 -- 172.21.15.150:0/68953866 shutdown_connections 2024-05-10T04:02:42.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.850+0000 7f69870fd700 1 -- 172.21.15.150:0/68953866 wait complete. 2024-05-10T04:02:42.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.851+0000 7f69870fd700 1 Processor -- start 2024-05-10T04:02:42.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.851+0000 7f69870fd700 1 -- start start 2024-05-10T04:02:42.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.851+0000 7f69870fd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6980101c00 0x7f6980197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:42.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.851+0000 7f69870fd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6980102d20 0x7f69801984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:42.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.851+0000 7f69870fd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69801036c0 0x7f698019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:42.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.851+0000 7f69870fd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698019fa50 con 0x7f6980102d20 2024-05-10T04:02:42.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.851+0000 7f69870fd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698019fbc0 con 0x7f6980101c00 2024-05-10T04:02:42.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.851+0000 7f69870fd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698019fd80 con 0x7f69801036c0 2024-05-10T04:02:42.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.851+0000 7f697ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6980102d20 0x7f69801984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:42.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.851+0000 7f697ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6980102d20 0x7f69801984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37334/0 (socket says 172.21.15.150:37334) 2024-05-10T04:02:42.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.851+0000 7f697ffff700 1 -- 172.21.15.150:0/3868570073 learned_addr learned my addr 172.21.15.150:0/3868570073 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:42.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.851+0000 7f698569a700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69801036c0 0x7f698019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:42.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.851+0000 7f6984e99700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6980101c00 0x7f6980197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:42.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.852+0000 7f697ffff700 1 -- 172.21.15.150:0/3868570073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69801036c0 msgr2=0x7f698019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:42.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.852+0000 7f697ffff700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69801036c0 0x7f698019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.852+0000 7f697ffff700 1 -- 172.21.15.150:0/3868570073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6980101c00 msgr2=0x7f6980197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:42.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.852+0000 7f697ffff700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6980101c00 0x7f6980197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:42.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.852+0000 7f697ffff700 1 -- 172.21.15.150:0/3868570073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69740095f0 con 0x7f6980102d20 2024-05-10T04:02:42.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.852+0000 7f6984e99700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6980101c00 0x7f6980197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:42.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.852+0000 7f697ffff700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6980102d20 0x7f69801984b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f696c00d950 tx=0x7f696c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:42.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.852+0000 7f697dffb700 1 -- 172.21.15.150:0/3868570073 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c009940 con 0x7f6980102d20 2024-05-10T04:02:42.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.852+0000 7f697dffb700 1 -- 172.21.15.150:0/3868570073 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f696c010460 con 0x7f6980102d20 2024-05-10T04:02:42.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.852+0000 7f69870fd700 1 -- 172.21.15.150:0/3868570073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69801a0060 con 0x7f6980102d20 2024-05-10T04:02:42.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.852+0000 7f697dffb700 1 -- 172.21.15.150:0/3868570073 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c00f620 con 0x7f6980102d20 2024-05-10T04:02:42.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.852+0000 7f69870fd700 1 -- 172.21.15.150:0/3868570073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69801a0570 con 0x7f6980102d20 2024-05-10T04:02:42.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.853+0000 7f69870fd700 1 -- 172.21.15.150:0/3868570073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6980108e10 con 0x7f6980102d20 2024-05-10T04:02:42.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.854+0000 7f697dffb700 1 -- 172.21.15.150:0/3868570073 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f696c0108d0 con 0x7f6980102d20 2024-05-10T04:02:42.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.855+0000 7f697dffb700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69700a11c0 0x7f69700a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:42.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.855+0000 7f697dffb700 1 -- 172.21.15.150:0/3868570073 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f696c0cc970 con 0x7f6980102d20 2024-05-10T04:02:42.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.855+0000 7f6984e99700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69700a11c0 0x7f69700a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:42.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.855+0000 7f6984e99700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69700a11c0 0x7f69700a36d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6974006010 tx=0x7f697401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:42.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:42.860+0000 7f697dffb700 1 -- 172.21.15.150:0/3868570073 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f696c095f10 con 0x7f6980102d20 2024-05-10T04:02:43.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.056+0000 7f69870fd700 1 -- 172.21.15.150:0/3868570073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6980068730 con 0x7f6980102d20 2024-05-10T04:02:43.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.056+0000 7f697dffb700 1 -- 172.21.15.150:0/3868570073 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f696c0160a0 con 0x7f6980102d20 2024-05-10T04:02:43.059 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:02:43.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.058+0000 7f69870fd700 1 -- 172.21.15.150:0/3868570073 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69700a11c0 msgr2=0x7f69700a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.058+0000 7f69870fd700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69700a11c0 0x7f69700a36d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6974006010 tx=0x7f697401a040 comp rx=0 tx=0).stop 2024-05-10T04:02:43.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.058+0000 7f69870fd700 1 -- 172.21.15.150:0/3868570073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6980102d20 msgr2=0x7f69801984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.058+0000 7f69870fd700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6980102d20 0x7f69801984b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f696c00d950 tx=0x7f696c00dc60 comp rx=0 tx=0).stop 2024-05-10T04:02:43.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.059+0000 7f69870fd700 1 -- 172.21.15.150:0/3868570073 shutdown_connections 2024-05-10T04:02:43.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.059+0000 7f69870fd700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69700a11c0 0x7f69700a36d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.059+0000 7f69870fd700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69801036c0 0x7f698019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.059+0000 7f69870fd700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6980101c00 0x7f6980197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.059+0000 7f69870fd700 1 --2- 172.21.15.150:0/3868570073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6980102d20 0x7f69801984b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.059+0000 7f69870fd700 1 -- 172.21.15.150:0/3868570073 >> 172.21.15.150:0/3868570073 conn(0x7f69800fd210 msgr2=0x7f6980105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:43.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.059+0000 7f69870fd700 1 -- 172.21.15.150:0/3868570073 shutdown_connections 2024-05-10T04:02:43.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.059+0000 7f69870fd700 1 -- 172.21.15.150:0/3868570073 wait complete. 2024-05-10T04:02:43.072 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:42 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/591836447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:43.074 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:02:43.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.207+0000 7f38e8885700 1 -- 172.21.15.150:0/2766266118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e4107080 msgr2=0x7f38e41094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.207+0000 7f38e8885700 1 --2- 172.21.15.150:0/2766266118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e4107080 0x7f38e41094a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009990 tx=0x7f38d8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.207+0000 7f38e8885700 1 -- 172.21.15.150:0/2766266118 shutdown_connections 2024-05-10T04:02:43.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.207+0000 7f38e8885700 1 --2- 172.21.15.150:0/2766266118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e4109ac0 0x7f38e410bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.207+0000 7f38e8885700 1 --2- 172.21.15.150:0/2766266118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38e4069070 0x7f38e4106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.207+0000 7f38e8885700 1 --2- 172.21.15.150:0/2766266118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e4107080 0x7f38e41094a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.207+0000 7f38e8885700 1 -- 172.21.15.150:0/2766266118 >> 172.21.15.150:0/2766266118 conn(0x7f38e40fc0d0 msgr2=0x7f38e40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:43.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.207+0000 7f38e8885700 1 -- 172.21.15.150:0/2766266118 shutdown_connections 2024-05-10T04:02:43.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.207+0000 7f38e8885700 1 -- 172.21.15.150:0/2766266118 wait complete. 2024-05-10T04:02:43.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.208+0000 7f38e8885700 1 Processor -- start 2024-05-10T04:02:43.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.208+0000 7f38e8885700 1 -- start start 2024-05-10T04:02:43.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.208+0000 7f38e8885700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38e4069070 0x7f38e419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:43.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.208+0000 7f38e8885700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e4107080 0x7f38e419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:43.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.208+0000 7f38e8885700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e4109ac0 0x7f38e41a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:43.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.208+0000 7f38e8885700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41a1b90 con 0x7f38e4109ac0 2024-05-10T04:02:43.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.208+0000 7f38e8885700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41a1d00 con 0x7f38e4069070 2024-05-10T04:02:43.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.208+0000 7f38e8885700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41a1ec0 con 0x7f38e4107080 2024-05-10T04:02:43.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e2d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e4109ac0 0x7f38e41a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:43.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e2d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e4109ac0 0x7f38e41a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37340/0 (socket says 172.21.15.150:37340) 2024-05-10T04:02:43.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e2d9d700 1 -- 172.21.15.150:0/3988699970 learned_addr learned my addr 172.21.15.150:0/3988699970 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:43.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e259c700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38e4069070 0x7f38e419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:43.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e1d9b700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e4107080 0x7f38e419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:43.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e2d9d700 1 -- 172.21.15.150:0/3988699970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e4107080 msgr2=0x7f38e419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e2d9d700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e4107080 0x7f38e419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e2d9d700 1 -- 172.21.15.150:0/3988699970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38e4069070 msgr2=0x7f38e419a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e2d9d700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38e4069070 0x7f38e419a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e2d9d700 1 -- 172.21.15.150:0/3988699970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d80095f0 con 0x7f38e4109ac0 2024-05-10T04:02:43.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e1d9b700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e4107080 0x7f38e419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:43.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e2d9d700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e4109ac0 0x7f38e41a1580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00ef30 tx=0x7f38dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:43.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38d37fe700 1 -- 172.21.15.150:0/3988699970 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc018070 con 0x7f38e4109ac0 2024-05-10T04:02:43.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38d37fe700 1 -- 172.21.15.150:0/3988699970 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38dc00f040 con 0x7f38e4109ac0 2024-05-10T04:02:43.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e8885700 1 -- 172.21.15.150:0/3988699970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e41a2140 con 0x7f38e4109ac0 2024-05-10T04:02:43.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38d37fe700 1 -- 172.21.15.150:0/3988699970 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00cdb0 con 0x7f38e4109ac0 2024-05-10T04:02:43.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.209+0000 7f38e8885700 1 -- 172.21.15.150:0/3988699970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e41a2600 con 0x7f38e4109ac0 2024-05-10T04:02:43.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.210+0000 7f38e8885700 1 -- 172.21.15.150:0/3988699970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e40fdc80 con 0x7f38e4109ac0 2024-05-10T04:02:43.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.211+0000 7f38d37fe700 1 -- 172.21.15.150:0/3988699970 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f38dc003a00 con 0x7f38e4109ac0 2024-05-10T04:02:43.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.212+0000 7f38d37fe700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38cc0a11c0 0x7f38cc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:43.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.212+0000 7f38e259c700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38cc0a11c0 0x7f38cc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:43.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.212+0000 7f38d37fe700 1 -- 172.21.15.150:0/3988699970 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f38dc0cc660 con 0x7f38e4109ac0 2024-05-10T04:02:43.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.213+0000 7f38e259c700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38cc0a11c0 0x7f38cc0a36d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005950 tx=0x7f38d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:43.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.217+0000 7f38d37fe700 1 -- 172.21.15.150:0/3988699970 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f38dc095c90 con 0x7f38e4109ac0 2024-05-10T04:02:43.237 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:42 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/591836447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:43.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:42 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/591836447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:43.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.413+0000 7f38e8885700 1 -- 172.21.15.150:0/3988699970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38e410f5b0 con 0x7f38e4109ac0 2024-05-10T04:02:43.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.413+0000 7f38d37fe700 1 -- 172.21.15.150:0/3988699970 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38dc0992b0 con 0x7f38e4109ac0 2024-05-10T04:02:43.415 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:02:43.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.415+0000 7f38e8885700 1 -- 172.21.15.150:0/3988699970 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38cc0a11c0 msgr2=0x7f38cc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.415+0000 7f38e8885700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38cc0a11c0 0x7f38cc0a36d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005950 tx=0x7f38d400b410 comp rx=0 tx=0).stop 2024-05-10T04:02:43.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.415+0000 7f38e8885700 1 -- 172.21.15.150:0/3988699970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e4109ac0 msgr2=0x7f38e41a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.415+0000 7f38e8885700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e4109ac0 0x7f38e41a1580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00ef30 tx=0x7f38dc00c3a0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.415+0000 7f38e8885700 1 -- 172.21.15.150:0/3988699970 shutdown_connections 2024-05-10T04:02:43.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.415+0000 7f38e8885700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38cc0a11c0 0x7f38cc0a36d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.415+0000 7f38e8885700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e4109ac0 0x7f38e41a1580 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.415+0000 7f38e8885700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38e4069070 0x7f38e419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.415+0000 7f38e8885700 1 --2- 172.21.15.150:0/3988699970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e4107080 0x7f38e419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.415+0000 7f38e8885700 1 -- 172.21.15.150:0/3988699970 >> 172.21.15.150:0/3988699970 conn(0x7f38e40fc0d0 msgr2=0x7f38e4107fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:43.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.416+0000 7f38e8885700 1 -- 172.21.15.150:0/3988699970 shutdown_connections 2024-05-10T04:02:43.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.416+0000 7f38e8885700 1 -- 172.21.15.150:0/3988699970 wait complete. 2024-05-10T04:02:43.431 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:02:43.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.565+0000 7f5774827700 1 -- 172.21.15.150:0/760163948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5770100ac0 msgr2=0x7f5770100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.565+0000 7f5774827700 1 --2- 172.21.15.150:0/760163948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5770100ac0 0x7f5770100ed0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5760009990 tx=0x7f5760009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.565+0000 7f5774827700 1 -- 172.21.15.150:0/760163948 shutdown_connections 2024-05-10T04:02:43.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.565+0000 7f5774827700 1 --2- 172.21.15.150:0/760163948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5770102580 0x7f5770108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.565+0000 7f5774827700 1 --2- 172.21.15.150:0/760163948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5770100ac0 0x7f5770100ed0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.565+0000 7f5774827700 1 --2- 172.21.15.150:0/760163948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770101be0 0x7f5770101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.565+0000 7f5774827700 1 -- 172.21.15.150:0/760163948 >> 172.21.15.150:0/760163948 conn(0x7f57700fc0d0 msgr2=0x7f57700fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:43.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.565+0000 7f5774827700 1 -- 172.21.15.150:0/760163948 shutdown_connections 2024-05-10T04:02:43.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.565+0000 7f5774827700 1 -- 172.21.15.150:0/760163948 wait complete. 2024-05-10T04:02:43.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f5774827700 1 Processor -- start 2024-05-10T04:02:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f5774827700 1 -- start start 2024-05-10T04:02:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f5774827700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100ac0 0x7f5770068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f5774827700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5770101be0 0x7f577019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f5774827700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5770102580 0x7f577019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f5774827700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577019f9a0 con 0x7f5770101be0 2024-05-10T04:02:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f5774827700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577019fb10 con 0x7f5770102580 2024-05-10T04:02:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f5774827700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577019fcd0 con 0x7f5770100ac0 2024-05-10T04:02:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f576dd9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5770101be0 0x7f577019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f576dd9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5770101be0 0x7f577019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37356/0 (socket says 172.21.15.150:37356) 2024-05-10T04:02:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f576dd9b700 1 -- 172.21.15.150:0/4064009020 learned_addr learned my addr 172.21.15.150:0/4064009020 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f576e59c700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100ac0 0x7f5770068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f576ed9d700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5770102580 0x7f577019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f576dd9b700 1 -- 172.21.15.150:0/4064009020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100ac0 msgr2=0x7f5770068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f576dd9b700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100ac0 0x7f5770068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f576dd9b700 1 -- 172.21.15.150:0/4064009020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5770102580 msgr2=0x7f577019f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f576dd9b700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5770102580 0x7f577019f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.566+0000 7f576dd9b700 1 -- 172.21.15.150:0/4064009020 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600095f0 con 0x7f5770101be0 2024-05-10T04:02:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.567+0000 7f576dd9b700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5770101be0 0x7f577019cc90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f576400d980 tx=0x7f576400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.567+0000 7f575f7fe700 1 -- 172.21.15.150:0/4064009020 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764004220 con 0x7f5770101be0 2024-05-10T04:02:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.567+0000 7f575f7fe700 1 -- 172.21.15.150:0/4064009020 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5764007640 con 0x7f5770101be0 2024-05-10T04:02:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.567+0000 7f5774827700 1 -- 172.21.15.150:0/4064009020 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577019ffb0 con 0x7f5770101be0 2024-05-10T04:02:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.567+0000 7f575f7fe700 1 -- 172.21.15.150:0/4064009020 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764005690 con 0x7f5770101be0 2024-05-10T04:02:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.567+0000 7f5774827700 1 -- 172.21.15.150:0/4064009020 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57701a03d0 con 0x7f5770101be0 2024-05-10T04:02:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.568+0000 7f5774827700 1 -- 172.21.15.150:0/4064009020 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5770107cd0 con 0x7f5770101be0 2024-05-10T04:02:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.569+0000 7f575f7fe700 1 -- 172.21.15.150:0/4064009020 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5764004750 con 0x7f5770101be0 2024-05-10T04:02:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.570+0000 7f575f7fe700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f57580a11c0 0x7f57580a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.570+0000 7f575f7fe700 1 -- 172.21.15.150:0/4064009020 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f57640cc930 con 0x7f5770101be0 2024-05-10T04:02:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.570+0000 7f576e59c700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f57580a11c0 0x7f57580a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.570+0000 7f576e59c700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f57580a11c0 0x7f57580a36d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5760006010 tx=0x7f576001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:43.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.574+0000 7f575f7fe700 1 -- 172.21.15.150:0/4064009020 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5764095e70 con 0x7f5770101be0 2024-05-10T04:02:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.776+0000 7f5774827700 1 -- 172.21.15.150:0/4064009020 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f577002cf80 con 0x7f5770101be0 2024-05-10T04:02:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.777+0000 7f575f7fe700 1 -- 172.21.15.150:0/4064009020 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5764016020 con 0x7f5770101be0 2024-05-10T04:02:43.779 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:02:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.779+0000 7f5774827700 1 -- 172.21.15.150:0/4064009020 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f57580a11c0 msgr2=0x7f57580a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.779+0000 7f5774827700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f57580a11c0 0x7f57580a36d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5760006010 tx=0x7f576001a040 comp rx=0 tx=0).stop 2024-05-10T04:02:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.779+0000 7f5774827700 1 -- 172.21.15.150:0/4064009020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5770101be0 msgr2=0x7f577019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.779+0000 7f5774827700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5770101be0 0x7f577019cc90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f576400d980 tx=0x7f576400dc90 comp rx=0 tx=0).stop 2024-05-10T04:02:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.779+0000 7f5774827700 1 -- 172.21.15.150:0/4064009020 shutdown_connections 2024-05-10T04:02:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.779+0000 7f5774827700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f57580a11c0 0x7f57580a36d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.779+0000 7f5774827700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5770102580 0x7f577019f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.779+0000 7f5774827700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770100ac0 0x7f5770068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.779+0000 7f5774827700 1 --2- 172.21.15.150:0/4064009020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5770101be0 0x7f577019cc90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.779+0000 7f5774827700 1 -- 172.21.15.150:0/4064009020 >> 172.21.15.150:0/4064009020 conn(0x7f57700fc0d0 msgr2=0x7f5770104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.779+0000 7f5774827700 1 -- 172.21.15.150:0/4064009020 shutdown_connections 2024-05-10T04:02:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.779+0000 7f5774827700 1 -- 172.21.15.150:0/4064009020 wait complete. 2024-05-10T04:02:43.794 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:02:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.927+0000 7ff4a85ea700 1 -- 172.21.15.150:0/3375078963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a0101a40 msgr2=0x7ff4a0101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.927+0000 7ff4a85ea700 1 --2- 172.21.15.150:0/3375078963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a0101a40 0x7ff4a0101f10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009990 tx=0x7ff49c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.927+0000 7ff4a85ea700 1 -- 172.21.15.150:0/3375078963 shutdown_connections 2024-05-10T04:02:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.927+0000 7ff4a85ea700 1 --2- 172.21.15.150:0/3375078963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4a01025a0 0x7ff4a0109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.927+0000 7ff4a85ea700 1 --2- 172.21.15.150:0/3375078963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4a0101060 0x7ff4a0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.927+0000 7ff4a85ea700 1 --2- 172.21.15.150:0/3375078963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a0101a40 0x7ff4a0101f10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.927+0000 7ff4a85ea700 1 -- 172.21.15.150:0/3375078963 >> 172.21.15.150:0/3375078963 conn(0x7ff4a00fb880 msgr2=0x7ff4a00fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.927+0000 7ff4a85ea700 1 -- 172.21.15.150:0/3375078963 shutdown_connections 2024-05-10T04:02:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.927+0000 7ff4a85ea700 1 -- 172.21.15.150:0/3375078963 wait complete. 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a85ea700 1 Processor -- start 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a85ea700 1 -- start start 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a85ea700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4a0101060 0x7ff4a0104a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a85ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4a0101a40 0x7ff4a0103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a85ea700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a01025a0 0x7ff4a0103650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a85ea700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a0103d30 con 0x7ff4a01025a0 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a85ea700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a0103ea0 con 0x7ff4a0101060 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a85ea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a01a1fc0 con 0x7ff4a0101a40 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a6b87700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a01025a0 0x7ff4a0103650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a6b87700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a01025a0 0x7ff4a0103650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37374/0 (socket says 172.21.15.150:37374) 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a6b87700 1 -- 172.21.15.150:0/2633783581 learned_addr learned my addr 172.21.15.150:0/2633783581 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a5b85700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4a0101a40 0x7ff4a0103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a6386700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4a0101060 0x7ff4a0104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a6b87700 1 -- 172.21.15.150:0/2633783581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4a0101a40 msgr2=0x7ff4a0103110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a6b87700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4a0101a40 0x7ff4a0103110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a6b87700 1 -- 172.21.15.150:0/2633783581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4a0101060 msgr2=0x7ff4a0104a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a6b87700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4a0101060 0x7ff4a0104a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a6b87700 1 -- 172.21.15.150:0/2633783581 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c0095f0 con 0x7ff4a01025a0 2024-05-10T04:02:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.928+0000 7ff4a6386700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4a0101060 0x7ff4a0104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.929+0000 7ff4a6b87700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a01025a0 0x7ff4a0103650 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff498007ee0 tx=0x7ff49800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.929+0000 7ff4977fe700 1 -- 172.21.15.150:0/2633783581 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49800ddd0 con 0x7ff4a01025a0 2024-05-10T04:02:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.929+0000 7ff4977fe700 1 -- 172.21.15.150:0/2633783581 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49800f040 con 0x7ff4a01025a0 2024-05-10T04:02:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.929+0000 7ff4977fe700 1 -- 172.21.15.150:0/2633783581 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4980048e0 con 0x7ff4a01025a0 2024-05-10T04:02:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.929+0000 7ff4a85ea700 1 -- 172.21.15.150:0/2633783581 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a01a2220 con 0x7ff4a01025a0 2024-05-10T04:02:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.929+0000 7ff4a85ea700 1 -- 172.21.15.150:0/2633783581 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a01a26b0 con 0x7ff4a01025a0 2024-05-10T04:02:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.930+0000 7ff4a85ea700 1 -- 172.21.15.150:0/2633783581 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a0068730 con 0x7ff4a01025a0 2024-05-10T04:02:43.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.932+0000 7ff4977fe700 1 -- 172.21.15.150:0/2633783581 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff498003a00 con 0x7ff4a01025a0 2024-05-10T04:02:43.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.932+0000 7ff4977fe700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff48c0a11c0 0x7ff48c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:43.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.933+0000 7ff4977fe700 1 -- 172.21.15.150:0/2633783581 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff4980ccbb0 con 0x7ff4a01025a0 2024-05-10T04:02:43.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.933+0000 7ff4a6386700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff48c0a11c0 0x7ff48c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:43.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.933+0000 7ff4a6386700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff48c0a11c0 0x7ff48c0a36d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0069af0 tx=0x7ff490009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:43.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:43.936+0000 7ff4977fe700 1 -- 172.21.15.150:0/2633783581 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff4980960f0 con 0x7ff4a01025a0 2024-05-10T04:02:44.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.132+0000 7ff4a85ea700 1 -- 172.21.15.150:0/2633783581 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff4a01a29d0 con 0x7ff4a01025a0 2024-05-10T04:02:44.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.133+0000 7ff4977fe700 1 -- 172.21.15.150:0/2633783581 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff4980230a0 con 0x7ff4a01025a0 2024-05-10T04:02:44.135 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:02:44.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.135+0000 7ff4a85ea700 1 -- 172.21.15.150:0/2633783581 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff48c0a11c0 msgr2=0x7ff48c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:44.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.135+0000 7ff4a85ea700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff48c0a11c0 0x7ff48c0a36d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0069af0 tx=0x7ff490009260 comp rx=0 tx=0).stop 2024-05-10T04:02:44.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.135+0000 7ff4a85ea700 1 -- 172.21.15.150:0/2633783581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a01025a0 msgr2=0x7ff4a0103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:44.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.135+0000 7ff4a85ea700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a01025a0 0x7ff4a0103650 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff498007ee0 tx=0x7ff49800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.135+0000 7ff4a85ea700 1 -- 172.21.15.150:0/2633783581 shutdown_connections 2024-05-10T04:02:44.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.135+0000 7ff4a85ea700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff48c0a11c0 0x7ff48c0a36d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.135+0000 7ff4a85ea700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a01025a0 0x7ff4a0103650 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.135+0000 7ff4a85ea700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4a0101060 0x7ff4a0104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.135+0000 7ff4a85ea700 1 --2- 172.21.15.150:0/2633783581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4a0101a40 0x7ff4a0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.135+0000 7ff4a85ea700 1 -- 172.21.15.150:0/2633783581 >> 172.21.15.150:0/2633783581 conn(0x7ff4a00fb880 msgr2=0x7ff4a00fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:44.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.135+0000 7ff4a85ea700 1 -- 172.21.15.150:0/2633783581 shutdown_connections 2024-05-10T04:02:44.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.135+0000 7ff4a85ea700 1 -- 172.21.15.150:0/2633783581 wait complete. 2024-05-10T04:02:44.149 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:43 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3868570073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:44.149 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:43 smithi150 ceph-mon[42477]: pgmap v232: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-10T04:02:44.149 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:43 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3988699970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:02:44.150 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T04:02:44.237 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:43 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3868570073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:44.237 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:43 smithi181 ceph-mon[46602]: pgmap v232: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-10T04:02:44.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:43 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3988699970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:02:44.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:43 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3868570073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:44.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:43 smithi161 ceph-mon[46918]: pgmap v232: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-10T04:02:44.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:43 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3988699970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:02:44.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.280+0000 7f83e1fb9700 1 -- 172.21.15.150:0/143893726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83dc102580 msgr2=0x7f83dc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:44.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.280+0000 7f83e1fb9700 1 --2- 172.21.15.150:0/143893726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83dc102580 0x7f83dc108e30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f83d400b0a0 tx=0x7f83d400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.280+0000 7f83e1fb9700 1 -- 172.21.15.150:0/143893726 shutdown_connections 2024-05-10T04:02:44.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.280+0000 7f83e1fb9700 1 --2- 172.21.15.150:0/143893726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83dc102580 0x7f83dc108e30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.280+0000 7f83e1fb9700 1 --2- 172.21.15.150:0/143893726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83dc100ac0 0x7f83dc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.280+0000 7f83e1fb9700 1 --2- 172.21.15.150:0/143893726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83dc101be0 0x7f83dc101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.280+0000 7f83e1fb9700 1 -- 172.21.15.150:0/143893726 >> 172.21.15.150:0/143893726 conn(0x7f83dc0fc0d0 msgr2=0x7f83dc0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:44.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.280+0000 7f83e1fb9700 1 -- 172.21.15.150:0/143893726 shutdown_connections 2024-05-10T04:02:44.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.280+0000 7f83e1fb9700 1 -- 172.21.15.150:0/143893726 wait complete. 2024-05-10T04:02:44.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.281+0000 7f83e1fb9700 1 Processor -- start 2024-05-10T04:02:44.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.281+0000 7f83e1fb9700 1 -- start start 2024-05-10T04:02:44.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.281+0000 7f83e1fb9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83dc100ac0 0x7f83dc068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:44.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.281+0000 7f83e1fb9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83dc101be0 0x7f83dc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:44.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.281+0000 7f83e1fb9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83dc102580 0x7f83dc19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:44.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.281+0000 7f83e1fb9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc19fa70 con 0x7f83dc101be0 2024-05-10T04:02:44.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.281+0000 7f83e1fb9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc19fbe0 con 0x7f83dc102580 2024-05-10T04:02:44.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.281+0000 7f83e1fb9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc19fda0 con 0x7f83dc100ac0 2024-05-10T04:02:44.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.281+0000 7f83daffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83dc101be0 0x7f83dc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:44.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.281+0000 7f83daffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83dc101be0 0x7f83dc19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37386/0 (socket says 172.21.15.150:37386) 2024-05-10T04:02:44.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.281+0000 7f83daffd700 1 -- 172.21.15.150:0/3618216149 learned_addr learned my addr 172.21.15.150:0/3618216149 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.282+0000 7f83dbfff700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83dc102580 0x7f83dc19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.282+0000 7f83db7fe700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83dc100ac0 0x7f83dc068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.282+0000 7f83daffd700 1 -- 172.21.15.150:0/3618216149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83dc100ac0 msgr2=0x7f83dc068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.282+0000 7f83daffd700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83dc100ac0 0x7f83dc068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.282+0000 7f83daffd700 1 -- 172.21.15.150:0/3618216149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83dc102580 msgr2=0x7f83dc19f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.282+0000 7f83daffd700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83dc102580 0x7f83dc19f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.282+0000 7f83daffd700 1 -- 172.21.15.150:0/3618216149 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d4009d00 con 0x7f83dc101be0 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.282+0000 7f83daffd700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83dc101be0 0x7f83dc19cd60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00c920 tx=0x7f83cc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.282+0000 7f83d8ff9700 1 -- 172.21.15.150:0/3618216149 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc004a30 con 0x7f83dc101be0 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.282+0000 7f83d8ff9700 1 -- 172.21.15.150:0/3618216149 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83cc0054e0 con 0x7f83dc101be0 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.282+0000 7f83d8ff9700 1 -- 172.21.15.150:0/3618216149 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc007bd0 con 0x7f83dc101be0 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.282+0000 7f83e1fb9700 1 -- 172.21.15.150:0/3618216149 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83dc1a0080 con 0x7f83dc101be0 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.282+0000 7f83e1fb9700 1 -- 172.21.15.150:0/3618216149 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83dc1a0540 con 0x7f83dc101be0 2024-05-10T04:02:44.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.283+0000 7f83e1fb9700 1 -- 172.21.15.150:0/3618216149 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83dc107cd0 con 0x7f83dc101be0 2024-05-10T04:02:44.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.284+0000 7f83d8ff9700 1 -- 172.21.15.150:0/3618216149 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f83cc007d50 con 0x7f83dc101be0 2024-05-10T04:02:44.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.285+0000 7f83d8ff9700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f83c40a11c0 0x7f83c40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:44.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.285+0000 7f83db7fe700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f83c40a11c0 0x7f83c40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:44.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.285+0000 7f83d8ff9700 1 -- 172.21.15.150:0/3618216149 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f83cc0cca10 con 0x7f83dc101be0 2024-05-10T04:02:44.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.286+0000 7f83db7fe700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f83c40a11c0 0x7f83c40a36d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f83dc101ae0 tx=0x7f83d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:44.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.289+0000 7f83d8ff9700 1 -- 172.21.15.150:0/3618216149 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f83cc095fb0 con 0x7f83dc101be0 2024-05-10T04:02:44.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.503+0000 7f83e1fb9700 1 -- 172.21.15.150:0/3618216149 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f83dc10c510 con 0x7f83dc101be0 2024-05-10T04:02:44.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.503+0000 7f83d8ff9700 1 -- 172.21.15.150:0/3618216149 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) v1 ==== 81+0+597 (secure 0 0 0) 0x7f83cc01d020 con 0x7f83dc101be0 2024-05-10T04:02:44.506 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3457770606 2024-05-11T03:57:29.013448+0000 2024-05-10T04:02:44.506 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6801/999775795 2024-05-11T03:57:29.013448+0000 2024-05-10T04:02:44.507 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/977507934 2024-05-11T03:57:08.747290+0000 2024-05-10T04:02:44.507 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6800/3392737437 2024-05-11T03:57:08.747290+0000 2024-05-10T04:02:44.507 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3909466387 2024-05-11T03:57:29.013448+0000 2024-05-10T04:02:44.507 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/2393499570 2024-05-11T03:57:29.013448+0000 2024-05-10T04:02:44.507 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6800/999775795 2024-05-11T03:57:29.013448+0000 2024-05-10T04:02:44.507 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6801/3392737437 2024-05-11T03:57:08.747290+0000 2024-05-10T04:02:44.508 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/679926778 2024-05-11T03:57:08.747290+0000 2024-05-10T04:02:44.508 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/431676611 2024-05-11T03:57:08.747290+0000 2024-05-10T04:02:44.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.506+0000 7f83e1fb9700 1 -- 172.21.15.150:0/3618216149 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f83c40a11c0 msgr2=0x7f83c40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:44.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.506+0000 7f83e1fb9700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f83c40a11c0 0x7f83c40a36d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f83dc101ae0 tx=0x7f83d000b410 comp rx=0 tx=0).stop 2024-05-10T04:02:44.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.506+0000 7f83e1fb9700 1 -- 172.21.15.150:0/3618216149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83dc101be0 msgr2=0x7f83dc19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:44.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.506+0000 7f83e1fb9700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83dc101be0 0x7f83dc19cd60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00c920 tx=0x7f83cc00cc30 comp rx=0 tx=0).stop 2024-05-10T04:02:44.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.506+0000 7f83e1fb9700 1 -- 172.21.15.150:0/3618216149 shutdown_connections 2024-05-10T04:02:44.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.506+0000 7f83e1fb9700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f83c40a11c0 0x7f83c40a36d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.506+0000 7f83e1fb9700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83dc102580 0x7f83dc19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.506+0000 7f83e1fb9700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83dc100ac0 0x7f83dc068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.506+0000 7f83e1fb9700 1 --2- 172.21.15.150:0/3618216149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83dc101be0 0x7f83dc19cd60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.506+0000 7f83e1fb9700 1 -- 172.21.15.150:0/3618216149 >> 172.21.15.150:0/3618216149 conn(0x7f83dc0fc0d0 msgr2=0x7f83dc104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:44.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.506+0000 7f83e1fb9700 1 -- 172.21.15.150:0/3618216149 shutdown_connections 2024-05-10T04:02:44.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.506+0000 7f83e1fb9700 1 -- 172.21.15.150:0/3618216149 wait complete. 2024-05-10T04:02:44.511 INFO:teuthology.orchestra.run.smithi150.stderr:listed 10 entries 2024-05-10T04:02:44.521 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:02:44.521 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T04:02:44.544 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:02:44.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.713+0000 7f77deac8700 1 -- 172.21.15.150:0/2755475558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8100ac0 msgr2=0x7f77d8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:44.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.713+0000 7f77deac8700 1 --2- 172.21.15.150:0/2755475558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8100ac0 0x7f77d8100ed0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009990 tx=0x7f77cc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.713+0000 7f77deac8700 1 -- 172.21.15.150:0/2755475558 shutdown_connections 2024-05-10T04:02:44.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.713+0000 7f77deac8700 1 --2- 172.21.15.150:0/2755475558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d8102580 0x7f77d8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.713+0000 7f77deac8700 1 --2- 172.21.15.150:0/2755475558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8100ac0 0x7f77d8100ed0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.713+0000 7f77deac8700 1 --2- 172.21.15.150:0/2755475558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77d8101be0 0x7f77d8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.713+0000 7f77deac8700 1 -- 172.21.15.150:0/2755475558 >> 172.21.15.150:0/2755475558 conn(0x7f77d80fc0d0 msgr2=0x7f77d80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:44.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.713+0000 7f77deac8700 1 -- 172.21.15.150:0/2755475558 shutdown_connections 2024-05-10T04:02:44.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.713+0000 7f77deac8700 1 -- 172.21.15.150:0/2755475558 wait complete. 2024-05-10T04:02:44.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.714+0000 7f77deac8700 1 Processor -- start 2024-05-10T04:02:44.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.714+0000 7f77deac8700 1 -- start start 2024-05-10T04:02:44.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.714+0000 7f77deac8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77d8100ac0 0x7f77d8196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:44.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.714+0000 7f77deac8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8101be0 0x7f77d8197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:44.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.714+0000 7f77deac8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d8102580 0x7f77d819e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:44.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.714+0000 7f77deac8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d819e9c0 con 0x7f77d8101be0 2024-05-10T04:02:44.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.714+0000 7f77deac8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d819eb30 con 0x7f77d8102580 2024-05-10T04:02:44.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.714+0000 7f77deac8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d819ecf0 con 0x7f77d8100ac0 2024-05-10T04:02:44.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.714+0000 7f77d7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8101be0 0x7f77d8197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:44.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.714+0000 7f77d7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8101be0 0x7f77d8197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37402/0 (socket says 172.21.15.150:37402) 2024-05-10T04:02:44.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.714+0000 7f77d7fff700 1 -- 172.21.15.150:0/1103574061 learned_addr learned my addr 172.21.15.150:0/1103574061 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:44.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.714+0000 7f77dc864700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77d8100ac0 0x7f77d8196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:44.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.714+0000 7f77dd065700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d8102580 0x7f77d819e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:44.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.715+0000 7f77d7fff700 1 -- 172.21.15.150:0/1103574061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77d8100ac0 msgr2=0x7f77d8196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:44.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.715+0000 7f77d7fff700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77d8100ac0 0x7f77d8196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.715+0000 7f77d7fff700 1 -- 172.21.15.150:0/1103574061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d8102580 msgr2=0x7f77d819e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:44.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.715+0000 7f77d7fff700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d8102580 0x7f77d819e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.715+0000 7f77d7fff700 1 -- 172.21.15.150:0/1103574061 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc0095f0 con 0x7f77d8101be0 2024-05-10T04:02:44.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.715+0000 7f77d7fff700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8101be0 0x7f77d8197420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f77c400e9a0 tx=0x7f77c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:44.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.715+0000 7f77d5ffb700 1 -- 172.21.15.150:0/1103574061 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4009790 con 0x7f77d8101be0 2024-05-10T04:02:44.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.715+0000 7f77d5ffb700 1 -- 172.21.15.150:0/1103574061 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77c4010430 con 0x7f77d8101be0 2024-05-10T04:02:44.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.715+0000 7f77deac8700 1 -- 172.21.15.150:0/1103574061 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d819efd0 con 0x7f77d8101be0 2024-05-10T04:02:44.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.715+0000 7f77d5ffb700 1 -- 172.21.15.150:0/1103574061 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4018680 con 0x7f77d8101be0 2024-05-10T04:02:44.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.715+0000 7f77deac8700 1 -- 172.21.15.150:0/1103574061 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d819f3f0 con 0x7f77d8101be0 2024-05-10T04:02:44.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.716+0000 7f77deac8700 1 -- 172.21.15.150:0/1103574061 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d8107cd0 con 0x7f77d8101be0 2024-05-10T04:02:44.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.717+0000 7f77d5ffb700 1 -- 172.21.15.150:0/1103574061 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f77c40105a0 con 0x7f77d8101be0 2024-05-10T04:02:44.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.718+0000 7f77d5ffb700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f77c80a1120 0x7f77c80a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:44.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.718+0000 7f77d5ffb700 1 -- 172.21.15.150:0/1103574061 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f77c4014070 con 0x7f77d8101be0 2024-05-10T04:02:44.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.718+0000 7f77dc864700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f77c80a1120 0x7f77c80a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:44.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.718+0000 7f77dc864700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f77c80a1120 0x7f77c80a3630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f77cc006010 tx=0x7f77cc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:44.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.721+0000 7f77d5ffb700 1 -- 172.21.15.150:0/1103574061 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f77c4096460 con 0x7f77d8101be0 2024-05-10T04:02:44.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.921+0000 7f77deac8700 1 -- 172.21.15.150:0/1103574061 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77d810c2a0 con 0x7f77d8101be0 2024-05-10T04:02:44.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.922+0000 7f77d5ffb700 1 -- 172.21.15.150:0/1103574061 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24027 (secure 0 0 0) 0x7f77c4026090 con 0x7f77d8101be0 2024-05-10T04:02:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:44.925 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":85,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:02:36.063310+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:02:44.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.925+0000 7f77deac8700 1 -- 172.21.15.150:0/1103574061 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f77c80a1120 msgr2=0x7f77c80a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:44.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.925+0000 7f77deac8700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f77c80a1120 0x7f77c80a3630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f77cc006010 tx=0x7f77cc01a040 comp rx=0 tx=0).stop 2024-05-10T04:02:44.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.925+0000 7f77deac8700 1 -- 172.21.15.150:0/1103574061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8101be0 msgr2=0x7f77d8197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:44.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.925+0000 7f77deac8700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8101be0 0x7f77d8197420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f77c400e9a0 tx=0x7f77c400ecb0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.925+0000 7f77deac8700 1 -- 172.21.15.150:0/1103574061 shutdown_connections 2024-05-10T04:02:44.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.925+0000 7f77deac8700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f77c80a1120 0x7f77c80a3630 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.925+0000 7f77deac8700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d8102580 0x7f77d819e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.925+0000 7f77deac8700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77d8100ac0 0x7f77d8196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.925+0000 7f77deac8700 1 --2- 172.21.15.150:0/1103574061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8101be0 0x7f77d8197420 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:44.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.925+0000 7f77deac8700 1 -- 172.21.15.150:0/1103574061 >> 172.21.15.150:0/1103574061 conn(0x7f77d80fc0d0 msgr2=0x7f77d8104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:44.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.925+0000 7f77deac8700 1 -- 172.21.15.150:0/1103574061 shutdown_connections 2024-05-10T04:02:44.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:44.925+0000 7f77deac8700 1 -- 172.21.15.150:0/1103574061 wait complete. 2024-05-10T04:02:44.939 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:44 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4064009020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:02:44.939 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:44 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2633783581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:02:44.939 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:44 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3618216149' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:02:44.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:02:30.076694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:02:30.760710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:02:33.850714+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:02:44.941 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:02:45.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.082+0000 7f60f3120700 1 -- 172.21.15.150:0/1378359391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ec100570 msgr2=0x7f60ec10a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:45.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.082+0000 7f60f3120700 1 --2- 172.21.15.150:0/1378359391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ec100570 0x7f60ec10a190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f60e400b0a0 tx=0x7f60e400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.082+0000 7f60f3120700 1 -- 172.21.15.150:0/1378359391 shutdown_connections 2024-05-10T04:02:45.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.082+0000 7f60f3120700 1 --2- 172.21.15.150:0/1378359391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ec100570 0x7f60ec10a190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.082+0000 7f60f3120700 1 --2- 172.21.15.150:0/1378359391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ec1031e0 0x7f60ec1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.082+0000 7f60f3120700 1 --2- 172.21.15.150:0/1378359391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60ec0ffc20 0x7f60ec100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.082+0000 7f60f3120700 1 -- 172.21.15.150:0/1378359391 >> 172.21.15.150:0/1378359391 conn(0x7f60ec0fb8f0 msgr2=0x7f60ec0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:45.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.082+0000 7f60f3120700 1 -- 172.21.15.150:0/1378359391 shutdown_connections 2024-05-10T04:02:45.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.082+0000 7f60f3120700 1 -- 172.21.15.150:0/1378359391 wait complete. 2024-05-10T04:02:45.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60f3120700 1 Processor -- start 2024-05-10T04:02:45.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60f3120700 1 -- start start 2024-05-10T04:02:45.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60f3120700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60ec0ffc20 0x7f60ec197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:45.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60f3120700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ec100570 0x7f60ec198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:45.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60f3120700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ec1031e0 0x7f60ec19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:45.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60f3120700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec19fad0 con 0x7f60ec100570 2024-05-10T04:02:45.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60f3120700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec19fc40 con 0x7f60ec1031e0 2024-05-10T04:02:45.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60f3120700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec19fe00 con 0x7f60ec0ffc20 2024-05-10T04:02:45.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60ebfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ec100570 0x7f60ec198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:45.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60ebfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ec100570 0x7f60ec198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37434/0 (socket says 172.21.15.150:37434) 2024-05-10T04:02:45.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60ebfff700 1 -- 172.21.15.150:0/436954626 learned_addr learned my addr 172.21.15.150:0/436954626 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:45.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60f16bd700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ec1031e0 0x7f60ec19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:45.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60f0ebc700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60ec0ffc20 0x7f60ec197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:45.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60ebfff700 1 -- 172.21.15.150:0/436954626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60ec0ffc20 msgr2=0x7f60ec197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:45.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60ebfff700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60ec0ffc20 0x7f60ec197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60ebfff700 1 -- 172.21.15.150:0/436954626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ec1031e0 msgr2=0x7f60ec19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:45.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60ebfff700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ec1031e0 0x7f60ec19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60ebfff700 1 -- 172.21.15.150:0/436954626 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e4009d00 con 0x7f60ec100570 2024-05-10T04:02:45.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.083+0000 7f60ebfff700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ec100570 0x7f60ec198530 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f60d800d950 tx=0x7f60d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:45.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.084+0000 7f60e9ffb700 1 -- 172.21.15.150:0/436954626 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8009940 con 0x7f60ec100570 2024-05-10T04:02:45.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.084+0000 7f60e9ffb700 1 -- 172.21.15.150:0/436954626 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60d8010460 con 0x7f60ec100570 2024-05-10T04:02:45.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.084+0000 7f60f3120700 1 -- 172.21.15.150:0/436954626 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ec1a00e0 con 0x7f60ec100570 2024-05-10T04:02:45.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.084+0000 7f60e9ffb700 1 -- 172.21.15.150:0/436954626 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d800f620 con 0x7f60ec100570 2024-05-10T04:02:45.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.084+0000 7f60f3120700 1 -- 172.21.15.150:0/436954626 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ec1a05f0 con 0x7f60ec100570 2024-05-10T04:02:45.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.085+0000 7f60f3120700 1 -- 172.21.15.150:0/436954626 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ec109030 con 0x7f60ec100570 2024-05-10T04:02:45.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.086+0000 7f60e9ffb700 1 -- 172.21.15.150:0/436954626 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f60d80108d0 con 0x7f60ec100570 2024-05-10T04:02:45.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.086+0000 7f60e9ffb700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f60dc0a11c0 0x7f60dc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:45.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.087+0000 7f60f0ebc700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f60dc0a11c0 0x7f60dc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:45.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.087+0000 7f60e9ffb700 1 -- 172.21.15.150:0/436954626 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f60d80cc9d0 con 0x7f60ec100570 2024-05-10T04:02:45.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.087+0000 7f60f0ebc700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f60dc0a11c0 0x7f60dc0a36d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60ec068120 tx=0x7f60e000d4b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:45.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.090+0000 7f60e9ffb700 1 -- 172.21.15.150:0/436954626 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f60d8095f10 con 0x7f60ec100570 2024-05-10T04:02:45.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:44 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4064009020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:02:45.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:44 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2633783581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:02:45.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:44 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3618216149' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:02:45.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:44 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4064009020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:02:45.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:44 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2633783581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:02:45.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:44 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3618216149' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:02:45.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.289+0000 7f60f3120700 1 -- 172.21.15.150:0/436954626 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60ec10d3d0 con 0x7f60ec100570 2024-05-10T04:02:45.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.290+0000 7f60e9ffb700 1 -- 172.21.15.150:0/436954626 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f60d8020020 con 0x7f60ec100570 2024-05-10T04:02:45.293 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:02:45.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.292+0000 7f60f3120700 1 -- 172.21.15.150:0/436954626 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f60dc0a11c0 msgr2=0x7f60dc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:45.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.292+0000 7f60f3120700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f60dc0a11c0 0x7f60dc0a36d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60ec068120 tx=0x7f60e000d4b0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.292+0000 7f60f3120700 1 -- 172.21.15.150:0/436954626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ec100570 msgr2=0x7f60ec198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:45.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.292+0000 7f60f3120700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ec100570 0x7f60ec198530 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f60d800d950 tx=0x7f60d800dc60 comp rx=0 tx=0).stop 2024-05-10T04:02:45.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.292+0000 7f60f3120700 1 -- 172.21.15.150:0/436954626 shutdown_connections 2024-05-10T04:02:45.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.292+0000 7f60f3120700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f60dc0a11c0 0x7f60dc0a36d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.292+0000 7f60f3120700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ec1031e0 0x7f60ec19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.292+0000 7f60f3120700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60ec0ffc20 0x7f60ec197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.292+0000 7f60f3120700 1 --2- 172.21.15.150:0/436954626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ec100570 0x7f60ec198530 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.292+0000 7f60f3120700 1 -- 172.21.15.150:0/436954626 >> 172.21.15.150:0/436954626 conn(0x7f60ec0fb8f0 msgr2=0x7f60ec106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:45.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.292+0000 7f60f3120700 1 -- 172.21.15.150:0/436954626 shutdown_connections 2024-05-10T04:02:45.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.292+0000 7f60f3120700 1 -- 172.21.15.150:0/436954626 wait complete. 2024-05-10T04:02:45.308 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:02:45.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.445+0000 7f11aee1b700 1 -- 172.21.15.150:0/3676723978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8101c00 msgr2=0x7f11a8102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:45.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.445+0000 7f11aee1b700 1 --2- 172.21.15.150:0/3676723978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8101c00 0x7f11a8102010 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1194009990 tx=0x7f1194009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.445+0000 7f11aee1b700 1 -- 172.21.15.150:0/3676723978 shutdown_connections 2024-05-10T04:02:45.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.445+0000 7f11aee1b700 1 --2- 172.21.15.150:0/3676723978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a81025a0 0x7f11a8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.445+0000 7f11aee1b700 1 --2- 172.21.15.150:0/3676723978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11a8100ae0 0x7f11a8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.445+0000 7f11aee1b700 1 --2- 172.21.15.150:0/3676723978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8101c00 0x7f11a8102010 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.445+0000 7f11aee1b700 1 -- 172.21.15.150:0/3676723978 >> 172.21.15.150:0/3676723978 conn(0x7f11a80fc0f0 msgr2=0x7f11a80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:45.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.446+0000 7f11aee1b700 1 -- 172.21.15.150:0/3676723978 shutdown_connections 2024-05-10T04:02:45.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.446+0000 7f11aee1b700 1 -- 172.21.15.150:0/3676723978 wait complete. 2024-05-10T04:02:45.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.446+0000 7f11aee1b700 1 Processor -- start 2024-05-10T04:02:45.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.446+0000 7f11aee1b700 1 -- start start 2024-05-10T04:02:45.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.446+0000 7f11aee1b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8100ae0 0x7f11a8196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:45.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.446+0000 7f11aee1b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8101c00 0x7f11a8197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:45.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.446+0000 7f11aee1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11a81025a0 0x7f11a819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:45.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.446+0000 7f11aee1b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a819e8d0 con 0x7f11a8101c00 2024-05-10T04:02:45.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.446+0000 7f11aee1b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a819ea40 con 0x7f11a8100ae0 2024-05-10T04:02:45.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.446+0000 7f11aee1b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a819ec00 con 0x7f11a81025a0 2024-05-10T04:02:45.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11a7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8101c00 0x7f11a8197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:45.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11a7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8101c00 0x7f11a8197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37460/0 (socket says 172.21.15.150:37460) 2024-05-10T04:02:45.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11a7fff700 1 -- 172.21.15.150:0/4287393071 learned_addr learned my addr 172.21.15.150:0/4287393071 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:45.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11ad3b8700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11a81025a0 0x7f11a819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:45.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11acbb7700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8100ae0 0x7f11a8196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:45.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11a7fff700 1 -- 172.21.15.150:0/4287393071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11a81025a0 msgr2=0x7f11a819e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:45.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11a7fff700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11a81025a0 0x7f11a819e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11a7fff700 1 -- 172.21.15.150:0/4287393071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8100ae0 msgr2=0x7f11a8196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:45.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11a7fff700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8100ae0 0x7f11a8196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11a7fff700 1 -- 172.21.15.150:0/4287393071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11940095f0 con 0x7f11a8101c00 2024-05-10T04:02:45.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11a7fff700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8101c00 0x7f11a8197330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f11940098e0 tx=0x7f1194009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:45.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11a5ffb700 1 -- 172.21.15.150:0/4287393071 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119401d070 con 0x7f11a8101c00 2024-05-10T04:02:45.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11a5ffb700 1 -- 172.21.15.150:0/4287393071 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1194021410 con 0x7f11a8101c00 2024-05-10T04:02:45.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11a5ffb700 1 -- 172.21.15.150:0/4287393071 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194017650 con 0x7f11a8101c00 2024-05-10T04:02:45.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11aee1b700 1 -- 172.21.15.150:0/4287393071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a819ee80 con 0x7f11a8101c00 2024-05-10T04:02:45.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.447+0000 7f11aee1b700 1 -- 172.21.15.150:0/4287393071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a819f1f0 con 0x7f11a8101c00 2024-05-10T04:02:45.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.448+0000 7f11aee1b700 1 -- 172.21.15.150:0/4287393071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a8107cf0 con 0x7f11a8101c00 2024-05-10T04:02:45.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.450+0000 7f11a5ffb700 1 -- 172.21.15.150:0/4287393071 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1194021580 con 0x7f11a8101c00 2024-05-10T04:02:45.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.450+0000 7f11a5ffb700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f11980a11c0 0x7f11980a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:45.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.450+0000 7f11a5ffb700 1 -- 172.21.15.150:0/4287393071 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f11940ce540 con 0x7f11a8101c00 2024-05-10T04:02:45.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.450+0000 7f11acbb7700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f11980a11c0 0x7f11980a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:45.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.451+0000 7f11acbb7700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f11980a11c0 0x7f11980a36d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f11a8101b00 tx=0x7f119c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:45.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.454+0000 7f11a5ffb700 1 -- 172.21.15.150:0/4287393071 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1194097b70 con 0x7f11a8101c00 2024-05-10T04:02:45.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.651+0000 7f11aee1b700 1 -- 172.21.15.150:0/4287393071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11a810c2e0 con 0x7f11a8101c00 2024-05-10T04:02:45.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.651+0000 7f11a5ffb700 1 -- 172.21.15.150:0/4287393071 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f119409b190 con 0x7f11a8101c00 2024-05-10T04:02:45.654 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:02:45.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.654+0000 7f11aee1b700 1 -- 172.21.15.150:0/4287393071 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f11980a11c0 msgr2=0x7f11980a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:45.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.654+0000 7f11aee1b700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f11980a11c0 0x7f11980a36d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f11a8101b00 tx=0x7f119c009260 comp rx=0 tx=0).stop 2024-05-10T04:02:45.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.654+0000 7f11aee1b700 1 -- 172.21.15.150:0/4287393071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8101c00 msgr2=0x7f11a8197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:45.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.654+0000 7f11aee1b700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8101c00 0x7f11a8197330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f11940098e0 tx=0x7f1194009580 comp rx=0 tx=0).stop 2024-05-10T04:02:45.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.654+0000 7f11aee1b700 1 -- 172.21.15.150:0/4287393071 shutdown_connections 2024-05-10T04:02:45.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.654+0000 7f11aee1b700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f11980a11c0 0x7f11980a36d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.654+0000 7f11aee1b700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11a81025a0 0x7f11a819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.654+0000 7f11aee1b700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8100ae0 0x7f11a8196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.654+0000 7f11aee1b700 1 --2- 172.21.15.150:0/4287393071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11a8101c00 0x7f11a8197330 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.654+0000 7f11aee1b700 1 -- 172.21.15.150:0/4287393071 >> 172.21.15.150:0/4287393071 conn(0x7f11a80fc0f0 msgr2=0x7f11a8104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:45.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.654+0000 7f11aee1b700 1 -- 172.21.15.150:0/4287393071 shutdown_connections 2024-05-10T04:02:45.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.654+0000 7f11aee1b700 1 -- 172.21.15.150:0/4287393071 wait complete. 2024-05-10T04:02:45.669 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:02:45.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.806+0000 7f9bf99c7700 1 -- 172.21.15.150:0/3053403647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4100ac0 msgr2=0x7f9bf4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:45.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.806+0000 7f9bf99c7700 1 --2- 172.21.15.150:0/3053403647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4100ac0 0x7f9bf4100ed0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9be400b0a0 tx=0x7f9be400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.806+0000 7f9bf99c7700 1 -- 172.21.15.150:0/3053403647 shutdown_connections 2024-05-10T04:02:45.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.806+0000 7f9bf99c7700 1 --2- 172.21.15.150:0/3053403647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bf4102580 0x7f9bf4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.806+0000 7f9bf99c7700 1 --2- 172.21.15.150:0/3053403647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4100ac0 0x7f9bf4100ed0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.806+0000 7f9bf99c7700 1 --2- 172.21.15.150:0/3053403647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf4101be0 0x7f9bf4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.806+0000 7f9bf99c7700 1 -- 172.21.15.150:0/3053403647 >> 172.21.15.150:0/3053403647 conn(0x7f9bf40fc0d0 msgr2=0x7f9bf40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:45.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.806+0000 7f9bf99c7700 1 -- 172.21.15.150:0/3053403647 shutdown_connections 2024-05-10T04:02:45.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.806+0000 7f9bf99c7700 1 -- 172.21.15.150:0/3053403647 wait complete. 2024-05-10T04:02:45.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.807+0000 7f9bf99c7700 1 Processor -- start 2024-05-10T04:02:45.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf99c7700 1 -- start start 2024-05-10T04:02:45.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf99c7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bf4100ac0 0x7f9bf4068e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:45.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf99c7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4101be0 0x7f9bf419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf99c7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf4102580 0x7f9bf419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf99c7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf419fa30 con 0x7f9bf4101be0 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf99c7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf419fba0 con 0x7f9bf4100ac0 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf99c7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf419fd60 con 0x7f9bf4102580 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf27fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4101be0 0x7f9bf419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf27fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4101be0 0x7f9bf419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37478/0 (socket says 172.21.15.150:37478) 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf27fc700 1 -- 172.21.15.150:0/2515274500 learned_addr learned my addr 172.21.15.150:0/2515274500 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf2ffd700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bf4100ac0 0x7f9bf4068e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf37fe700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf4102580 0x7f9bf419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf27fc700 1 -- 172.21.15.150:0/2515274500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf4102580 msgr2=0x7f9bf419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf27fc700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf4102580 0x7f9bf419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf27fc700 1 -- 172.21.15.150:0/2515274500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bf4100ac0 msgr2=0x7f9bf4068e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf27fc700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bf4100ac0 0x7f9bf4068e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf27fc700 1 -- 172.21.15.150:0/2515274500 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be4009d00 con 0x7f9bf4101be0 2024-05-10T04:02:45.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf37fe700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf4102580 0x7f9bf419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:45.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf2ffd700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bf4100ac0 0x7f9bf4068e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:45.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.808+0000 7f9bf27fc700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4101be0 0x7f9bf419cd50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00c9b0 tx=0x7f9bdc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:45.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.809+0000 7f9bf89c5700 1 -- 172.21.15.150:0/2515274500 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc007b00 con 0x7f9bf4101be0 2024-05-10T04:02:45.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.809+0000 7f9bf89c5700 1 -- 172.21.15.150:0/2515274500 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bdc007c60 con 0x7f9bf4101be0 2024-05-10T04:02:45.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.809+0000 7f9bf99c7700 1 -- 172.21.15.150:0/2515274500 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf419ffe0 con 0x7f9bf4101be0 2024-05-10T04:02:45.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.809+0000 7f9bf89c5700 1 -- 172.21.15.150:0/2515274500 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc018660 con 0x7f9bf4101be0 2024-05-10T04:02:45.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.809+0000 7f9bf99c7700 1 -- 172.21.15.150:0/2515274500 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf41a0490 con 0x7f9bf4101be0 2024-05-10T04:02:45.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.810+0000 7f9bf99c7700 1 -- 172.21.15.150:0/2515274500 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf4107cd0 con 0x7f9bf4101be0 2024-05-10T04:02:45.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.811+0000 7f9bf89c5700 1 -- 172.21.15.150:0/2515274500 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f9bdc018880 con 0x7f9bf4101be0 2024-05-10T04:02:45.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.811+0000 7f9bf89c5700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9be00a11c0 0x7f9be00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:45.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.812+0000 7f9bf89c5700 1 -- 172.21.15.150:0/2515274500 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9bdc0cce40 con 0x7f9bf4101be0 2024-05-10T04:02:45.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.812+0000 7f9bf2ffd700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9be00a11c0 0x7f9be00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:45.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.812+0000 7f9bf2ffd700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9be00a11c0 0x7f9be00a36d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9be400b830 tx=0x7f9be4007ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:45.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:45.815+0000 7f9bf89c5700 1 -- 172.21.15.150:0/2515274500 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9bdc096470 con 0x7f9bf4101be0 2024-05-10T04:02:45.963 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:45 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1103574061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:45.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:45 smithi150 ceph-mon[42477]: pgmap v233: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 541 B/s wr, 0 op/s 2024-05-10T04:02:45.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:45 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/436954626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:45.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:45 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4287393071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:02:46.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.013+0000 7f9bf99c7700 1 -- 172.21.15.150:0/2515274500 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9bf410c2a0 con 0x7f9bf4101be0 2024-05-10T04:02:46.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.013+0000 7f9bf89c5700 1 -- 172.21.15.150:0/2515274500 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9bdc099a90 con 0x7f9bf4101be0 2024-05-10T04:02:46.016 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:02:46.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.015+0000 7f9bf99c7700 1 -- 172.21.15.150:0/2515274500 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9be00a11c0 msgr2=0x7f9be00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:46.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.015+0000 7f9bf99c7700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9be00a11c0 0x7f9be00a36d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9be400b830 tx=0x7f9be4007ae0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.015+0000 7f9bf99c7700 1 -- 172.21.15.150:0/2515274500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4101be0 msgr2=0x7f9bf419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:46.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.015+0000 7f9bf99c7700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4101be0 0x7f9bf419cd50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00c9b0 tx=0x7f9bdc00cd70 comp rx=0 tx=0).stop 2024-05-10T04:02:46.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.015+0000 7f9bf99c7700 1 -- 172.21.15.150:0/2515274500 shutdown_connections 2024-05-10T04:02:46.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.015+0000 7f9bf99c7700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9be00a11c0 0x7f9be00a36d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.015+0000 7f9bf99c7700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf4102580 0x7f9bf419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.015+0000 7f9bf99c7700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bf4100ac0 0x7f9bf4068e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.015+0000 7f9bf99c7700 1 --2- 172.21.15.150:0/2515274500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4101be0 0x7f9bf419cd50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.015+0000 7f9bf99c7700 1 -- 172.21.15.150:0/2515274500 >> 172.21.15.150:0/2515274500 conn(0x7f9bf40fc0d0 msgr2=0x7f9bf4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:46.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.015+0000 7f9bf99c7700 1 -- 172.21.15.150:0/2515274500 shutdown_connections 2024-05-10T04:02:46.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.015+0000 7f9bf99c7700 1 -- 172.21.15.150:0/2515274500 wait complete. 2024-05-10T04:02:46.030 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:02:46.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.165+0000 7f9b6b43a700 1 -- 172.21.15.150:0/2983374368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b64105020 msgr2=0x7f9b64107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:46.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.165+0000 7f9b6b43a700 1 --2- 172.21.15.150:0/2983374368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b64105020 0x7f9b64107440 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009990 tx=0x7f9b54009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.165+0000 7f9b6b43a700 1 -- 172.21.15.150:0/2983374368 shutdown_connections 2024-05-10T04:02:46.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.165+0000 7f9b6b43a700 1 --2- 172.21.15.150:0/2983374368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b64107980 0x7f9b64109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.165+0000 7f9b6b43a700 1 --2- 172.21.15.150:0/2983374368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b641026c0 0x7f9b64104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.165+0000 7f9b6b43a700 1 --2- 172.21.15.150:0/2983374368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b64105020 0x7f9b64107440 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.165+0000 7f9b6b43a700 1 -- 172.21.15.150:0/2983374368 >> 172.21.15.150:0/2983374368 conn(0x7f9b640fc0e0 msgr2=0x7f9b640fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:46.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.165+0000 7f9b6b43a700 1 -- 172.21.15.150:0/2983374368 shutdown_connections 2024-05-10T04:02:46.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.165+0000 7f9b6b43a700 1 -- 172.21.15.150:0/2983374368 wait complete. 2024-05-10T04:02:46.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.165+0000 7f9b6b43a700 1 Processor -- start 2024-05-10T04:02:46.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.165+0000 7f9b6b43a700 1 -- start start 2024-05-10T04:02:46.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b6b43a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b641026c0 0x7f9b64198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:46.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b6b43a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b64105020 0x7f9b64198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:46.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b6b43a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b64107980 0x7f9b6419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:46.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b6b43a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6419faa0 con 0x7f9b64105020 2024-05-10T04:02:46.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b6b43a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6419fc10 con 0x7f9b64107980 2024-05-10T04:02:46.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b6b43a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6419fdd0 con 0x7f9b641026c0 2024-05-10T04:02:46.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b689d5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b64105020 0x7f9b64198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:46.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b691d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b641026c0 0x7f9b64198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:46.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b689d5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b64105020 0x7f9b64198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37496/0 (socket says 172.21.15.150:37496) 2024-05-10T04:02:46.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b689d5700 1 -- 172.21.15.150:0/583699751 learned_addr learned my addr 172.21.15.150:0/583699751 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:46.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b699d7700 1 --2- 172.21.15.150:0/583699751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b64107980 0x7f9b6419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:46.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b689d5700 1 -- 172.21.15.150:0/583699751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b641026c0 msgr2=0x7f9b64198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:46.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b689d5700 1 --2- 172.21.15.150:0/583699751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b641026c0 0x7f9b64198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b689d5700 1 -- 172.21.15.150:0/583699751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b64107980 msgr2=0x7f9b6419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:46.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b689d5700 1 --2- 172.21.15.150:0/583699751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b64107980 0x7f9b6419f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b689d5700 1 -- 172.21.15.150:0/583699751 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b540095f0 con 0x7f9b64105020 2024-05-10T04:02:46.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b689d5700 1 --2- 172.21.15.150:0/583699751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b64105020 0x7f9b64198550 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400c010 tx=0x7f9b54005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:46.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.166+0000 7f9b5a7fc700 1 -- 172.21.15.150:0/583699751 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54021070 con 0x7f9b64105020 2024-05-10T04:02:46.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.167+0000 7f9b5a7fc700 1 -- 172.21.15.150:0/583699751 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b540054f0 con 0x7f9b64105020 2024-05-10T04:02:46.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.167+0000 7f9b5a7fc700 1 -- 172.21.15.150:0/583699751 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54004bd0 con 0x7f9b64105020 2024-05-10T04:02:46.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.167+0000 7f9b6b43a700 1 -- 172.21.15.150:0/583699751 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b641004b0 con 0x7f9b64105020 2024-05-10T04:02:46.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.167+0000 7f9b6b43a700 1 -- 172.21.15.150:0/583699751 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b641008b0 con 0x7f9b64105020 2024-05-10T04:02:46.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.168+0000 7f9b6b43a700 1 -- 172.21.15.150:0/583699751 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b640fdc90 con 0x7f9b64105020 2024-05-10T04:02:46.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.169+0000 7f9b5a7fc700 1 -- 172.21.15.150:0/583699751 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f9b54025070 con 0x7f9b64105020 2024-05-10T04:02:46.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.169+0000 7f9b5a7fc700 1 --2- 172.21.15.150:0/583699751 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9b500a1240 0x7f9b500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:46.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.169+0000 7f9b5a7fc700 1 -- 172.21.15.150:0/583699751 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9b540d1e30 con 0x7f9b64105020 2024-05-10T04:02:46.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.169+0000 7f9b691d6700 1 --2- 172.21.15.150:0/583699751 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9b500a1240 0x7f9b500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:46.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.170+0000 7f9b691d6700 1 --2- 172.21.15.150:0/583699751 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9b500a1240 0x7f9b500a3750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9b64073e60 tx=0x7f9b600073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:46.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.173+0000 7f9b5a7fc700 1 -- 172.21.15.150:0/583699751 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9b5409b460 con 0x7f9b64105020 2024-05-10T04:02:46.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:45 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1103574061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:46.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:45 smithi181 ceph-mon[46602]: pgmap v233: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 541 B/s wr, 0 op/s 2024-05-10T04:02:46.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:45 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/436954626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:46.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:45 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4287393071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:02:46.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:45 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1103574061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:46.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:45 smithi161 ceph-mon[46918]: pgmap v233: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 541 B/s wr, 0 op/s 2024-05-10T04:02:46.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:45 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/436954626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:02:46.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:45 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4287393071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:02:46.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.371+0000 7f9b6b43a700 1 -- 172.21.15.150:0/583699751 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9b6410d440 con 0x7f9b64105020 2024-05-10T04:02:46.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.371+0000 7f9b5a7fc700 1 -- 172.21.15.150:0/583699751 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9b5409ea80 con 0x7f9b64105020 2024-05-10T04:02:46.374 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:02:46.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.374+0000 7f9b6b43a700 1 -- 172.21.15.150:0/583699751 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9b500a1240 msgr2=0x7f9b500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:46.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.374+0000 7f9b6b43a700 1 --2- 172.21.15.150:0/583699751 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9b500a1240 0x7f9b500a3750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9b64073e60 tx=0x7f9b600073d0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.374+0000 7f9b6b43a700 1 -- 172.21.15.150:0/583699751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b64105020 msgr2=0x7f9b64198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:46.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.374+0000 7f9b6b43a700 1 --2- 172.21.15.150:0/583699751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b64105020 0x7f9b64198550 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400c010 tx=0x7f9b54005f10 comp rx=0 tx=0).stop 2024-05-10T04:02:46.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.374+0000 7f9b6b43a700 1 -- 172.21.15.150:0/583699751 shutdown_connections 2024-05-10T04:02:46.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.374+0000 7f9b6b43a700 1 --2- 172.21.15.150:0/583699751 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9b500a1240 0x7f9b500a3750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.374+0000 7f9b6b43a700 1 --2- 172.21.15.150:0/583699751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b64107980 0x7f9b6419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.374+0000 7f9b6b43a700 1 --2- 172.21.15.150:0/583699751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b641026c0 0x7f9b64198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.374+0000 7f9b6b43a700 1 --2- 172.21.15.150:0/583699751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b64105020 0x7f9b64198550 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.374+0000 7f9b6b43a700 1 -- 172.21.15.150:0/583699751 >> 172.21.15.150:0/583699751 conn(0x7f9b640fc0e0 msgr2=0x7f9b64103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:46.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.374+0000 7f9b6b43a700 1 -- 172.21.15.150:0/583699751 shutdown_connections 2024-05-10T04:02:46.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.374+0000 7f9b6b43a700 1 -- 172.21.15.150:0/583699751 wait complete. 2024-05-10T04:02:46.389 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T04:02:46.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.519+0000 7fcf93e19700 1 -- 172.21.15.150:0/4062069307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8c102580 msgr2=0x7fcf8c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:46.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.519+0000 7fcf93e19700 1 --2- 172.21.15.150:0/4062069307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8c102580 0x7fcf8c108e30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400b0a0 tx=0x7fcf8400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.519+0000 7fcf93e19700 1 -- 172.21.15.150:0/4062069307 shutdown_connections 2024-05-10T04:02:46.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.519+0000 7fcf93e19700 1 --2- 172.21.15.150:0/4062069307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8c102580 0x7fcf8c108e30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.519+0000 7fcf93e19700 1 --2- 172.21.15.150:0/4062069307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf8c100ac0 0x7fcf8c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.519+0000 7fcf93e19700 1 --2- 172.21.15.150:0/4062069307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf8c101be0 0x7fcf8c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.519+0000 7fcf93e19700 1 -- 172.21.15.150:0/4062069307 >> 172.21.15.150:0/4062069307 conn(0x7fcf8c0fc0d0 msgr2=0x7fcf8c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:46.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.519+0000 7fcf93e19700 1 -- 172.21.15.150:0/4062069307 shutdown_connections 2024-05-10T04:02:46.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.519+0000 7fcf93e19700 1 -- 172.21.15.150:0/4062069307 wait complete. 2024-05-10T04:02:46.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf93e19700 1 Processor -- start 2024-05-10T04:02:46.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf93e19700 1 -- start start 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf93e19700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8c100ac0 0x7fcf8c104880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf93e19700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf8c101be0 0x7fcf8c102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf93e19700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf8c102580 0x7fcf8c103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf93e19700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c103b40 con 0x7fcf8c100ac0 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf93e19700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c103cb0 con 0x7fcf8c101be0 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf93e19700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c1a20d0 con 0x7fcf8c102580 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf91bb5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8c100ac0 0x7fcf8c104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf91bb5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8c100ac0 0x7fcf8c104880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37502/0 (socket says 172.21.15.150:37502) 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf91bb5700 1 -- 172.21.15.150:0/3016366679 learned_addr learned my addr 172.21.15.150:0/3016366679 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf913b4700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf8c101be0 0x7fcf8c102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf923b6700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf8c102580 0x7fcf8c103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf91bb5700 1 -- 172.21.15.150:0/3016366679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf8c102580 msgr2=0x7fcf8c103460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf91bb5700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf8c102580 0x7fcf8c103460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf91bb5700 1 -- 172.21.15.150:0/3016366679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf8c101be0 msgr2=0x7fcf8c102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:46.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf91bb5700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf8c101be0 0x7fcf8c102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf91bb5700 1 -- 172.21.15.150:0/3016366679 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf84009d00 con 0x7fcf8c100ac0 2024-05-10T04:02:46.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.520+0000 7fcf923b6700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf8c102580 0x7fcf8c103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:46.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.521+0000 7fcf91bb5700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8c100ac0 0x7fcf8c104880 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00c970 tx=0x7fcf7c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:46.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.521+0000 7fcf82ffd700 1 -- 172.21.15.150:0/3016366679 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c004220 con 0x7fcf8c100ac0 2024-05-10T04:02:46.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.521+0000 7fcf82ffd700 1 -- 172.21.15.150:0/3016366679 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf7c00ce60 con 0x7fcf8c100ac0 2024-05-10T04:02:46.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.521+0000 7fcf93e19700 1 -- 172.21.15.150:0/3016366679 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8c1a2330 con 0x7fcf8c100ac0 2024-05-10T04:02:46.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.521+0000 7fcf82ffd700 1 -- 172.21.15.150:0/3016366679 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c003c60 con 0x7fcf8c100ac0 2024-05-10T04:02:46.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.521+0000 7fcf93e19700 1 -- 172.21.15.150:0/3016366679 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8c1a2860 con 0x7fcf8c100ac0 2024-05-10T04:02:46.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.522+0000 7fcf93e19700 1 -- 172.21.15.150:0/3016366679 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8c0694d0 con 0x7fcf8c100ac0 2024-05-10T04:02:46.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.524+0000 7fcf82ffd700 1 -- 172.21.15.150:0/3016366679 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fcf7c007900 con 0x7fcf8c100ac0 2024-05-10T04:02:46.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.524+0000 7fcf82ffd700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fcf780a1240 0x7fcf780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:46.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.524+0000 7fcf82ffd700 1 -- 172.21.15.150:0/3016366679 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcf7c0cc950 con 0x7fcf8c100ac0 2024-05-10T04:02:46.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.524+0000 7fcf913b4700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fcf780a1240 0x7fcf780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:46.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.525+0000 7fcf913b4700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fcf780a1240 0x7fcf780a3750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c104220 tx=0x7fcf8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:46.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.528+0000 7fcf82ffd700 1 -- 172.21.15.150:0/3016366679 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcf7c095f70 con 0x7fcf8c100ac0 2024-05-10T04:02:46.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.741+0000 7fcf93e19700 1 -- 172.21.15.150:0/3016366679 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcf8c10c430 con 0x7fcf8c100ac0 2024-05-10T04:02:46.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.741+0000 7fcf82ffd700 1 -- 172.21.15.150:0/3016366679 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) v1 ==== 81+0+597 (secure 0 0 0) 0x7fcf7c01d020 con 0x7fcf8c100ac0 2024-05-10T04:02:46.744 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3457770606 2024-05-11T03:57:29.013448+0000 2024-05-10T04:02:46.744 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6801/999775795 2024-05-11T03:57:29.013448+0000 2024-05-10T04:02:46.744 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/977507934 2024-05-11T03:57:08.747290+0000 2024-05-10T04:02:46.744 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6800/3392737437 2024-05-11T03:57:08.747290+0000 2024-05-10T04:02:46.744 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3909466387 2024-05-11T03:57:29.013448+0000 2024-05-10T04:02:46.744 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/2393499570 2024-05-11T03:57:29.013448+0000 2024-05-10T04:02:46.744 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6800/999775795 2024-05-11T03:57:29.013448+0000 2024-05-10T04:02:46.744 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6801/3392737437 2024-05-11T03:57:08.747290+0000 2024-05-10T04:02:46.744 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/679926778 2024-05-11T03:57:08.747290+0000 2024-05-10T04:02:46.744 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/431676611 2024-05-11T03:57:08.747290+0000 2024-05-10T04:02:46.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.743+0000 7fcf93e19700 1 -- 172.21.15.150:0/3016366679 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fcf780a1240 msgr2=0x7fcf780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:46.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.743+0000 7fcf93e19700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fcf780a1240 0x7fcf780a3750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c104220 tx=0x7fcf8800b410 comp rx=0 tx=0).stop 2024-05-10T04:02:46.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.743+0000 7fcf93e19700 1 -- 172.21.15.150:0/3016366679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8c100ac0 msgr2=0x7fcf8c104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:46.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.743+0000 7fcf93e19700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8c100ac0 0x7fcf8c104880 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00c970 tx=0x7fcf7c00cc80 comp rx=0 tx=0).stop 2024-05-10T04:02:46.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.743+0000 7fcf93e19700 1 -- 172.21.15.150:0/3016366679 shutdown_connections 2024-05-10T04:02:46.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.743+0000 7fcf93e19700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fcf780a1240 0x7fcf780a3750 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.743+0000 7fcf93e19700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf8c102580 0x7fcf8c103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.743+0000 7fcf93e19700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf8c100ac0 0x7fcf8c104880 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.743+0000 7fcf93e19700 1 --2- 172.21.15.150:0/3016366679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf8c101be0 0x7fcf8c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:46.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.743+0000 7fcf93e19700 1 -- 172.21.15.150:0/3016366679 >> 172.21.15.150:0/3016366679 conn(0x7fcf8c0fc0d0 msgr2=0x7fcf8c0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:46.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.743+0000 7fcf93e19700 1 -- 172.21.15.150:0/3016366679 shutdown_connections 2024-05-10T04:02:46.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:46.743+0000 7fcf93e19700 1 -- 172.21.15.150:0/3016366679 wait complete. 2024-05-10T04:02:46.747 INFO:teuthology.orchestra.run.smithi150.stderr:listed 10 entries 2024-05-10T04:02:46.758 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi150.front.sepia.ceph.com... 2024-05-10T04:02:46.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:46.759 DEBUG:teuthology.orchestra.run.smithi150:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:02:46.791 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi150.stderr:ceph-fuse[95517]: fuse finished with error 0 and tester_r 0 2024-05-10T04:02:46.792 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T04:02:47.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:46 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2515274500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:02:47.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:46 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/583699751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:02:47.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:46 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3016366679' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:02:47.237 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:46 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2515274500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:02:47.237 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:46 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/583699751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:02:47.238 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:46 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3016366679' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:02:47.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:46 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2515274500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:02:47.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:46 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/583699751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:02:47.242 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:46 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3016366679' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:02:48.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:47 smithi150 ceph-mon[42477]: pgmap v234: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 464 B/s wr, 0 op/s 2024-05-10T04:02:48.237 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:47 smithi181 ceph-mon[46602]: pgmap v234: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 464 B/s wr, 0 op/s 2024-05-10T04:02:48.243 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:47 smithi161 ceph-mon[46918]: pgmap v234: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 464 B/s wr, 0 op/s 2024-05-10T04:02:50.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:50 smithi150 ceph-mon[42477]: pgmap v235: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-10T04:02:50.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:50 smithi181 ceph-mon[46602]: pgmap v235: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-10T04:02:50.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:50 smithi161 ceph-mon[46918]: pgmap v235: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-10T04:02:52.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:52 smithi150 ceph-mon[42477]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:02:52.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:52 smithi181 ceph-mon[46602]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:02:52.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:52 smithi161 ceph-mon[46918]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:02:52.894 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:02:52.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:02:52.895 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-10T04:02:52.929 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:02:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.114+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/718676181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a68069050 msgr2=0x7f3a68104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.114+0000 7f3a6fbd7700 1 --2- 172.21.15.150:0/718676181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a68069050 0x7f3a68104760 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009990 tx=0x7f3a64009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.114+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/718676181 shutdown_connections 2024-05-10T04:02:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.114+0000 7f3a6fbd7700 1 --2- 172.21.15.150:0/718676181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a681078a0 0x7f3a68109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.114+0000 7f3a6fbd7700 1 --2- 172.21.15.150:0/718676181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a68069050 0x7f3a68104760 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.114+0000 7f3a6fbd7700 1 --2- 172.21.15.150:0/718676181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68104d30 0x7f3a68107210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.114+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/718676181 >> 172.21.15.150:0/718676181 conn(0x7f3a680fc0d0 msgr2=0x7f3a680fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.114+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/718676181 shutdown_connections 2024-05-10T04:02:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.114+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/718676181 wait complete. 2024-05-10T04:02:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6fbd7700 1 Processor -- start 2024-05-10T04:02:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6fbd7700 1 -- start start 2024-05-10T04:02:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6fbd7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a68069050 0x7f3a68197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6fbd7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a68104d30 0x7f3a681984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6fbd7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a681078a0 0x7f3a6819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6fbd7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6819fa30 con 0x7f3a68069050 2024-05-10T04:02:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6fbd7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6819fb70 con 0x7f3a68104d30 2024-05-10T04:02:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6fbd7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6819fd00 con 0x7f3a681078a0 2024-05-10T04:02:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6d973700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a68069050 0x7f3a68197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6d973700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a68069050 0x7f3a68197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60586/0 (socket says 172.21.15.150:60586) 2024-05-10T04:02:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6d973700 1 -- 172.21.15.150:0/3741663404 learned_addr learned my addr 172.21.15.150:0/3741663404 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6e174700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a681078a0 0x7f3a6819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6d973700 1 -- 172.21.15.150:0/3741663404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a681078a0 msgr2=0x7f3a6819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6d172700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a68104d30 0x7f3a681984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6d973700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a681078a0 0x7f3a6819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6d973700 1 -- 172.21.15.150:0/3741663404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a68104d30 msgr2=0x7f3a681984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6d973700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a68104d30 0x7f3a681984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6d973700 1 -- 172.21.15.150:0/3741663404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a640095f0 con 0x7f3a68069050 2024-05-10T04:02:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6e174700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a681078a0 0x7f3a6819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6d172700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a68104d30 0x7f3a681984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.115+0000 7f3a6d973700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a68069050 0x7f3a68197f90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3a640098e0 tx=0x7f3a6400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.116+0000 7f3a5effd700 1 -- 172.21.15.150:0/3741663404 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6401d070 con 0x7f3a68069050 2024-05-10T04:02:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.116+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/3741663404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6819ff80 con 0x7f3a68069050 2024-05-10T04:02:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.116+0000 7f3a5effd700 1 -- 172.21.15.150:0/3741663404 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a64022470 con 0x7f3a68069050 2024-05-10T04:02:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.116+0000 7f3a5effd700 1 -- 172.21.15.150:0/3741663404 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64017650 con 0x7f3a68069050 2024-05-10T04:02:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.116+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/3741663404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a681a02f0 con 0x7f3a68069050 2024-05-10T04:02:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.117+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/3741663404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a680fdc80 con 0x7f3a68069050 2024-05-10T04:02:53.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.118+0000 7f3a5effd700 1 -- 172.21.15.150:0/3741663404 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3a640225e0 con 0x7f3a68069050 2024-05-10T04:02:53.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.118+0000 7f3a5effd700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3a540a1120 0x7f3a540a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:53.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.119+0000 7f3a6d172700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3a540a1120 0x7f3a540a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:53.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.119+0000 7f3a5effd700 1 -- 172.21.15.150:0/3741663404 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3a640ce1d0 con 0x7f3a68069050 2024-05-10T04:02:53.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.119+0000 7f3a6d172700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3a540a1120 0x7f3a540a3630 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3a58005fd0 tx=0x7f3a58005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:53.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.122+0000 7f3a5effd700 1 -- 172.21.15.150:0/3741663404 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3a640978a0 con 0x7f3a68069050 2024-05-10T04:02:53.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.321+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/3741663404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a6810d380 con 0x7f3a68069050 2024-05-10T04:02:53.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.323+0000 7f3a5effd700 1 -- 172.21.15.150:0/3741663404 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24027 (secure 0 0 0) 0x7f3a64097020 con 0x7f3a68069050 2024-05-10T04:02:53.326 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:53.326 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":85,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:02:36.063310+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:02:53.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.325+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/3741663404 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3a540a1120 msgr2=0x7f3a540a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:53.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.325+0000 7f3a6fbd7700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3a540a1120 0x7f3a540a3630 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3a58005fd0 tx=0x7f3a58005f00 comp rx=0 tx=0).stop 2024-05-10T04:02:53.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.325+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/3741663404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a68069050 msgr2=0x7f3a68197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:53.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.325+0000 7f3a6fbd7700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a68069050 0x7f3a68197f90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3a640098e0 tx=0x7f3a6400f690 comp rx=0 tx=0).stop 2024-05-10T04:02:53.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.325+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/3741663404 shutdown_connections 2024-05-10T04:02:53.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.325+0000 7f3a6fbd7700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3a540a1120 0x7f3a540a3630 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.325+0000 7f3a6fbd7700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a681078a0 0x7f3a6819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.325+0000 7f3a6fbd7700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a68069050 0x7f3a68197f90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.325+0000 7f3a6fbd7700 1 --2- 172.21.15.150:0/3741663404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a68104d30 0x7f3a681984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.325+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/3741663404 >> 172.21.15.150:0/3741663404 conn(0x7f3a680fc0d0 msgr2=0x7f3a68103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:53.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.325+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/3741663404 shutdown_connections 2024-05-10T04:02:53.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.325+0000 7f3a6fbd7700 1 -- 172.21.15.150:0/3741663404 wait complete. 2024-05-10T04:02:53.342 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:02:53.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.473+0000 7f5671a1a700 1 -- 172.21.15.150:0/534974264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f566c102590 msgr2=0x7f566c108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:53.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.473+0000 7f5671a1a700 1 --2- 172.21.15.150:0/534974264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f566c102590 0x7f566c108e40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f566400b0a0 tx=0x7f566400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.473+0000 7f5671a1a700 1 -- 172.21.15.150:0/534974264 shutdown_connections 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.473+0000 7f5671a1a700 1 --2- 172.21.15.150:0/534974264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f566c102590 0x7f566c108e40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.473+0000 7f5671a1a700 1 --2- 172.21.15.150:0/534974264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f566c100ad0 0x7f566c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.473+0000 7f5671a1a700 1 --2- 172.21.15.150:0/534974264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f566c101bf0 0x7f566c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.473+0000 7f5671a1a700 1 -- 172.21.15.150:0/534974264 >> 172.21.15.150:0/534974264 conn(0x7f566c0fc0e0 msgr2=0x7f566c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.473+0000 7f5671a1a700 1 -- 172.21.15.150:0/534974264 shutdown_connections 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.473+0000 7f5671a1a700 1 -- 172.21.15.150:0/534974264 wait complete. 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.474+0000 7f5671a1a700 1 Processor -- start 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.474+0000 7f5671a1a700 1 -- start start 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.474+0000 7f5671a1a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f566c100ad0 0x7f566c073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.474+0000 7f5671a1a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f566c101bf0 0x7f566c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.474+0000 7f5671a1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f566c102590 0x7f566c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.474+0000 7f5671a1a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c19fbb0 con 0x7f566c100ad0 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.474+0000 7f5671a1a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c19fd20 con 0x7f566c101bf0 2024-05-10T04:02:53.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.474+0000 7f5671a1a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c19fee0 con 0x7f566c102590 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.474+0000 7f566affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f566c100ad0 0x7f566c073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.474+0000 7f566affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f566c100ad0 0x7f566c073eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60606/0 (socket says 172.21.15.150:60606) 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.474+0000 7f566affd700 1 -- 172.21.15.150:0/677150711 learned_addr learned my addr 172.21.15.150:0/677150711 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.474+0000 7f566b7fe700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f566c102590 0x7f566c19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.475+0000 7f566a7fc700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f566c101bf0 0x7f566c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.475+0000 7f566affd700 1 -- 172.21.15.150:0/677150711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f566c102590 msgr2=0x7f566c19f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.475+0000 7f566affd700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f566c102590 0x7f566c19f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.475+0000 7f566affd700 1 -- 172.21.15.150:0/677150711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f566c101bf0 msgr2=0x7f566c19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.475+0000 7f566affd700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f566c101bf0 0x7f566c19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.475+0000 7f566affd700 1 -- 172.21.15.150:0/677150711 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5664009d00 con 0x7f566c100ad0 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.475+0000 7f566affd700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f566c100ad0 0x7f566c073eb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f565c00e940 tx=0x7f565c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.475+0000 7f5670a18700 1 -- 172.21.15.150:0/677150711 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c00cbd0 con 0x7f566c100ad0 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.475+0000 7f5671a1a700 1 -- 172.21.15.150:0/677150711 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566c1a01c0 con 0x7f566c100ad0 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.475+0000 7f5670a18700 1 -- 172.21.15.150:0/677150711 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f565c00ee90 con 0x7f566c100ad0 2024-05-10T04:02:53.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.475+0000 7f5670a18700 1 -- 172.21.15.150:0/677150711 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c003cc0 con 0x7f566c100ad0 2024-05-10T04:02:53.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.475+0000 7f5671a1a700 1 -- 172.21.15.150:0/677150711 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566c1a06d0 con 0x7f566c100ad0 2024-05-10T04:02:53.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.476+0000 7f5671a1a700 1 -- 172.21.15.150:0/677150711 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566c107ce0 con 0x7f566c100ad0 2024-05-10T04:02:53.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.477+0000 7f5670a18700 1 -- 172.21.15.150:0/677150711 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f565c00cd30 con 0x7f566c100ad0 2024-05-10T04:02:53.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.478+0000 7f5670a18700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56540a1120 0x7f56540a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:53.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.478+0000 7f5670a18700 1 -- 172.21.15.150:0/677150711 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f565c014070 con 0x7f566c100ad0 2024-05-10T04:02:53.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.478+0000 7f566a7fc700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56540a1120 0x7f56540a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:53.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.478+0000 7f566a7fc700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56540a1120 0x7f56540a3630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5660005950 tx=0x7f56600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:53.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.482+0000 7f5670a18700 1 -- 172.21.15.150:0/677150711 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f565c0963f0 con 0x7f566c100ad0 2024-05-10T04:02:53.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.681+0000 7f5671a1a700 1 -- 172.21.15.150:0/677150711 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f566c068730 con 0x7f566c100ad0 2024-05-10T04:02:53.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.683+0000 7f5670a18700 1 -- 172.21.15.150:0/677150711 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24027 (secure 0 0 0) 0x7f565c099a10 con 0x7f566c100ad0 2024-05-10T04:02:53.686 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:53.686 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":85,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:02:36.063310+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:02:53.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.685+0000 7f5671a1a700 1 -- 172.21.15.150:0/677150711 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56540a1120 msgr2=0x7f56540a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:53.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.685+0000 7f5671a1a700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56540a1120 0x7f56540a3630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5660005950 tx=0x7f56600058e0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.685+0000 7f5671a1a700 1 -- 172.21.15.150:0/677150711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f566c100ad0 msgr2=0x7f566c073eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:53.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.685+0000 7f5671a1a700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f566c100ad0 0x7f566c073eb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f565c00e940 tx=0x7f565c00ec50 comp rx=0 tx=0).stop 2024-05-10T04:02:53.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.685+0000 7f5671a1a700 1 -- 172.21.15.150:0/677150711 shutdown_connections 2024-05-10T04:02:53.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.685+0000 7f5671a1a700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56540a1120 0x7f56540a3630 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.685+0000 7f5671a1a700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f566c102590 0x7f566c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.685+0000 7f5671a1a700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f566c100ad0 0x7f566c073eb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.685+0000 7f5671a1a700 1 --2- 172.21.15.150:0/677150711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f566c101bf0 0x7f566c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.685+0000 7f5671a1a700 1 -- 172.21.15.150:0/677150711 >> 172.21.15.150:0/677150711 conn(0x7f566c0fc0e0 msgr2=0x7f566c104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:53.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.685+0000 7f5671a1a700 1 -- 172.21.15.150:0/677150711 shutdown_connections 2024-05-10T04:02:53.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.685+0000 7f5671a1a700 1 -- 172.21.15.150:0/677150711 wait complete. 2024-05-10T04:02:53.701 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T04:02:53.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.835+0000 7f2f54c7c700 1 -- 172.21.15.150:0/426992058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f500ffc20 msgr2=0x7f2f50100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:53.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.835+0000 7f2f54c7c700 1 --2- 172.21.15.150:0/426992058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f500ffc20 0x7f2f50100030 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009990 tx=0x7f2f40009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.835+0000 7f2f54c7c700 1 -- 172.21.15.150:0/426992058 shutdown_connections 2024-05-10T04:02:53.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.835+0000 7f2f54c7c700 1 --2- 172.21.15.150:0/426992058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f50100570 0x7f2f5010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.835+0000 7f2f54c7c700 1 --2- 172.21.15.150:0/426992058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f501031e0 0x7f2f501035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.835+0000 7f2f54c7c700 1 --2- 172.21.15.150:0/426992058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f500ffc20 0x7f2f50100030 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.835+0000 7f2f54c7c700 1 -- 172.21.15.150:0/426992058 >> 172.21.15.150:0/426992058 conn(0x7f2f500fb8f0 msgr2=0x7f2f500fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:53.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.835+0000 7f2f54c7c700 1 -- 172.21.15.150:0/426992058 shutdown_connections 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.835+0000 7f2f54c7c700 1 -- 172.21.15.150:0/426992058 wait complete. 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.836+0000 7f2f54c7c700 1 Processor -- start 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.836+0000 7f2f54c7c700 1 -- start start 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.836+0000 7f2f54c7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f500ffc20 0x7f2f50197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.836+0000 7f2f54c7c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50100570 0x7f2f50198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.836+0000 7f2f54c7c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f501031e0 0x7f2f5019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.836+0000 7f2f54c7c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5019f9c0 con 0x7f2f50100570 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.836+0000 7f2f54c7c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5019fb30 con 0x7f2f501031e0 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.836+0000 7f2f54c7c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5019fcf0 con 0x7f2f500ffc20 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.836+0000 7f2f4dd9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50100570 0x7f2f50198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.836+0000 7f2f4dd9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50100570 0x7f2f50198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60622/0 (socket says 172.21.15.150:60622) 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.836+0000 7f2f4dd9b700 1 -- 172.21.15.150:0/3730052673 learned_addr learned my addr 172.21.15.150:0/3730052673 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.836+0000 7f2f4ed9d700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f501031e0 0x7f2f5019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.836+0000 7f2f4e59c700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f500ffc20 0x7f2f50197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:53.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.837+0000 7f2f4dd9b700 1 -- 172.21.15.150:0/3730052673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f500ffc20 msgr2=0x7f2f50197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:53.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.837+0000 7f2f4dd9b700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f500ffc20 0x7f2f50197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.837+0000 7f2f4dd9b700 1 -- 172.21.15.150:0/3730052673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f501031e0 msgr2=0x7f2f5019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:53.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.837+0000 7f2f4dd9b700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f501031e0 0x7f2f5019f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:53.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.837+0000 7f2f4dd9b700 1 -- 172.21.15.150:0/3730052673 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f400095f0 con 0x7f2f50100570 2024-05-10T04:02:53.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.837+0000 7f2f4e59c700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f500ffc20 0x7f2f50197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:53.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.837+0000 7f2f4dd9b700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50100570 0x7f2f50198420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2f400098e0 tx=0x7f2f40009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:53.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.837+0000 7f2f3f7fe700 1 -- 172.21.15.150:0/3730052673 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4001d070 con 0x7f2f50100570 2024-05-10T04:02:53.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.837+0000 7f2f3f7fe700 1 -- 172.21.15.150:0/3730052673 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f40021410 con 0x7f2f50100570 2024-05-10T04:02:53.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.837+0000 7f2f54c7c700 1 -- 172.21.15.150:0/3730052673 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5019ff70 con 0x7f2f50100570 2024-05-10T04:02:53.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.837+0000 7f2f3f7fe700 1 -- 172.21.15.150:0/3730052673 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40017650 con 0x7f2f50100570 2024-05-10T04:02:53.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.837+0000 7f2f54c7c700 1 -- 172.21.15.150:0/3730052673 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f501a02e0 con 0x7f2f50100570 2024-05-10T04:02:53.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.838+0000 7f2f54c7c700 1 -- 172.21.15.150:0/3730052673 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f50109030 con 0x7f2f50100570 2024-05-10T04:02:53.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.839+0000 7f2f3f7fe700 1 -- 172.21.15.150:0/3730052673 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2f40021580 con 0x7f2f50100570 2024-05-10T04:02:53.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.840+0000 7f2f3f7fe700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2f380a11c0 0x7f2f380a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:53.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.840+0000 7f2f4e59c700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2f380a11c0 0x7f2f380a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:53.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.840+0000 7f2f3f7fe700 1 -- 172.21.15.150:0/3730052673 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2f400ce670 con 0x7f2f50100570 2024-05-10T04:02:53.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.840+0000 7f2f4e59c700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2f380a11c0 0x7f2f380a36d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2f50068120 tx=0x7f2f44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:53.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:53.843+0000 7f2f3f7fe700 1 -- 172.21.15.150:0/3730052673 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2f40097c40 con 0x7f2f50100570 2024-05-10T04:02:54.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.085+0000 7f2f54c7c700 1 -- 172.21.15.150:0/3730052673 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2f500678b0 con 0x7f2f50100570 2024-05-10T04:02:54.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.151+0000 7f2f3f7fe700 1 -- 172.21.15.150:0/3730052673 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2f400b9e60 con 0x7f2f50100570 2024-05-10T04:02:54.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.153+0000 7f2f54c7c700 1 -- 172.21.15.150:0/3730052673 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2f380a11c0 msgr2=0x7f2f380a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:54.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.153+0000 7f2f54c7c700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2f380a11c0 0x7f2f380a36d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2f50068120 tx=0x7f2f44009260 comp rx=0 tx=0).stop 2024-05-10T04:02:54.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.153+0000 7f2f54c7c700 1 -- 172.21.15.150:0/3730052673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50100570 msgr2=0x7f2f50198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:54.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.153+0000 7f2f54c7c700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50100570 0x7f2f50198420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2f400098e0 tx=0x7f2f40009580 comp rx=0 tx=0).stop 2024-05-10T04:02:54.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.153+0000 7f2f54c7c700 1 -- 172.21.15.150:0/3730052673 shutdown_connections 2024-05-10T04:02:54.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.153+0000 7f2f54c7c700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2f380a11c0 0x7f2f380a36d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:54.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.153+0000 7f2f54c7c700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f501031e0 0x7f2f5019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:54.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.153+0000 7f2f54c7c700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f500ffc20 0x7f2f50197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:54.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.153+0000 7f2f54c7c700 1 --2- 172.21.15.150:0/3730052673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50100570 0x7f2f50198420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:54.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.153+0000 7f2f54c7c700 1 -- 172.21.15.150:0/3730052673 >> 172.21.15.150:0/3730052673 conn(0x7f2f500fb8f0 msgr2=0x7f2f50106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:54.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.154+0000 7f2f54c7c700 1 -- 172.21.15.150:0/3730052673 shutdown_connections 2024-05-10T04:02:54.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.154+0000 7f2f54c7c700 1 -- 172.21.15.150:0/3730052673 wait complete. 2024-05-10T04:02:54.169 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-10T04:02:54.169 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-10T04:02:54.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.303+0000 7f9186b7f700 1 -- 172.21.15.150:0/3299189712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91801025a0 msgr2=0x7f9180108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:54.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.303+0000 7f9186b7f700 1 --2- 172.21.15.150:0/3299189712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91801025a0 0x7f9180108e50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f917800b0a0 tx=0x7f917800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:54.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.303+0000 7f9186b7f700 1 -- 172.21.15.150:0/3299189712 shutdown_connections 2024-05-10T04:02:54.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.303+0000 7f9186b7f700 1 --2- 172.21.15.150:0/3299189712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91801025a0 0x7f9180108e50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:54.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.303+0000 7f9186b7f700 1 --2- 172.21.15.150:0/3299189712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9180100ae0 0x7f9180100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:54.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.303+0000 7f9186b7f700 1 --2- 172.21.15.150:0/3299189712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9180101c00 0x7f9180102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:54.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.303+0000 7f9186b7f700 1 -- 172.21.15.150:0/3299189712 >> 172.21.15.150:0/3299189712 conn(0x7f91800fc0f0 msgr2=0x7f91800fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:54.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.303+0000 7f9186b7f700 1 -- 172.21.15.150:0/3299189712 shutdown_connections 2024-05-10T04:02:54.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.303+0000 7f9186b7f700 1 -- 172.21.15.150:0/3299189712 wait complete. 2024-05-10T04:02:54.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.303+0000 7f9186b7f700 1 Processor -- start 2024-05-10T04:02:54.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f9186b7f700 1 -- start start 2024-05-10T04:02:54.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f9186b7f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9180100ae0 0x7f918006a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:54.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f9186b7f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9180101c00 0x7f918019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:54.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f9186b7f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91801025a0 0x7f918019f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:54.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f9186b7f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918019fa50 con 0x7f9180101c00 2024-05-10T04:02:54.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f9186b7f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918019fbc0 con 0x7f91801025a0 2024-05-10T04:02:54.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f9186b7f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918019fd80 con 0x7f9180100ae0 2024-05-10T04:02:54.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f917ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9180101c00 0x7f918019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:54.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f917ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9180101c00 0x7f918019cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60640/0 (socket says 172.21.15.150:60640) 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f917ffff700 1 -- 172.21.15.150:0/3784393736 learned_addr learned my addr 172.21.15.150:0/3784393736 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f918491b700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9180100ae0 0x7f918006a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f918511c700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91801025a0 0x7f918019f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f917ffff700 1 -- 172.21.15.150:0/3784393736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9180100ae0 msgr2=0x7f918006a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f917ffff700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9180100ae0 0x7f918006a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f917ffff700 1 -- 172.21.15.150:0/3784393736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91801025a0 msgr2=0x7f918019f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f917ffff700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91801025a0 0x7f918019f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f917ffff700 1 -- 172.21.15.150:0/3784393736 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9178009d00 con 0x7f9180101c00 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f918511c700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91801025a0 0x7f918019f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f917ffff700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9180101c00 0x7f918019cdc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f916c00d950 tx=0x7f916c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.304+0000 7f917dffb700 1 -- 172.21.15.150:0/3784393736 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c009940 con 0x7f9180101c00 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.305+0000 7f917dffb700 1 -- 172.21.15.150:0/3784393736 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f916c010460 con 0x7f9180101c00 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.305+0000 7f9186b7f700 1 -- 172.21.15.150:0/3784393736 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91801a0060 con 0x7f9180101c00 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.305+0000 7f917dffb700 1 -- 172.21.15.150:0/3784393736 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c00f620 con 0x7f9180101c00 2024-05-10T04:02:54.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.305+0000 7f9186b7f700 1 -- 172.21.15.150:0/3784393736 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91801a0570 con 0x7f9180101c00 2024-05-10T04:02:54.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.306+0000 7f9186b7f700 1 -- 172.21.15.150:0/3784393736 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9180107cf0 con 0x7f9180101c00 2024-05-10T04:02:54.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.307+0000 7f917dffb700 1 -- 172.21.15.150:0/3784393736 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f916c0108d0 con 0x7f9180101c00 2024-05-10T04:02:54.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.307+0000 7f917dffb700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f91700a11c0 0x7f91700a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:54.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.308+0000 7f917dffb700 1 -- 172.21.15.150:0/3784393736 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f916c0cc970 con 0x7f9180101c00 2024-05-10T04:02:54.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.308+0000 7f918491b700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f91700a11c0 0x7f91700a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:54.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.308+0000 7f918491b700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f91700a11c0 0x7f91700a36d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9180101b00 tx=0x7f917400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:54.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.312+0000 7f917dffb700 1 -- 172.21.15.150:0/3784393736 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f916c095f10 con 0x7f9180101c00 2024-05-10T04:02:54.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:54 smithi150 ceph-mon[42477]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:02:54.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:54 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3741663404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:54.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:54 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/677150711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:54.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:54 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3730052673' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T04:02:54.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:54 smithi181 ceph-mon[46602]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:02:54.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:54 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3741663404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:54.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:54 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/677150711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:54.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:54 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3730052673' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T04:02:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:54 smithi161 ceph-mon[46918]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:02:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:54 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3741663404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:54 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/677150711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:02:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:54 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3730052673' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T04:02:54.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:54.554+0000 7f9186b7f700 1 -- 172.21.15.150:0/3784393736 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f918010c5a0 con 0x7f9180101c00 2024-05-10T04:02:55.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.158+0000 7f917dffb700 1 -- 172.21.15.150:0/3784393736 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f916c0160b0 con 0x7f9180101c00 2024-05-10T04:02:55.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.161+0000 7f9186b7f700 1 -- 172.21.15.150:0/3784393736 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f91700a11c0 msgr2=0x7f91700a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:55.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.161+0000 7f9186b7f700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f91700a11c0 0x7f91700a36d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9180101b00 tx=0x7f917400a3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:55.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.161+0000 7f9186b7f700 1 -- 172.21.15.150:0/3784393736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9180101c00 msgr2=0x7f918019cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:55.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.161+0000 7f9186b7f700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9180101c00 0x7f918019cdc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f916c00d950 tx=0x7f916c00dc60 comp rx=0 tx=0).stop 2024-05-10T04:02:55.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.161+0000 7f9186b7f700 1 -- 172.21.15.150:0/3784393736 shutdown_connections 2024-05-10T04:02:55.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.161+0000 7f9186b7f700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f91700a11c0 0x7f91700a36d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:55.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.161+0000 7f9186b7f700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91801025a0 0x7f918019f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:55.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.161+0000 7f9186b7f700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9180100ae0 0x7f918006a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:55.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.161+0000 7f9186b7f700 1 --2- 172.21.15.150:0/3784393736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9180101c00 0x7f918019cdc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:55.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.161+0000 7f9186b7f700 1 -- 172.21.15.150:0/3784393736 >> 172.21.15.150:0/3784393736 conn(0x7f91800fc0f0 msgr2=0x7f9180104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:55.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.161+0000 7f9186b7f700 1 -- 172.21.15.150:0/3784393736 shutdown_connections 2024-05-10T04:02:55.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.161+0000 7f9186b7f700 1 -- 172.21.15.150:0/3784393736 wait complete. 2024-05-10T04:02:55.183 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-10T04:02:55.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.319+0000 7f5645bcd700 1 -- 172.21.15.150:0/2996289714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5640105020 msgr2=0x7f5640107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:55.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.319+0000 7f5645bcd700 1 --2- 172.21.15.150:0/2996289714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5640105020 0x7f5640107440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5634009990 tx=0x7f5634009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:55.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.319+0000 7f5645bcd700 1 -- 172.21.15.150:0/2996289714 shutdown_connections 2024-05-10T04:02:55.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.319+0000 7f5645bcd700 1 --2- 172.21.15.150:0/2996289714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5640107980 0x7f5640109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:55.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.319+0000 7f5645bcd700 1 --2- 172.21.15.150:0/2996289714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56401026c0 0x7f5640104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:55.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.319+0000 7f5645bcd700 1 --2- 172.21.15.150:0/2996289714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5640105020 0x7f5640107440 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:55.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.320+0000 7f5645bcd700 1 -- 172.21.15.150:0/2996289714 >> 172.21.15.150:0/2996289714 conn(0x7f56400fc0e0 msgr2=0x7f56400fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:55.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.320+0000 7f5645bcd700 1 -- 172.21.15.150:0/2996289714 shutdown_connections 2024-05-10T04:02:55.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.320+0000 7f5645bcd700 1 -- 172.21.15.150:0/2996289714 wait complete. 2024-05-10T04:02:55.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.320+0000 7f5645bcd700 1 Processor -- start 2024-05-10T04:02:55.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f5645bcd700 1 -- start start 2024-05-10T04:02:55.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f5645bcd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56401026c0 0x7f5640197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:55.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f5645bcd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5640105020 0x7f5640198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:55.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f5645bcd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5640107980 0x7f564019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:55.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f5645bcd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564019fa50 con 0x7f5640107980 2024-05-10T04:02:55.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f5645bcd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564019fbc0 con 0x7f5640105020 2024-05-10T04:02:55.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f5645bcd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564019fd80 con 0x7f56401026c0 2024-05-10T04:02:55.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f563ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5640107980 0x7f564019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:55.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f563ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5640107980 0x7f564019f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60664/0 (socket says 172.21.15.150:60664) 2024-05-10T04:02:55.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f563ffff700 1 -- 172.21.15.150:0/960777114 learned_addr learned my addr 172.21.15.150:0/960777114 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f563f7fe700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56401026c0 0x7f5640197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f563effd700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5640105020 0x7f5640198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f563ffff700 1 -- 172.21.15.150:0/960777114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56401026c0 msgr2=0x7f5640197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f563ffff700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56401026c0 0x7f5640197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f563ffff700 1 -- 172.21.15.150:0/960777114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5640105020 msgr2=0x7f5640198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f563ffff700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5640105020 0x7f5640198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f563ffff700 1 -- 172.21.15.150:0/960777114 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56340095f0 con 0x7f5640107980 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.321+0000 7f563effd700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5640105020 0x7f5640198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.322+0000 7f563ffff700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5640107980 0x7f564019f440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f56380094f0 tx=0x7f563800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.322+0000 7f563cff9700 1 -- 172.21.15.150:0/960777114 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800ddd0 con 0x7f5640107980 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.322+0000 7f563cff9700 1 -- 172.21.15.150:0/960777114 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f563800f040 con 0x7f5640107980 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.322+0000 7f5645bcd700 1 -- 172.21.15.150:0/960777114 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56401a0060 con 0x7f5640107980 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.322+0000 7f563cff9700 1 -- 172.21.15.150:0/960777114 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638015400 con 0x7f5640107980 2024-05-10T04:02:55.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.322+0000 7f5645bcd700 1 -- 172.21.15.150:0/960777114 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56401a0520 con 0x7f5640107980 2024-05-10T04:02:55.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.323+0000 7f5645bcd700 1 -- 172.21.15.150:0/960777114 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56400fdc90 con 0x7f5640107980 2024-05-10T04:02:55.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.324+0000 7f563cff9700 1 -- 172.21.15.150:0/960777114 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5638004c40 con 0x7f5640107980 2024-05-10T04:02:55.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.325+0000 7f563cff9700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56280a11c0 0x7f56280a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:55.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.325+0000 7f563f7fe700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56280a11c0 0x7f56280a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:55.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.325+0000 7f563cff9700 1 -- 172.21.15.150:0/960777114 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f56380cc370 con 0x7f5640107980 2024-05-10T04:02:55.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.325+0000 7f563f7fe700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56280a11c0 0x7f56280a36d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5640073e60 tx=0x7f5630009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:55.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.329+0000 7f563cff9700 1 -- 172.21.15.150:0/960777114 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5638095910 con 0x7f5640107980 2024-05-10T04:02:55.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:55 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3730052673' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T04:02:55.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:55 smithi150 ceph-mon[42477]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:55.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:55 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3784393736' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-10T04:02:55.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:55 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3730052673' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T04:02:55.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:55 smithi181 ceph-mon[46602]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:55.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:55 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3784393736' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-10T04:02:55.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:55 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3730052673' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T04:02:55.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:55 smithi161 ceph-mon[46918]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:55.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:55 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3784393736' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-10T04:02:55.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:55.577+0000 7f5645bcd700 1 -- 172.21.15.150:0/960777114 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f5640068730 con 0x7f5640107980 2024-05-10T04:02:56.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.166+0000 7f563cff9700 1 -- 172.21.15.150:0/960777114 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f5638098f30 con 0x7f5640107980 2024-05-10T04:02:56.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.168+0000 7f5645bcd700 1 -- 172.21.15.150:0/960777114 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56280a11c0 msgr2=0x7f56280a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:56.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.168+0000 7f5645bcd700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56280a11c0 0x7f56280a36d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5640073e60 tx=0x7f5630009260 comp rx=0 tx=0).stop 2024-05-10T04:02:56.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.168+0000 7f5645bcd700 1 -- 172.21.15.150:0/960777114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5640107980 msgr2=0x7f564019f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:56.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.168+0000 7f5645bcd700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5640107980 0x7f564019f440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f56380094f0 tx=0x7f563800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:56.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.168+0000 7f5645bcd700 1 -- 172.21.15.150:0/960777114 shutdown_connections 2024-05-10T04:02:56.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.168+0000 7f5645bcd700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56280a11c0 0x7f56280a36d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:56.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.168+0000 7f5645bcd700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5640107980 0x7f564019f440 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:56.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.168+0000 7f5645bcd700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56401026c0 0x7f5640197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:56.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.169+0000 7f5645bcd700 1 --2- 172.21.15.150:0/960777114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5640105020 0x7f5640198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:56.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.169+0000 7f5645bcd700 1 -- 172.21.15.150:0/960777114 >> 172.21.15.150:0/960777114 conn(0x7f56400fc0e0 msgr2=0x7f5640103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:56.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.169+0000 7f5645bcd700 1 -- 172.21.15.150:0/960777114 shutdown_connections 2024-05-10T04:02:56.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.169+0000 7f5645bcd700 1 -- 172.21.15.150:0/960777114 wait complete. 2024-05-10T04:02:56.190 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-10T04:02:56.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.327+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2638888225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d48100ac0 msgr2=0x7f7d48100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:56.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.327+0000 7f7d4dc6d700 1 --2- 172.21.15.150:0/2638888225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d48100ac0 0x7f7d48100ed0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009990 tx=0x7f7d38009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:56.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.327+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2638888225 shutdown_connections 2024-05-10T04:02:56.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.327+0000 7f7d4dc6d700 1 --2- 172.21.15.150:0/2638888225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d48102580 0x7f7d48108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:56.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.327+0000 7f7d4dc6d700 1 --2- 172.21.15.150:0/2638888225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d48100ac0 0x7f7d48100ed0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:56.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.327+0000 7f7d4dc6d700 1 --2- 172.21.15.150:0/2638888225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d48101be0 0x7f7d48101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:56.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.327+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2638888225 >> 172.21.15.150:0/2638888225 conn(0x7f7d480fc0d0 msgr2=0x7f7d480fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.327+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2638888225 shutdown_connections 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.327+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2638888225 wait complete. 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.328+0000 7f7d4dc6d700 1 Processor -- start 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.328+0000 7f7d4dc6d700 1 -- start start 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.328+0000 7f7d4dc6d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d48100ac0 0x7f7d48196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.328+0000 7f7d4dc6d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d48101be0 0x7f7d48197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.328+0000 7f7d4dc6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d48102580 0x7f7d4819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.328+0000 7f7d4dc6d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4819e910 con 0x7f7d48101be0 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.328+0000 7f7d4dc6d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4819ea80 con 0x7f7d48100ac0 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.328+0000 7f7d4dc6d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4819ec40 con 0x7f7d48102580 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d46ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d48101be0 0x7f7d48197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d46ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d48101be0 0x7f7d48197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60676/0 (socket says 172.21.15.150:60676) 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d46ffd700 1 -- 172.21.15.150:0/2622052560 learned_addr learned my addr 172.21.15.150:0/2622052560 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d477fe700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d48100ac0 0x7f7d48196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:56.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d47fff700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d48102580 0x7f7d4819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:56.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d46ffd700 1 -- 172.21.15.150:0/2622052560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d48102580 msgr2=0x7f7d4819e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:56.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d46ffd700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d48102580 0x7f7d4819e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:56.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d46ffd700 1 -- 172.21.15.150:0/2622052560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d48100ac0 msgr2=0x7f7d48196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:56.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d46ffd700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d48100ac0 0x7f7d48196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:56.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d46ffd700 1 -- 172.21.15.150:0/2622052560 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d380095f0 con 0x7f7d48101be0 2024-05-10T04:02:56.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d46ffd700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d48101be0 0x7f7d48197370 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000d950 tx=0x7f7d3000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:56.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d44ff9700 1 -- 172.21.15.150:0/2622052560 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30009940 con 0x7f7d48101be0 2024-05-10T04:02:56.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d44ff9700 1 -- 172.21.15.150:0/2622052560 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d30010460 con 0x7f7d48101be0 2024-05-10T04:02:56.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2622052560 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4819ef20 con 0x7f7d48101be0 2024-05-10T04:02:56.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d44ff9700 1 -- 172.21.15.150:0/2622052560 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000f620 con 0x7f7d48101be0 2024-05-10T04:02:56.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.329+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2622052560 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4819f340 con 0x7f7d48101be0 2024-05-10T04:02:56.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.330+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2622052560 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d48107cd0 con 0x7f7d48101be0 2024-05-10T04:02:56.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.332+0000 7f7d44ff9700 1 -- 172.21.15.150:0/2622052560 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7d300108d0 con 0x7f7d48101be0 2024-05-10T04:02:56.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.333+0000 7f7d44ff9700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7d340a11c0 0x7f7d340a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:56.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.333+0000 7f7d44ff9700 1 -- 172.21.15.150:0/2622052560 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7d300cc980 con 0x7f7d48101be0 2024-05-10T04:02:56.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.333+0000 7f7d477fe700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7d340a11c0 0x7f7d340a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:56.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.333+0000 7f7d477fe700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7d340a11c0 0x7f7d340a36d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d380038c0 tx=0x7f7d3801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:56.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.337+0000 7f7d44ff9700 1 -- 172.21.15.150:0/2622052560 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7d30095ec0 con 0x7f7d48101be0 2024-05-10T04:02:56.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:56 smithi150 ceph-mon[42477]: pgmap v238: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:02:56.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:56 smithi150 ceph-mon[42477]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T04:02:56.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:56 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3784393736' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-10T04:02:56.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:56 smithi150 ceph-mon[42477]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T04:02:56.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:56 smithi150 ceph-mon[42477]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:56.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:56 smithi150 ceph-mon[42477]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-10T04:02:56.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:56 smithi150 ceph-mon[42477]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-10T04:02:56.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:56 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/960777114' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T04:02:56.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:56 smithi181 ceph-mon[46602]: pgmap v238: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:02:56.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:56 smithi181 ceph-mon[46602]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T04:02:56.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:56 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3784393736' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-10T04:02:56.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:56 smithi181 ceph-mon[46602]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T04:02:56.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:56 smithi181 ceph-mon[46602]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:56.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:56 smithi181 ceph-mon[46602]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-10T04:02:56.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:56 smithi181 ceph-mon[46602]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-10T04:02:56.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:56 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/960777114' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T04:02:56.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:56 smithi161 ceph-mon[46918]: pgmap v238: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:02:56.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:56 smithi161 ceph-mon[46918]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T04:02:56.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:56 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3784393736' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-10T04:02:56.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:56 smithi161 ceph-mon[46918]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T04:02:56.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:56 smithi161 ceph-mon[46918]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:02:56.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:56 smithi161 ceph-mon[46918]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-10T04:02:56.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:56 smithi161 ceph-mon[46918]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-10T04:02:56.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:56 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/960777114' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T04:02:56.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:56.586+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2622052560 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f7d480694d0 con 0x7f7d48101be0 2024-05-10T04:02:57.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.182+0000 7f7d44ff9700 1 -- 172.21.15.150:0/2622052560 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f7d300160b0 con 0x7f7d48101be0 2024-05-10T04:02:57.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.185+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2622052560 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7d340a11c0 msgr2=0x7f7d340a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:57.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.185+0000 7f7d4dc6d700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7d340a11c0 0x7f7d340a36d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d380038c0 tx=0x7f7d3801a040 comp rx=0 tx=0).stop 2024-05-10T04:02:57.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.185+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2622052560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d48101be0 msgr2=0x7f7d48197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:57.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.185+0000 7f7d4dc6d700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d48101be0 0x7f7d48197370 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000d950 tx=0x7f7d3000dc60 comp rx=0 tx=0).stop 2024-05-10T04:02:57.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.185+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2622052560 shutdown_connections 2024-05-10T04:02:57.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.185+0000 7f7d4dc6d700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7d340a11c0 0x7f7d340a36d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.185+0000 7f7d4dc6d700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d48102580 0x7f7d4819e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.185+0000 7f7d4dc6d700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d48100ac0 0x7f7d48196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.185+0000 7f7d4dc6d700 1 --2- 172.21.15.150:0/2622052560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d48101be0 0x7f7d48197370 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.185+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2622052560 >> 172.21.15.150:0/2622052560 conn(0x7f7d480fc0d0 msgr2=0x7f7d48104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:57.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.185+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2622052560 shutdown_connections 2024-05-10T04:02:57.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.185+0000 7f7d4dc6d700 1 -- 172.21.15.150:0/2622052560 wait complete. 2024-05-10T04:02:57.203 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-10T04:02:57.203 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-10T04:02:57.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.337+0000 7fc26de29700 1 -- 172.21.15.150:0/1338290963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc268100ac0 msgr2=0x7fc268100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:57.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.337+0000 7fc26de29700 1 --2- 172.21.15.150:0/1338290963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc268100ac0 0x7fc268100ed0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc250009990 tx=0x7fc250009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.337+0000 7fc26de29700 1 -- 172.21.15.150:0/1338290963 shutdown_connections 2024-05-10T04:02:57.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.337+0000 7fc26de29700 1 --2- 172.21.15.150:0/1338290963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc268102580 0x7fc268108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.337+0000 7fc26de29700 1 --2- 172.21.15.150:0/1338290963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc268100ac0 0x7fc268100ed0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.337+0000 7fc26de29700 1 --2- 172.21.15.150:0/1338290963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc268101be0 0x7fc268101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.337+0000 7fc26de29700 1 -- 172.21.15.150:0/1338290963 >> 172.21.15.150:0/1338290963 conn(0x7fc2680fc0d0 msgr2=0x7fc2680fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.337+0000 7fc26de29700 1 -- 172.21.15.150:0/1338290963 shutdown_connections 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.337+0000 7fc26de29700 1 -- 172.21.15.150:0/1338290963 wait complete. 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.338+0000 7fc26de29700 1 Processor -- start 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.338+0000 7fc26de29700 1 -- start start 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.338+0000 7fc26de29700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc268100ac0 0x7fc26810b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.338+0000 7fc26de29700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc268101be0 0x7fc2681099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.338+0000 7fc26de29700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc268102580 0x7fc268109f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.338+0000 7fc26de29700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26810a600 con 0x7fc268101be0 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.338+0000 7fc26de29700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26810a770 con 0x7fc268102580 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.338+0000 7fc26de29700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2681a1ff0 con 0x7fc268100ac0 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.338+0000 7fc266ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc268101be0 0x7fc2681099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.338+0000 7fc266ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc268101be0 0x7fc2681099e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60696/0 (socket says 172.21.15.150:60696) 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.338+0000 7fc266ffd700 1 -- 172.21.15.150:0/3195173055 learned_addr learned my addr 172.21.15.150:0/3195173055 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.338+0000 7fc267fff700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc268102580 0x7fc268109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.339+0000 7fc2677fe700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc268100ac0 0x7fc26810b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.339+0000 7fc266ffd700 1 -- 172.21.15.150:0/3195173055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc268100ac0 msgr2=0x7fc26810b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.339+0000 7fc266ffd700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc268100ac0 0x7fc26810b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.339+0000 7fc266ffd700 1 -- 172.21.15.150:0/3195173055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc268102580 msgr2=0x7fc268109f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.339+0000 7fc266ffd700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc268102580 0x7fc268109f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.339+0000 7fc266ffd700 1 -- 172.21.15.150:0/3195173055 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2500095f0 con 0x7fc268101be0 2024-05-10T04:02:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.339+0000 7fc2677fe700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc268100ac0 0x7fc26810b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.339+0000 7fc266ffd700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc268101be0 0x7fc2681099e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc25800e9a0 tx=0x7fc25800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.339+0000 7fc264ff9700 1 -- 172.21.15.150:0/3195173055 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258009790 con 0x7fc268101be0 2024-05-10T04:02:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.339+0000 7fc26de29700 1 -- 172.21.15.150:0/3195173055 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2681a2250 con 0x7fc268101be0 2024-05-10T04:02:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.339+0000 7fc264ff9700 1 -- 172.21.15.150:0/3195173055 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc25800cbd0 con 0x7fc268101be0 2024-05-10T04:02:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.339+0000 7fc264ff9700 1 -- 172.21.15.150:0/3195173055 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258018680 con 0x7fc268101be0 2024-05-10T04:02:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.339+0000 7fc26de29700 1 -- 172.21.15.150:0/3195173055 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2681a2730 con 0x7fc268101be0 2024-05-10T04:02:57.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.340+0000 7fc26de29700 1 -- 172.21.15.150:0/3195173055 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc268107cd0 con 0x7fc268101be0 2024-05-10T04:02:57.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.341+0000 7fc264ff9700 1 -- 172.21.15.150:0/3195173055 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc25800cd40 con 0x7fc268101be0 2024-05-10T04:02:57.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.342+0000 7fc264ff9700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc2540a11c0 0x7fc2540a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:57.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.342+0000 7fc2677fe700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc2540a11c0 0x7fc2540a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:57.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.342+0000 7fc264ff9700 1 -- 172.21.15.150:0/3195173055 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc258014070 con 0x7fc268101be0 2024-05-10T04:02:57.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.342+0000 7fc2677fe700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc2540a11c0 0x7fc2540a36d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc25000b2e0 tx=0x7fc250005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.346+0000 7fc264ff9700 1 -- 172.21.15.150:0/3195173055 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc258052d60 con 0x7fc268101be0 2024-05-10T04:02:57.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:57 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] up:active 2024-05-10T04:02:57.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:57 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/960777114' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T04:02:57.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:57 smithi150 ceph-mon[42477]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T04:02:57.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:57 smithi150 ceph-mon[42477]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T04:02:57.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:57 smithi150 ceph-mon[42477]: Cluster is now healthy 2024-05-10T04:02:57.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:57 smithi150 ceph-mon[42477]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-10T04:02:57.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:57 smithi150 ceph-mon[42477]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-10T04:02:57.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:57 smithi150 ceph-mon[42477]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-10T04:02:57.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:57 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2622052560' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T04:02:57.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:57 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] up:active 2024-05-10T04:02:57.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:57 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/960777114' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T04:02:57.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:57 smithi181 ceph-mon[46602]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T04:02:57.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:57 smithi181 ceph-mon[46602]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T04:02:57.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:57 smithi181 ceph-mon[46602]: Cluster is now healthy 2024-05-10T04:02:57.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:57 smithi181 ceph-mon[46602]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-10T04:02:57.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:57 smithi181 ceph-mon[46602]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-10T04:02:57.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:57 smithi181 ceph-mon[46602]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-10T04:02:57.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:57 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2622052560' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T04:02:57.492 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:02:57 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-b[83518]: 2024-05-10T04:02:57.086+0000 7f6a83230700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-10T04:02:57.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:57 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] up:active 2024-05-10T04:02:57.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:57 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/960777114' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T04:02:57.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:57 smithi161 ceph-mon[46918]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T04:02:57.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:57 smithi161 ceph-mon[46918]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T04:02:57.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:57 smithi161 ceph-mon[46918]: Cluster is now healthy 2024-05-10T04:02:57.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:57 smithi161 ceph-mon[46918]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-10T04:02:57.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:57 smithi161 ceph-mon[46918]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-10T04:02:57.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:57 smithi161 ceph-mon[46918]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-10T04:02:57.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:57 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2622052560' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T04:02:57.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.577+0000 7fc26de29700 1 -- 172.21.15.150:0/3195173055 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fc268061650 con 0x7fc2540a11c0 2024-05-10T04:02:57.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.605+0000 7fc264ff9700 1 -- 172.21.15.150:0/3195173055 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc268061650 con 0x7fc2540a11c0 2024-05-10T04:02:57.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.607+0000 7fc26de29700 1 -- 172.21.15.150:0/3195173055 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc2540a11c0 msgr2=0x7fc2540a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:57.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.607+0000 7fc26de29700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc2540a11c0 0x7fc2540a36d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc25000b2e0 tx=0x7fc250005fb0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.607+0000 7fc26de29700 1 -- 172.21.15.150:0/3195173055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc268101be0 msgr2=0x7fc2681099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:57.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.607+0000 7fc26de29700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc268101be0 0x7fc2681099e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc25800e9a0 tx=0x7fc25800ed60 comp rx=0 tx=0).stop 2024-05-10T04:02:57.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.607+0000 7fc26de29700 1 -- 172.21.15.150:0/3195173055 shutdown_connections 2024-05-10T04:02:57.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.607+0000 7fc26de29700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc2540a11c0 0x7fc2540a36d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.607+0000 7fc26de29700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc268102580 0x7fc268109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.607+0000 7fc26de29700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc268100ac0 0x7fc26810b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.607+0000 7fc26de29700 1 --2- 172.21.15.150:0/3195173055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc268101be0 0x7fc2681099e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.607+0000 7fc26de29700 1 -- 172.21.15.150:0/3195173055 >> 172.21.15.150:0/3195173055 conn(0x7fc2680fc0d0 msgr2=0x7fc268104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:57.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.608+0000 7fc26de29700 1 -- 172.21.15.150:0/3195173055 shutdown_connections 2024-05-10T04:02:57.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.608+0000 7fc26de29700 1 -- 172.21.15.150:0/3195173055 wait complete. 2024-05-10T04:02:57.623 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-10T04:02:57.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.758+0000 7fb45850e700 1 -- 172.21.15.150:0/3615860429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb450101110 msgr2=0x7fb450101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:57.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.758+0000 7fb45850e700 1 --2- 172.21.15.150:0/3615860429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb450101110 0x7fb450101520 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb440009990 tx=0x7fb440009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.758+0000 7fb45850e700 1 -- 172.21.15.150:0/3615860429 shutdown_connections 2024-05-10T04:02:57.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.758+0000 7fb45850e700 1 --2- 172.21.15.150:0/3615860429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb450102950 0x7fb450109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.758+0000 7fb45850e700 1 --2- 172.21.15.150:0/3615860429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb450101110 0x7fb450101520 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.758+0000 7fb45850e700 1 --2- 172.21.15.150:0/3615860429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb450102000 0x7fb450102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.758+0000 7fb45850e700 1 -- 172.21.15.150:0/3615860429 >> 172.21.15.150:0/3615860429 conn(0x7fb4500fb8f0 msgr2=0x7fb4500fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:57.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.758+0000 7fb45850e700 1 -- 172.21.15.150:0/3615860429 shutdown_connections 2024-05-10T04:02:57.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.758+0000 7fb45850e700 1 -- 172.21.15.150:0/3615860429 wait complete. 2024-05-10T04:02:57.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb45850e700 1 Processor -- start 2024-05-10T04:02:57.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb45850e700 1 -- start start 2024-05-10T04:02:57.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb45850e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb450101110 0x7fb450100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:57.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb45850e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb450102000 0x7fb45019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:57.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb45850e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb450102950 0x7fb45019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:57.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb45850e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45019fbe0 con 0x7fb450102000 2024-05-10T04:02:57.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb45850e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45019fd50 con 0x7fb450102950 2024-05-10T04:02:57.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb45850e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45019ff10 con 0x7fb450101110 2024-05-10T04:02:57.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb455aa9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb450102000 0x7fb45019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:57.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb455aa9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb450102000 0x7fb45019ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60736/0 (socket says 172.21.15.150:60736) 2024-05-10T04:02:57.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb455aa9700 1 -- 172.21.15.150:0/2003997676 learned_addr learned my addr 172.21.15.150:0/2003997676 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb4562aa700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb450101110 0x7fb450100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb456aab700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb450102950 0x7fb45019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb455aa9700 1 -- 172.21.15.150:0/2003997676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb450101110 msgr2=0x7fb450100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb455aa9700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb450101110 0x7fb450100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb455aa9700 1 -- 172.21.15.150:0/2003997676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb450102950 msgr2=0x7fb45019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb455aa9700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb450102950 0x7fb45019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb455aa9700 1 -- 172.21.15.150:0/2003997676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4400095f0 con 0x7fb450102000 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb456aab700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb450102950 0x7fb45019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.759+0000 7fb4562aa700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb450101110 0x7fb450100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.760+0000 7fb455aa9700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb450102000 0x7fb45019ce70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00e9a0 tx=0x7fb44c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.760+0000 7fb4477fe700 1 -- 172.21.15.150:0/2003997676 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c009790 con 0x7fb450102000 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.760+0000 7fb45850e700 1 -- 172.21.15.150:0/2003997676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4501a01f0 con 0x7fb450102000 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.760+0000 7fb4477fe700 1 -- 172.21.15.150:0/2003997676 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb44c00cbd0 con 0x7fb450102000 2024-05-10T04:02:57.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.760+0000 7fb4477fe700 1 -- 172.21.15.150:0/2003997676 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c018680 con 0x7fb450102000 2024-05-10T04:02:57.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.760+0000 7fb45850e700 1 -- 172.21.15.150:0/2003997676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4501a0750 con 0x7fb450102000 2024-05-10T04:02:57.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.761+0000 7fb45850e700 1 -- 172.21.15.150:0/2003997676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4501080f0 con 0x7fb450102000 2024-05-10T04:02:57.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.762+0000 7fb4477fe700 1 -- 172.21.15.150:0/2003997676 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb44c00cd40 con 0x7fb450102000 2024-05-10T04:02:57.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.763+0000 7fb4477fe700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb43c0a11c0 0x7fb43c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:57.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.763+0000 7fb4562aa700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb43c0a11c0 0x7fb43c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:57.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.763+0000 7fb4477fe700 1 -- 172.21.15.150:0/2003997676 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb44c014070 con 0x7fb450102000 2024-05-10T04:02:57.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.763+0000 7fb4562aa700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb43c0a11c0 0x7fb43c0a36d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb44000b2e0 tx=0x7fb44001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:57.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.767+0000 7fb4477fe700 1 -- 172.21.15.150:0/2003997676 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb44c096860 con 0x7fb450102000 2024-05-10T04:02:57.963 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:02:57 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T04:02:57.583+0000 7f4dcc453700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T04:02:57.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:02:57 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T04:02:57.583+0000 7f4dcc453700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T04:02:57.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:02:57 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T04:02:57.583+0000 7f4dcc453700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T04:02:57.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:02:57 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T04:02:57.583+0000 7f4dcc453700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T04:02:57.964 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:02:57 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T04:02:57.583+0000 7f4dcc453700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T04:02:57.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:57.997+0000 7fb45850e700 1 -- 172.21.15.150:0/2003997676 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fb45010c690 con 0x7fb43c0a11c0 2024-05-10T04:02:58.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.020+0000 7fb4477fe700 1 -- 172.21.15.150:0/2003997676 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb45010c690 con 0x7fb43c0a11c0 2024-05-10T04:02:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.022+0000 7fb45850e700 1 -- 172.21.15.150:0/2003997676 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb43c0a11c0 msgr2=0x7fb43c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.022+0000 7fb45850e700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb43c0a11c0 0x7fb43c0a36d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb44000b2e0 tx=0x7fb44001a040 comp rx=0 tx=0).stop 2024-05-10T04:02:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.022+0000 7fb45850e700 1 -- 172.21.15.150:0/2003997676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb450102000 msgr2=0x7fb45019ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.022+0000 7fb45850e700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb450102000 0x7fb45019ce70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00e9a0 tx=0x7fb44c00ed60 comp rx=0 tx=0).stop 2024-05-10T04:02:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.022+0000 7fb45850e700 1 -- 172.21.15.150:0/2003997676 shutdown_connections 2024-05-10T04:02:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.022+0000 7fb45850e700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb43c0a11c0 0x7fb43c0a36d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.022+0000 7fb45850e700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb450102950 0x7fb45019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.022+0000 7fb45850e700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb450101110 0x7fb450100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.022+0000 7fb45850e700 1 --2- 172.21.15.150:0/2003997676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb450102000 0x7fb45019ce70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.022+0000 7fb45850e700 1 -- 172.21.15.150:0/2003997676 >> 172.21.15.150:0/2003997676 conn(0x7fb4500fb8f0 msgr2=0x7fb450105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:58.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.022+0000 7fb45850e700 1 -- 172.21.15.150:0/2003997676 shutdown_connections 2024-05-10T04:02:58.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.023+0000 7fb45850e700 1 -- 172.21.15.150:0/2003997676 wait complete. 2024-05-10T04:02:58.038 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-10T04:02:58.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.173+0000 7f7111449700 1 -- 172.21.15.150:0/3419657739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c101bf0 msgr2=0x7f710c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:58.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.173+0000 7f7111449700 1 --2- 172.21.15.150:0/3419657739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c101bf0 0x7f710c102000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7100009990 tx=0x7f7100009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.173+0000 7f7111449700 1 -- 172.21.15.150:0/3419657739 shutdown_connections 2024-05-10T04:02:58.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.173+0000 7f7111449700 1 --2- 172.21.15.150:0/3419657739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f710c102590 0x7f710c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.173+0000 7f7111449700 1 --2- 172.21.15.150:0/3419657739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710c100ad0 0x7f710c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.173+0000 7f7111449700 1 --2- 172.21.15.150:0/3419657739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c101bf0 0x7f710c102000 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.173+0000 7f7111449700 1 -- 172.21.15.150:0/3419657739 >> 172.21.15.150:0/3419657739 conn(0x7f710c0fc0e0 msgr2=0x7f710c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:58.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.173+0000 7f7111449700 1 -- 172.21.15.150:0/3419657739 shutdown_connections 2024-05-10T04:02:58.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.173+0000 7f7111449700 1 -- 172.21.15.150:0/3419657739 wait complete. 2024-05-10T04:02:58.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f7111449700 1 Processor -- start 2024-05-10T04:02:58.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f7111449700 1 -- start start 2024-05-10T04:02:58.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f7111449700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f710c100ad0 0x7f710c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:58.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f7111449700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710c101bf0 0x7f710c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:58.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f7111449700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c102590 0x7f710c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:58.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f7111449700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c19fa40 con 0x7f710c102590 2024-05-10T04:02:58.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f7111449700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c19fbb0 con 0x7f710c100ad0 2024-05-10T04:02:58.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f7111449700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c19fd70 con 0x7f710c101bf0 2024-05-10T04:02:58.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f710b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c102590 0x7f710c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:58.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f710b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c102590 0x7f710c19f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60766/0 (socket says 172.21.15.150:60766) 2024-05-10T04:02:58.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f710b7fe700 1 -- 172.21.15.150:0/777765537 learned_addr learned my addr 172.21.15.150:0/777765537 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:58.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f710a7fc700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710c101bf0 0x7f710c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:58.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f710affd700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f710c100ad0 0x7f710c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:58.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f710b7fe700 1 -- 172.21.15.150:0/777765537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710c101bf0 msgr2=0x7f710c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:58.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f710b7fe700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710c101bf0 0x7f710c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f710b7fe700 1 -- 172.21.15.150:0/777765537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f710c100ad0 msgr2=0x7f710c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:58.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f710b7fe700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f710c100ad0 0x7f710c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f710b7fe700 1 -- 172.21.15.150:0/777765537 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71000095f0 con 0x7f710c102590 2024-05-10T04:02:58.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.174+0000 7f710a7fc700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710c101bf0 0x7f710c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:02:58.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.175+0000 7f710b7fe700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c102590 0x7f710c19f430 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f710400ada0 tx=0x7f710400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:58.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.175+0000 7f70f3fff700 1 -- 172.21.15.150:0/777765537 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710400f070 con 0x7f710c102590 2024-05-10T04:02:58.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.175+0000 7f70f3fff700 1 -- 172.21.15.150:0/777765537 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7104007b30 con 0x7f710c102590 2024-05-10T04:02:58.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.175+0000 7f7111449700 1 -- 172.21.15.150:0/777765537 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710c19fff0 con 0x7f710c102590 2024-05-10T04:02:58.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.175+0000 7f70f3fff700 1 -- 172.21.15.150:0/777765537 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104005440 con 0x7f710c102590 2024-05-10T04:02:58.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.175+0000 7f7111449700 1 -- 172.21.15.150:0/777765537 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710c1a0450 con 0x7f710c102590 2024-05-10T04:02:58.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.176+0000 7f7111449700 1 -- 172.21.15.150:0/777765537 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710c068730 con 0x7f710c102590 2024-05-10T04:02:58.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.177+0000 7f70f3fff700 1 -- 172.21.15.150:0/777765537 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7104004ad0 con 0x7f710c102590 2024-05-10T04:02:58.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.178+0000 7f70f3fff700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f70f40a11c0 0x7f70f40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:58.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.178+0000 7f710affd700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f70f40a11c0 0x7f70f40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:58.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.178+0000 7f70f3fff700 1 -- 172.21.15.150:0/777765537 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f71040cc710 con 0x7f710c102590 2024-05-10T04:02:58.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.178+0000 7f710affd700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f70f40a11c0 0x7f70f40a36d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f70fc005950 tx=0x7f70fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:58.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.182+0000 7f70f3fff700 1 -- 172.21.15.150:0/777765537 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7104095d40 con 0x7f710c102590 2024-05-10T04:02:58.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.416+0000 7f7111449700 1 -- 172.21.15.150:0/777765537 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f710c0fe150 con 0x7f70f40a11c0 2024-05-10T04:02:58.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.420+0000 7f70f3fff700 1 -- 172.21.15.150:0/777765537 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f710c0fe150 con 0x7f70f40a11c0 2024-05-10T04:02:58.423 INFO:teuthology.orchestra.run.smithi150.stdout:/volumes/qa/sv_0/dd4c9998-62a9-40e3-b309-3ae620ce3a51 2024-05-10T04:02:58.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.422+0000 7f7111449700 1 -- 172.21.15.150:0/777765537 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f70f40a11c0 msgr2=0x7f70f40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:58.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.422+0000 7f7111449700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f70f40a11c0 0x7f70f40a36d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f70fc005950 tx=0x7f70fc0058e0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.422+0000 7f7111449700 1 -- 172.21.15.150:0/777765537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c102590 msgr2=0x7f710c19f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:58.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.422+0000 7f7111449700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c102590 0x7f710c19f430 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f710400ada0 tx=0x7f710400c330 comp rx=0 tx=0).stop 2024-05-10T04:02:58.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.423+0000 7f7111449700 1 -- 172.21.15.150:0/777765537 shutdown_connections 2024-05-10T04:02:58.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.423+0000 7f7111449700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f70f40a11c0 0x7f70f40a36d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.423+0000 7f7111449700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c102590 0x7f710c19f430 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.423+0000 7f7111449700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f710c100ad0 0x7f710c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.423+0000 7f7111449700 1 --2- 172.21.15.150:0/777765537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710c101bf0 0x7f710c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.423+0000 7f7111449700 1 -- 172.21.15.150:0/777765537 >> 172.21.15.150:0/777765537 conn(0x7f710c0fc0e0 msgr2=0x7f710c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:58.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.423+0000 7f7111449700 1 -- 172.21.15.150:0/777765537 shutdown_connections 2024-05-10T04:02:58.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.423+0000 7f7111449700 1 -- 172.21.15.150:0/777765537 wait complete. 2024-05-10T04:02:58.437 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:58 smithi150 ceph-mon[42477]: pgmap v239: 129 pgs: 129 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-05-10T04:02:58.437 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:58 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] up:active 2024-05-10T04:02:58.437 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:58 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2622052560' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T04:02:58.437 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:58 smithi150 ceph-mon[42477]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-10T04:02:58.437 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:58 smithi150 ceph-mon[42477]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T04:02:58.438 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-10T04:02:58.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:58 smithi181 ceph-mon[46602]: pgmap v239: 129 pgs: 129 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-05-10T04:02:58.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:58 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] up:active 2024-05-10T04:02:58.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:58 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2622052560' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T04:02:58.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:58 smithi181 ceph-mon[46602]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-10T04:02:58.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:58 smithi181 ceph-mon[46602]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T04:02:58.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:58 smithi161 ceph-mon[46918]: pgmap v239: 129 pgs: 129 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-05-10T04:02:58.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:58 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] up:active 2024-05-10T04:02:58.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:58 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2622052560' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T04:02:58.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:58 smithi161 ceph-mon[46918]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-10T04:02:58.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:58 smithi161 ceph-mon[46918]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T04:02:58.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.575+0000 7fb04447b700 1 -- 172.21.15.150:0/2031892625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb03c107990 msgr2=0x7fb03c109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:58.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.575+0000 7fb04447b700 1 --2- 172.21.15.150:0/2031892625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb03c107990 0x7fb03c109db0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb03400b0a0 tx=0x7fb03400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.575+0000 7fb04447b700 1 -- 172.21.15.150:0/2031892625 shutdown_connections 2024-05-10T04:02:58.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.575+0000 7fb04447b700 1 --2- 172.21.15.150:0/2031892625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb03c107990 0x7fb03c109db0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.575+0000 7fb04447b700 1 --2- 172.21.15.150:0/2031892625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c1026d0 0x7fb03c104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.575+0000 7fb04447b700 1 --2- 172.21.15.150:0/2031892625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb03c105030 0x7fb03c107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.575+0000 7fb04447b700 1 -- 172.21.15.150:0/2031892625 >> 172.21.15.150:0/2031892625 conn(0x7fb03c0fc0f0 msgr2=0x7fb03c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:58.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.575+0000 7fb04447b700 1 -- 172.21.15.150:0/2031892625 shutdown_connections 2024-05-10T04:02:58.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.575+0000 7fb04447b700 1 -- 172.21.15.150:0/2031892625 wait complete. 2024-05-10T04:02:58.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb04447b700 1 Processor -- start 2024-05-10T04:02:58.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb04447b700 1 -- start start 2024-05-10T04:02:58.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb04447b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb03c1026d0 0x7fb03c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:58.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb04447b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c105030 0x7fb03c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:58.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb04447b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb03c107990 0x7fb03c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:58.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb04447b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c19fb90 con 0x7fb03c1026d0 2024-05-10T04:02:58.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb04447b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c19fd00 con 0x7fb03c107990 2024-05-10T04:02:58.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb04447b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c19fec0 con 0x7fb03c105030 2024-05-10T04:02:58.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb042217700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb03c1026d0 0x7fb03c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:58.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb042217700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb03c1026d0 0x7fb03c1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60776/0 (socket says 172.21.15.150:60776) 2024-05-10T04:02:58.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb042217700 1 -- 172.21.15.150:0/3324094586 learned_addr learned my addr 172.21.15.150:0/3324094586 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:58.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb041a16700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c105030 0x7fb03c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:58.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb042a18700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb03c107990 0x7fb03c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:58.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb042217700 1 -- 172.21.15.150:0/3324094586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c105030 msgr2=0x7fb03c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:58.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb042217700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c105030 0x7fb03c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb042217700 1 -- 172.21.15.150:0/3324094586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb03c107990 msgr2=0x7fb03c19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:58.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.576+0000 7fb042217700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb03c107990 0x7fb03c19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.577+0000 7fb042217700 1 -- 172.21.15.150:0/3324094586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb034009d00 con 0x7fb03c1026d0 2024-05-10T04:02:58.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.577+0000 7fb042217700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb03c1026d0 0x7fb03c1980b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e970 tx=0x7fb02c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:58.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.577+0000 7fb0337fe700 1 -- 172.21.15.150:0/3324094586 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c009790 con 0x7fb03c1026d0 2024-05-10T04:02:58.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.577+0000 7fb0337fe700 1 -- 172.21.15.150:0/3324094586 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb02c00c7e0 con 0x7fb03c1026d0 2024-05-10T04:02:58.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.577+0000 7fb0337fe700 1 -- 172.21.15.150:0/3324094586 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c0054e0 con 0x7fb03c1026d0 2024-05-10T04:02:58.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.577+0000 7fb04447b700 1 -- 172.21.15.150:0/3324094586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c1a01a0 con 0x7fb03c1026d0 2024-05-10T04:02:58.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.577+0000 7fb04447b700 1 -- 172.21.15.150:0/3324094586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c1a06b0 con 0x7fb03c1026d0 2024-05-10T04:02:58.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.578+0000 7fb04447b700 1 -- 172.21.15.150:0/3324094586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c0fdca0 con 0x7fb03c1026d0 2024-05-10T04:02:58.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.579+0000 7fb0337fe700 1 -- 172.21.15.150:0/3324094586 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb02c00c950 con 0x7fb03c1026d0 2024-05-10T04:02:58.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.580+0000 7fb0337fe700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0280a1240 0x7fb0280a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:58.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.580+0000 7fb0337fe700 1 -- 172.21.15.150:0/3324094586 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb02c014070 con 0x7fb03c1026d0 2024-05-10T04:02:58.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.580+0000 7fb041a16700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0280a1240 0x7fb0280a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:58.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.580+0000 7fb041a16700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0280a1240 0x7fb0280a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb03c199150 tx=0x7fb038009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:58.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.583+0000 7fb0337fe700 1 -- 172.21.15.150:0/3324094586 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb02c0964b0 con 0x7fb03c1026d0 2024-05-10T04:02:58.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.816+0000 7fb04447b700 1 -- 172.21.15.150:0/3324094586 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fb03c02ce90 con 0x7fb0280a1240 2024-05-10T04:02:58.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.840+0000 7fb0337fe700 1 -- 172.21.15.150:0/3324094586 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb03c02ce90 con 0x7fb0280a1240 2024-05-10T04:02:58.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.842+0000 7fb04447b700 1 -- 172.21.15.150:0/3324094586 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0280a1240 msgr2=0x7fb0280a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:58.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.842+0000 7fb04447b700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0280a1240 0x7fb0280a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb03c199150 tx=0x7fb038009260 comp rx=0 tx=0).stop 2024-05-10T04:02:58.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.842+0000 7fb04447b700 1 -- 172.21.15.150:0/3324094586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb03c1026d0 msgr2=0x7fb03c1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:58.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.842+0000 7fb04447b700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb03c1026d0 0x7fb03c1980b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e970 tx=0x7fb02c00ec80 comp rx=0 tx=0).stop 2024-05-10T04:02:58.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.842+0000 7fb04447b700 1 -- 172.21.15.150:0/3324094586 shutdown_connections 2024-05-10T04:02:58.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.842+0000 7fb04447b700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0280a1240 0x7fb0280a3750 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.842+0000 7fb04447b700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb03c107990 0x7fb03c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.842+0000 7fb04447b700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb03c1026d0 0x7fb03c1980b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.842+0000 7fb04447b700 1 --2- 172.21.15.150:0/3324094586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c105030 0x7fb03c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.842+0000 7fb04447b700 1 -- 172.21.15.150:0/3324094586 >> 172.21.15.150:0/3324094586 conn(0x7fb03c0fc0f0 msgr2=0x7fb03c103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:58.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.843+0000 7fb04447b700 1 -- 172.21.15.150:0/3324094586 shutdown_connections 2024-05-10T04:02:58.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.843+0000 7fb04447b700 1 -- 172.21.15.150:0/3324094586 wait complete. 2024-05-10T04:02:58.857 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-10T04:02:58.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.993+0000 7fa780fae700 1 -- 172.21.15.150:0/3247301898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa77c101c00 msgr2=0x7fa77c102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:58.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.993+0000 7fa780fae700 1 --2- 172.21.15.150:0/3247301898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa77c101c00 0x7fa77c102010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa76c009990 tx=0x7fa76c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.993+0000 7fa780fae700 1 -- 172.21.15.150:0/3247301898 shutdown_connections 2024-05-10T04:02:58.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.993+0000 7fa780fae700 1 --2- 172.21.15.150:0/3247301898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa77c1025a0 0x7fa77c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.993+0000 7fa780fae700 1 --2- 172.21.15.150:0/3247301898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77c100ae0 0x7fa77c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.993+0000 7fa780fae700 1 --2- 172.21.15.150:0/3247301898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa77c101c00 0x7fa77c102010 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:58.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.993+0000 7fa780fae700 1 -- 172.21.15.150:0/3247301898 >> 172.21.15.150:0/3247301898 conn(0x7fa77c0fc0f0 msgr2=0x7fa77c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:58.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.993+0000 7fa780fae700 1 -- 172.21.15.150:0/3247301898 shutdown_connections 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.993+0000 7fa780fae700 1 -- 172.21.15.150:0/3247301898 wait complete. 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.994+0000 7fa780fae700 1 Processor -- start 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.994+0000 7fa780fae700 1 -- start start 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.994+0000 7fa780fae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa77c100ae0 0x7fa77c06a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.994+0000 7fa780fae700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa77c101c00 0x7fa77c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.994+0000 7fa780fae700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77c1025a0 0x7fa77c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.994+0000 7fa780fae700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c19f9c0 con 0x7fa77c101c00 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.994+0000 7fa780fae700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c19fb30 con 0x7fa77c1025a0 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.994+0000 7fa780fae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c19fcf0 con 0x7fa77c100ae0 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.994+0000 7fa779d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa77c101c00 0x7fa77c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.994+0000 7fa779d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa77c101c00 0x7fa77c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60782/0 (socket says 172.21.15.150:60782) 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.994+0000 7fa779d9b700 1 -- 172.21.15.150:0/2844038043 learned_addr learned my addr 172.21.15.150:0/2844038043 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa77a59c700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa77c100ae0 0x7fa77c06a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa77ad9d700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77c1025a0 0x7fa77c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa779d9b700 1 -- 172.21.15.150:0/2844038043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa77c100ae0 msgr2=0x7fa77c06a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa779d9b700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa77c100ae0 0x7fa77c06a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa779d9b700 1 -- 172.21.15.150:0/2844038043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77c1025a0 msgr2=0x7fa77c19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa779d9b700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77c1025a0 0x7fa77c19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa779d9b700 1 -- 172.21.15.150:0/2844038043 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c0095f0 con 0x7fa77c101c00 2024-05-10T04:02:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa77ad9d700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77c1025a0 0x7fa77c19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa779d9b700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa77c101c00 0x7fa77c19ccb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa76c000c00 tx=0x7fa76c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa7737fe700 1 -- 172.21.15.150:0/2844038043 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c01d070 con 0x7fa77c101c00 2024-05-10T04:02:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa7737fe700 1 -- 172.21.15.150:0/2844038043 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa76c00fc20 con 0x7fa77c101c00 2024-05-10T04:02:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa780fae700 1 -- 172.21.15.150:0/2844038043 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77c19ff70 con 0x7fa77c101c00 2024-05-10T04:02:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa7737fe700 1 -- 172.21.15.150:0/2844038043 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c017650 con 0x7fa77c101c00 2024-05-10T04:02:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.995+0000 7fa780fae700 1 -- 172.21.15.150:0/2844038043 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77c1a0330 con 0x7fa77c101c00 2024-05-10T04:02:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.996+0000 7fa780fae700 1 -- 172.21.15.150:0/2844038043 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77c107cf0 con 0x7fa77c101c00 2024-05-10T04:02:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.998+0000 7fa7737fe700 1 -- 172.21.15.150:0/2844038043 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa76c022470 con 0x7fa77c101c00 2024-05-10T04:02:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.999+0000 7fa7737fe700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa7680a1240 0x7fa7680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.999+0000 7fa77a59c700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa7680a1240 0x7fa7680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.999+0000 7fa7737fe700 1 -- 172.21.15.150:0/2844038043 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa76c0ce600 con 0x7fa77c101c00 2024-05-10T04:02:59.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:58.999+0000 7fa77a59c700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa7680a1240 0x7fa7680a3750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa77c101b00 tx=0x7fa76400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.003+0000 7fa7737fe700 1 -- 172.21.15.150:0/2844038043 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa76c097cf0 con 0x7fa77c101c00 2024-05-10T04:02:59.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.199+0000 7fa7737fe700 1 -- 172.21.15.150:0/2844038043 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa76c0980c0 con 0x7fa77c101c00 2024-05-10T04:02:59.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.247+0000 7fa780fae700 1 -- 172.21.15.150:0/2844038043 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa77c02ce90 con 0x7fa7680a1240 2024-05-10T04:02:59.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.251+0000 7fa7737fe700 1 -- 172.21.15.150:0/2844038043 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fa77c02ce90 con 0x7fa7680a1240 2024-05-10T04:02:59.254 INFO:teuthology.orchestra.run.smithi150.stdout:/volumes/qa/sv_1/c6d4350e-b7bb-4db1-9eac-321589f390c3 2024-05-10T04:02:59.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.253+0000 7fa780fae700 1 -- 172.21.15.150:0/2844038043 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa7680a1240 msgr2=0x7fa7680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:59.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.253+0000 7fa780fae700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa7680a1240 0x7fa7680a3750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa77c101b00 tx=0x7fa76400a380 comp rx=0 tx=0).stop 2024-05-10T04:02:59.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.253+0000 7fa780fae700 1 -- 172.21.15.150:0/2844038043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa77c101c00 msgr2=0x7fa77c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:59.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.253+0000 7fa780fae700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa77c101c00 0x7fa77c19ccb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa76c000c00 tx=0x7fa76c00f670 comp rx=0 tx=0).stop 2024-05-10T04:02:59.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.253+0000 7fa780fae700 1 -- 172.21.15.150:0/2844038043 shutdown_connections 2024-05-10T04:02:59.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.253+0000 7fa780fae700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa7680a1240 0x7fa7680a3750 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.253+0000 7fa780fae700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77c1025a0 0x7fa77c19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.253+0000 7fa780fae700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa77c100ae0 0x7fa77c06a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.253+0000 7fa780fae700 1 --2- 172.21.15.150:0/2844038043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa77c101c00 0x7fa77c19ccb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.253+0000 7fa780fae700 1 -- 172.21.15.150:0/2844038043 >> 172.21.15.150:0/2844038043 conn(0x7fa77c0fc0f0 msgr2=0x7fa77c104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:59.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.254+0000 7fa780fae700 1 -- 172.21.15.150:0/2844038043 shutdown_connections 2024-05-10T04:02:59.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.254+0000 7fa780fae700 1 -- 172.21.15.150:0/2844038043 wait complete. 2024-05-10T04:02:59.269 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:02:59.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.400+0000 7fae41be2700 1 -- 172.21.15.150:0/1591967830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3c102580 msgr2=0x7fae3c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:59.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.400+0000 7fae41be2700 1 --2- 172.21.15.150:0/1591967830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3c102580 0x7fae3c108e30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae3400b0a0 tx=0x7fae3400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.400+0000 7fae41be2700 1 -- 172.21.15.150:0/1591967830 shutdown_connections 2024-05-10T04:02:59.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.400+0000 7fae41be2700 1 --2- 172.21.15.150:0/1591967830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3c102580 0x7fae3c108e30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.400+0000 7fae41be2700 1 --2- 172.21.15.150:0/1591967830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae3c100ac0 0x7fae3c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.400+0000 7fae41be2700 1 --2- 172.21.15.150:0/1591967830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae3c101be0 0x7fae3c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.400+0000 7fae41be2700 1 -- 172.21.15.150:0/1591967830 >> 172.21.15.150:0/1591967830 conn(0x7fae3c0fc0d0 msgr2=0x7fae3c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:59.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.400+0000 7fae41be2700 1 -- 172.21.15.150:0/1591967830 shutdown_connections 2024-05-10T04:02:59.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.400+0000 7fae41be2700 1 -- 172.21.15.150:0/1591967830 wait complete. 2024-05-10T04:02:59.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.401+0000 7fae41be2700 1 Processor -- start 2024-05-10T04:02:59.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.401+0000 7fae41be2700 1 -- start start 2024-05-10T04:02:59.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.401+0000 7fae41be2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae3c100ac0 0x7fae3c196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:59.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.401+0000 7fae41be2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae3c101be0 0x7fae3c1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:59.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.401+0000 7fae41be2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3c102580 0x7fae3c19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:59.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.401+0000 7fae41be2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c19ea10 con 0x7fae3c102580 2024-05-10T04:02:59.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.401+0000 7fae41be2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c19eb50 con 0x7fae3c101be0 2024-05-10T04:02:59.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.401+0000 7fae41be2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c19ece0 con 0x7fae3c100ac0 2024-05-10T04:02:59.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.401+0000 7fae3bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3c102580 0x7fae3c19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.401+0000 7fae3bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3c102580 0x7fae3c19e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57598/0 (socket says 172.21.15.150:57598) 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.401+0000 7fae3bfff700 1 -- 172.21.15.150:0/1765318820 learned_addr learned my addr 172.21.15.150:0/1765318820 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.401+0000 7fae3b7fe700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae3c100ac0 0x7fae3c196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.401+0000 7fae3affd700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae3c101be0 0x7fae3c1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.402+0000 7fae3bfff700 1 -- 172.21.15.150:0/1765318820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae3c100ac0 msgr2=0x7fae3c196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.402+0000 7fae3bfff700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae3c100ac0 0x7fae3c196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.402+0000 7fae3bfff700 1 -- 172.21.15.150:0/1765318820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae3c101be0 msgr2=0x7fae3c1974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.402+0000 7fae3bfff700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae3c101be0 0x7fae3c1974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.402+0000 7fae3bfff700 1 -- 172.21.15.150:0/1765318820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae34009d00 con 0x7fae3c102580 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.402+0000 7fae3affd700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae3c101be0 0x7fae3c1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.402+0000 7fae3bfff700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3c102580 0x7fae3c19e440 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fae34009040 tx=0x7fae340079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.402+0000 7fae38ff9700 1 -- 172.21.15.150:0/1765318820 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3400e040 con 0x7fae3c102580 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.402+0000 7fae38ff9700 1 -- 172.21.15.150:0/1765318820 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae34003c00 con 0x7fae3c102580 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.402+0000 7fae41be2700 1 -- 172.21.15.150:0/1765318820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c19ef60 con 0x7fae3c102580 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.402+0000 7fae38ff9700 1 -- 172.21.15.150:0/1765318820 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3401b6c0 con 0x7fae3c102580 2024-05-10T04:02:59.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.402+0000 7fae41be2700 1 -- 172.21.15.150:0/1765318820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c19f320 con 0x7fae3c102580 2024-05-10T04:02:59.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.403+0000 7fae41be2700 1 -- 172.21.15.150:0/1765318820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c107cd0 con 0x7fae3c102580 2024-05-10T04:02:59.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.405+0000 7fae38ff9700 1 -- 172.21.15.150:0/1765318820 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fae3401d470 con 0x7fae3c102580 2024-05-10T04:02:59.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.406+0000 7fae38ff9700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fae240a1230 0x7fae240a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:59.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.406+0000 7fae3b7fe700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fae240a1230 0x7fae240a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:59.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.406+0000 7fae38ff9700 1 -- 172.21.15.150:0/1765318820 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fae340cdce0 con 0x7fae3c102580 2024-05-10T04:02:59.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.406+0000 7fae3b7fe700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fae240a1230 0x7fae240a3740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae3c101ae0 tx=0x7fae30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:59.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.410+0000 7fae38ff9700 1 -- 172.21.15.150:0/1765318820 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fae340971d0 con 0x7fae3c102580 2024-05-10T04:02:59.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:59 smithi150 ceph-mon[42477]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:02:59.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:02:59 smithi150 ceph-mon[42477]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:02:59.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:59 smithi181 ceph-mon[46602]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:02:59.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:02:59 smithi181 ceph-mon[46602]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:02:59.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:59 smithi161 ceph-mon[46918]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:02:59.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:02:59 smithi161 ceph-mon[46918]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:02:59.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.658+0000 7fae41be2700 1 -- 172.21.15.150:0/1765318820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae3c0694d0 con 0x7fae3c102580 2024-05-10T04:02:59.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.659+0000 7fae38ff9700 1 -- 172.21.15.150:0/1765318820 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10886 (secure 0 0 0) 0x7fae3409a7f0 con 0x7fae3c102580 2024-05-10T04:02:59.662 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:02:59.662 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:02:57.183845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:02:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.661+0000 7fae41be2700 1 -- 172.21.15.150:0/1765318820 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fae240a1230 msgr2=0x7fae240a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.661+0000 7fae41be2700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fae240a1230 0x7fae240a3740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae3c101ae0 tx=0x7fae30009260 comp rx=0 tx=0).stop 2024-05-10T04:02:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.661+0000 7fae41be2700 1 -- 172.21.15.150:0/1765318820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3c102580 msgr2=0x7fae3c19e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.661+0000 7fae41be2700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3c102580 0x7fae3c19e440 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fae34009040 tx=0x7fae340079b0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.661+0000 7fae41be2700 1 -- 172.21.15.150:0/1765318820 shutdown_connections 2024-05-10T04:02:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.661+0000 7fae41be2700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fae240a1230 0x7fae240a3740 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.661+0000 7fae41be2700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3c102580 0x7fae3c19e440 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.661+0000 7fae41be2700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae3c100ac0 0x7fae3c196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.661+0000 7fae41be2700 1 --2- 172.21.15.150:0/1765318820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae3c101be0 0x7fae3c1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.661+0000 7fae41be2700 1 -- 172.21.15.150:0/1765318820 >> 172.21.15.150:0/1765318820 conn(0x7fae3c0fc0d0 msgr2=0x7fae3c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.662+0000 7fae41be2700 1 -- 172.21.15.150:0/1765318820 shutdown_connections 2024-05-10T04:02:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.662+0000 7fae41be2700 1 -- 172.21.15.150:0/1765318820 wait complete. 2024-05-10T04:02:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 24 2024-05-10T04:02:59.677 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:02:59.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.807+0000 7f5d7759e700 1 -- 172.21.15.150:0/3164990953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d78101b40 msgr2=0x7f5d781083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:59.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.807+0000 7f5d7759e700 1 --2- 172.21.15.150:0/3164990953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d78101b40 0x7f5d781083f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000c210 tx=0x7f5d7000c520 comp rx=0 tx=0).stop 2024-05-10T04:02:59.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.807+0000 7f5d7759e700 1 -- 172.21.15.150:0/3164990953 shutdown_connections 2024-05-10T04:02:59.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.807+0000 7f5d7759e700 1 --2- 172.21.15.150:0/3164990953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d78101b40 0x7f5d781083f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.807+0000 7f5d7759e700 1 --2- 172.21.15.150:0/3164990953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d78100080 0x7f5d78100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.807+0000 7f5d7759e700 1 --2- 172.21.15.150:0/3164990953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d781011a0 0x7f5d781015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.807+0000 7f5d7759e700 1 -- 172.21.15.150:0/3164990953 >> 172.21.15.150:0/3164990953 conn(0x7f5d780fb690 msgr2=0x7f5d780fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:02:59.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.807+0000 7f5d7759e700 1 -- 172.21.15.150:0/3164990953 shutdown_connections 2024-05-10T04:02:59.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.807+0000 7f5d7759e700 1 -- 172.21.15.150:0/3164990953 wait complete. 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.808+0000 7f5d7759e700 1 Processor -- start 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.808+0000 7f5d7759e700 1 -- start start 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.808+0000 7f5d7759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d78100080 0x7f5d78103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.808+0000 7f5d7759e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d781011a0 0x7f5d781024e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.808+0000 7f5d7759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d78101b40 0x7f5d78102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.808+0000 7f5d7759e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78103080 con 0x7f5d781011a0 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.808+0000 7f5d7759e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d781031f0 con 0x7f5d78100080 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.808+0000 7f5d7759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d781033b0 con 0x7f5d78101b40 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.808+0000 7f5d75d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d781011a0 0x7f5d781024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.808+0000 7f5d75d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d781011a0 0x7f5d781024e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57616/0 (socket says 172.21.15.150:57616) 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.808+0000 7f5d75d9b700 1 -- 172.21.15.150:0/1639623044 learned_addr learned my addr 172.21.15.150:0/1639623044 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.809+0000 7f5d76d9d700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d78101b40 0x7f5d78102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.809+0000 7f5d7659c700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d78100080 0x7f5d78103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:59.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.809+0000 7f5d75d9b700 1 -- 172.21.15.150:0/1639623044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d78101b40 msgr2=0x7f5d78102a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:59.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.809+0000 7f5d75d9b700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d78101b40 0x7f5d78102a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.809+0000 7f5d75d9b700 1 -- 172.21.15.150:0/1639623044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d78100080 msgr2=0x7f5d78103e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:02:59.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.809+0000 7f5d75d9b700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d78100080 0x7f5d78103e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:02:59.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.809+0000 7f5d75d9b700 1 -- 172.21.15.150:0/1639623044 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7000ae10 con 0x7f5d781011a0 2024-05-10T04:02:59.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.809+0000 7f5d75d9b700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d781011a0 0x7f5d781024e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800d980 tx=0x7f5d6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:59.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.809+0000 7f5d677fe700 1 -- 172.21.15.150:0/1639623044 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68004220 con 0x7f5d781011a0 2024-05-10T04:02:59.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.809+0000 7f5d677fe700 1 -- 172.21.15.150:0/1639623044 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d68007640 con 0x7f5d781011a0 2024-05-10T04:02:59.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.809+0000 7f5d7759e700 1 -- 172.21.15.150:0/1639623044 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d781a2070 con 0x7f5d781011a0 2024-05-10T04:02:59.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.809+0000 7f5d677fe700 1 -- 172.21.15.150:0/1639623044 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68005610 con 0x7f5d781011a0 2024-05-10T04:02:59.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.809+0000 7f5d7759e700 1 -- 172.21.15.150:0/1639623044 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d781a2580 con 0x7f5d781011a0 2024-05-10T04:02:59.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.810+0000 7f5d7759e700 1 -- 172.21.15.150:0/1639623044 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d780676e0 con 0x7f5d781011a0 2024-05-10T04:02:59.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.811+0000 7f5d677fe700 1 -- 172.21.15.150:0/1639623044 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5d68004750 con 0x7f5d781011a0 2024-05-10T04:02:59.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.812+0000 7f5d677fe700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5d600a1260 0x7f5d600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:02:59.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.812+0000 7f5d677fe700 1 -- 172.21.15.150:0/1639623044 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5d680ccfb0 con 0x7f5d781011a0 2024-05-10T04:02:59.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.812+0000 7f5d7659c700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5d600a1260 0x7f5d600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:02:59.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.812+0000 7f5d7659c700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5d600a1260 0x7f5d600a3770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5d781010a0 tx=0x7f5d6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:02:59.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:02:59.816+0000 7f5d677fe700 1 -- 172.21.15.150:0/1639623044 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5d680963f0 con 0x7f5d781011a0 2024-05-10T04:03:00.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.012+0000 7f5d7759e700 1 -- 172.21.15.150:0/1639623044 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d7810b080 con 0x7f5d781011a0 2024-05-10T04:03:00.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.013+0000 7f5d677fe700 1 -- 172.21.15.150:0/1639623044 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24027 (secure 0 0 0) 0x7f5d68016090 con 0x7f5d781011a0 2024-05-10T04:03:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:00.016 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":85,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:02:36.063310+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:03:00.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.016+0000 7f5d7759e700 1 -- 172.21.15.150:0/1639623044 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5d600a1260 msgr2=0x7f5d600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:00.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.016+0000 7f5d7759e700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5d600a1260 0x7f5d600a3770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5d781010a0 tx=0x7f5d6c009260 comp rx=0 tx=0).stop 2024-05-10T04:03:00.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.016+0000 7f5d7759e700 1 -- 172.21.15.150:0/1639623044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d781011a0 msgr2=0x7f5d781024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:00.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.016+0000 7f5d7759e700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d781011a0 0x7f5d781024e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800d980 tx=0x7f5d6800dc90 comp rx=0 tx=0).stop 2024-05-10T04:03:00.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.016+0000 7f5d7759e700 1 -- 172.21.15.150:0/1639623044 shutdown_connections 2024-05-10T04:03:00.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.016+0000 7f5d7759e700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5d600a1260 0x7f5d600a3770 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.016+0000 7f5d7759e700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d78101b40 0x7f5d78102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.016+0000 7f5d7759e700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d78100080 0x7f5d78103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.016+0000 7f5d7759e700 1 --2- 172.21.15.150:0/1639623044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d781011a0 0x7f5d781024e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.016+0000 7f5d7759e700 1 -- 172.21.15.150:0/1639623044 >> 172.21.15.150:0/1639623044 conn(0x7f5d780fb690 msgr2=0x7f5d781065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:00.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.016+0000 7f5d7759e700 1 -- 172.21.15.150:0/1639623044 shutdown_connections 2024-05-10T04:03:00.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.016+0000 7f5d7759e700 1 -- 172.21.15.150:0/1639623044 wait complete. 2024-05-10T04:03:00.031 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-10T04:03:00.032 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T04:03:00.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.162+0000 7f4b1114c700 1 -- 172.21.15.150:0/1419212142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c100ad0 msgr2=0x7f4b0c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:00.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.162+0000 7f4b1114c700 1 --2- 172.21.15.150:0/1419212142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c100ad0 0x7f4b0c100ee0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009990 tx=0x7f4afc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.162+0000 7f4b1114c700 1 -- 172.21.15.150:0/1419212142 shutdown_connections 2024-05-10T04:03:00.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.162+0000 7f4b1114c700 1 --2- 172.21.15.150:0/1419212142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b0c102590 0x7f4b0c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.162+0000 7f4b1114c700 1 --2- 172.21.15.150:0/1419212142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c100ad0 0x7f4b0c100ee0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.162+0000 7f4b1114c700 1 --2- 172.21.15.150:0/1419212142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b0c101bf0 0x7f4b0c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.162+0000 7f4b1114c700 1 -- 172.21.15.150:0/1419212142 >> 172.21.15.150:0/1419212142 conn(0x7f4b0c0fc0c0 msgr2=0x7f4b0c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:00.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.162+0000 7f4b1114c700 1 -- 172.21.15.150:0/1419212142 shutdown_connections 2024-05-10T04:03:00.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.162+0000 7f4b1114c700 1 -- 172.21.15.150:0/1419212142 wait complete. 2024-05-10T04:03:00.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.162+0000 7f4b1114c700 1 Processor -- start 2024-05-10T04:03:00.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b1114c700 1 -- start start 2024-05-10T04:03:00.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b1114c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b0c100ad0 0x7f4b0c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:00.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b1114c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b0c101bf0 0x7f4b0c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:00.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b1114c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c102590 0x7f4b0c19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:00.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b1114c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c19fb70 con 0x7f4b0c102590 2024-05-10T04:03:00.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b1114c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c19fce0 con 0x7f4b0c101bf0 2024-05-10T04:03:00.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b1114c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c19fea0 con 0x7f4b0c100ad0 2024-05-10T04:03:00.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b0b59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c102590 0x7f4b0c19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:00.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b0b59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c102590 0x7f4b0c19f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57626/0 (socket says 172.21.15.150:57626) 2024-05-10T04:03:00.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b0b59e700 1 -- 172.21.15.150:0/3188870233 learned_addr learned my addr 172.21.15.150:0/3188870233 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:00.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b0ad9d700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b0c100ad0 0x7f4b0c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:00.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b0a59c700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b0c101bf0 0x7f4b0c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:00.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b0b59e700 1 -- 172.21.15.150:0/3188870233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b0c100ad0 msgr2=0x7f4b0c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:00.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b0b59e700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b0c100ad0 0x7f4b0c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b0b59e700 1 -- 172.21.15.150:0/3188870233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b0c101bf0 msgr2=0x7f4b0c1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:00.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b0b59e700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b0c101bf0 0x7f4b0c1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.163+0000 7f4b0b59e700 1 -- 172.21.15.150:0/3188870233 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc0095f0 con 0x7f4b0c102590 2024-05-10T04:03:00.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.164+0000 7f4b0b59e700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c102590 0x7f4b0c19f560 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4b04007ee0 tx=0x7f4b0400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:00.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.164+0000 7f4af3fff700 1 -- 172.21.15.150:0/3188870233 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04017070 con 0x7f4b0c102590 2024-05-10T04:03:00.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.164+0000 7f4b1114c700 1 -- 172.21.15.150:0/3188870233 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0c1a0180 con 0x7f4b0c102590 2024-05-10T04:03:00.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.164+0000 7f4b1114c700 1 -- 172.21.15.150:0/3188870233 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0c1a0690 con 0x7f4b0c102590 2024-05-10T04:03:00.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.164+0000 7f4af3fff700 1 -- 172.21.15.150:0/3188870233 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b0400f040 con 0x7f4b0c102590 2024-05-10T04:03:00.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.164+0000 7f4af3fff700 1 -- 172.21.15.150:0/3188870233 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04005530 con 0x7f4b0c102590 2024-05-10T04:03:00.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.165+0000 7f4b1114c700 1 -- 172.21.15.150:0/3188870233 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af80052f0 con 0x7f4b0c102590 2024-05-10T04:03:00.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.166+0000 7f4af3fff700 1 -- 172.21.15.150:0/3188870233 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4b04005690 con 0x7f4b0c102590 2024-05-10T04:03:00.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.166+0000 7f4af3fff700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4af40a11b0 0x7f4af40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:00.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.167+0000 7f4b0ad9d700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4af40a11b0 0x7f4af40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:00.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.167+0000 7f4af3fff700 1 -- 172.21.15.150:0/3188870233 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4b040cbfe0 con 0x7f4b0c102590 2024-05-10T04:03:00.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.167+0000 7f4b0ad9d700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4af40a11b0 0x7f4af40a36c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4afc006010 tx=0x7f4afc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:00.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.171+0000 7f4af3fff700 1 -- 172.21.15.150:0/3188870233 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4b040ffd60 con 0x7f4b0c102590 2024-05-10T04:03:00.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.363+0000 7f4b1114c700 1 -- 172.21.15.150:0/3188870233 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4af8000bc0 con 0x7f4af40a11b0 2024-05-10T04:03:00.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.376+0000 7f4af3fff700 1 -- 172.21.15.150:0/3188870233 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418643 (secure 0 0 0) 0x7f4af8000bc0 con 0x7f4af40a11b0 2024-05-10T04:03:00.385 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:00.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.380+0000 7f4b1114c700 1 -- 172.21.15.150:0/3188870233 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4af40a11b0 msgr2=0x7f4af40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:00.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.380+0000 7f4b1114c700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4af40a11b0 0x7f4af40a36c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4afc006010 tx=0x7f4afc01a040 comp rx=0 tx=0).stop 2024-05-10T04:03:00.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.381+0000 7f4b1114c700 1 -- 172.21.15.150:0/3188870233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c102590 msgr2=0x7f4b0c19f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:00.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.381+0000 7f4b1114c700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c102590 0x7f4b0c19f560 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4b04007ee0 tx=0x7f4b0400d3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.381+0000 7f4b1114c700 1 -- 172.21.15.150:0/3188870233 shutdown_connections 2024-05-10T04:03:00.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.381+0000 7f4b1114c700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4af40a11b0 0x7f4af40a36c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.381+0000 7f4b1114c700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b0c102590 0x7f4b0c19f560 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.381+0000 7f4b1114c700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b0c100ad0 0x7f4b0c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.381+0000 7f4b1114c700 1 --2- 172.21.15.150:0/3188870233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b0c101bf0 0x7f4b0c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.381+0000 7f4b1114c700 1 -- 172.21.15.150:0/3188870233 >> 172.21.15.150:0/3188870233 conn(0x7f4b0c0fc0c0 msgr2=0x7f4b0c104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:00.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.381+0000 7f4b1114c700 1 -- 172.21.15.150:0/3188870233 shutdown_connections 2024-05-10T04:03:00.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.381+0000 7f4b1114c700 1 -- 172.21.15.150:0/3188870233 wait complete. 2024-05-10T04:03:00.389 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:03:00.395 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:00 smithi150 ceph-mon[42477]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:00.395 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:00 smithi150 ceph-mon[42477]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:00.395 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:00 smithi150 ceph-mon[42477]: pgmap v240: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-05-10T04:03:00.395 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:00 smithi150 ceph-mon[42477]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-10T04:03:00.395 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:00 smithi150 ceph-mon[42477]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:00.395 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:00 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1765318820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:00.395 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:00 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1639623044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:00.396 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-05-10T04:02:59.085237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597891,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":142,"num_read_kb":123,"num_write":293,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372176,"kb_used_data":11104,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569648,"statfs":{"total":1151940427776,"available":1151559319552,"internally_reserved":0,"allocated":11370496,"data_stored":7283974,"data_compressed":246795,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3070,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":65,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003642"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640492+0000","last_change":"2024-05-10T04:02:31.642089+0000","last_active":"2024-05-10T04:02:35.640492+0000","last_peered":"2024-05-10T04:02:35.640492+0000","last_clean":"2024-05-10T04:02:35.640492+0000","last_became_active":"2024-05-10T04:02:31.641080+0000","last_became_peered":"2024-05-10T04:02:31.641080+0000","last_unstale":"2024-05-10T04:02:35.640492+0000","last_undegraded":"2024-05-10T04:02:35.640492+0000","last_fullsized":"2024-05-10T04:02:35.640492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:29.693139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636966+0000","last_change":"2024-05-10T04:02:31.633056+0000","last_active":"2024-05-10T04:02:35.636966+0000","last_peered":"2024-05-10T04:02:35.636966+0000","last_clean":"2024-05-10T04:02:35.636966+0000","last_became_active":"2024-05-10T04:02:31.632725+0000","last_became_peered":"2024-05-10T04:02:31.632725+0000","last_unstale":"2024-05-10T04:02:35.636966+0000","last_undegraded":"2024-05-10T04:02:35.636966+0000","last_fullsized":"2024-05-10T04:02:35.636966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:08:40.365433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636999+0000","last_change":"2024-05-10T04:02:31.625219+0000","last_active":"2024-05-10T04:02:35.636999+0000","last_peered":"2024-05-10T04:02:35.636999+0000","last_clean":"2024-05-10T04:02:35.636999+0000","last_became_active":"2024-05-10T04:02:31.624889+0000","last_became_peered":"2024-05-10T04:02:31.624889+0000","last_unstale":"2024-05-10T04:02:35.636999+0000","last_undegraded":"2024-05-10T04:02:35.636999+0000","last_fullsized":"2024-05-10T04:02:35.636999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:10:59.081031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635079+0000","last_change":"2024-05-10T04:02:31.640651+0000","last_active":"2024-05-10T04:02:35.635079+0000","last_peered":"2024-05-10T04:02:35.635079+0000","last_clean":"2024-05-10T04:02:35.635079+0000","last_became_active":"2024-05-10T04:02:31.640242+0000","last_became_peered":"2024-05-10T04:02:31.640242+0000","last_unstale":"2024-05-10T04:02:35.635079+0000","last_undegraded":"2024-05-10T04:02:35.635079+0000","last_fullsized":"2024-05-10T04:02:35.635079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:07.562099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635834+0000","last_change":"2024-05-10T04:02:31.630688+0000","last_active":"2024-05-10T04:02:35.635834+0000","last_peered":"2024-05-10T04:02:35.635834+0000","last_clean":"2024-05-10T04:02:35.635834+0000","last_became_active":"2024-05-10T04:02:31.629252+0000","last_became_peered":"2024-05-10T04:02:31.629252+0000","last_unstale":"2024-05-10T04:02:35.635834+0000","last_undegraded":"2024-05-10T04:02:35.635834+0000","last_fullsized":"2024-05-10T04:02:35.635834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:37.647516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.898651+0000","last_change":"2024-05-10T04:02:31.642586+0000","last_active":"2024-05-10T04:02:35.898651+0000","last_peered":"2024-05-10T04:02:35.898651+0000","last_clean":"2024-05-10T04:02:35.898651+0000","last_became_active":"2024-05-10T04:02:31.642244+0000","last_became_peered":"2024-05-10T04:02:31.642244+0000","last_unstale":"2024-05-10T04:02:35.898651+0000","last_undegraded":"2024-05-10T04:02:35.898651+0000","last_fullsized":"2024-05-10T04:02:35.898651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:28.020151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.638641+0000","last_change":"2024-05-10T04:02:31.636706+0000","last_active":"2024-05-10T04:02:35.638641+0000","last_peered":"2024-05-10T04:02:35.638641+0000","last_clean":"2024-05-10T04:02:35.638641+0000","last_became_active":"2024-05-10T04:02:31.636303+0000","last_became_peered":"2024-05-10T04:02:31.636303+0000","last_unstale":"2024-05-10T04:02:35.638641+0000","last_undegraded":"2024-05-10T04:02:35.638641+0000","last_fullsized":"2024-05-10T04:02:35.638641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:16.941400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956821+0000","last_change":"2024-05-10T04:02:31.640032+0000","last_active":"2024-05-10T04:02:35.956821+0000","last_peered":"2024-05-10T04:02:35.956821+0000","last_clean":"2024-05-10T04:02:35.956821+0000","last_became_active":"2024-05-10T04:02:31.639733+0000","last_became_peered":"2024-05-10T04:02:31.639733+0000","last_unstale":"2024-05-10T04:02:35.956821+0000","last_undegraded":"2024-05-10T04:02:35.956821+0000","last_fullsized":"2024-05-10T04:02:35.956821+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:02:36.963414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641239+0000","last_change":"2024-05-10T04:02:31.638940+0000","last_active":"2024-05-10T04:02:35.641239+0000","last_peered":"2024-05-10T04:02:35.641239+0000","last_clean":"2024-05-10T04:02:35.641239+0000","last_became_active":"2024-05-10T04:02:31.638617+0000","last_became_peered":"2024-05-10T04:02:31.638617+0000","last_unstale":"2024-05-10T04:02:35.641239+0000","last_undegraded":"2024-05-10T04:02:35.641239+0000","last_fullsized":"2024-05-10T04:02:35.641239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:10:36.348117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.899440+0000","last_change":"2024-05-10T04:02:31.635702+0000","last_active":"2024-05-10T04:02:35.899440+0000","last_peered":"2024-05-10T04:02:35.899440+0000","last_clean":"2024-05-10T04:02:35.899440+0000","last_became_active":"2024-05-10T04:02:31.635330+0000","last_became_peered":"2024-05-10T04:02:31.635330+0000","last_unstale":"2024-05-10T04:02:35.899440+0000","last_undegraded":"2024-05-10T04:02:35.899440+0000","last_fullsized":"2024-05-10T04:02:35.899440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:21:15.160340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641553+0000","last_change":"2024-05-10T04:02:31.628252+0000","last_active":"2024-05-10T04:02:35.641553+0000","last_peered":"2024-05-10T04:02:35.641553+0000","last_clean":"2024-05-10T04:02:35.641553+0000","last_became_active":"2024-05-10T04:02:31.627394+0000","last_became_peered":"2024-05-10T04:02:31.627394+0000","last_unstale":"2024-05-10T04:02:35.641553+0000","last_undegraded":"2024-05-10T04:02:35.641553+0000","last_fullsized":"2024-05-10T04:02:35.641553+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:54.569786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635041+0000","last_change":"2024-05-10T04:02:31.641788+0000","last_active":"2024-05-10T04:02:35.635041+0000","last_peered":"2024-05-10T04:02:35.635041+0000","last_clean":"2024-05-10T04:02:35.635041+0000","last_became_active":"2024-05-10T04:02:31.641328+0000","last_became_peered":"2024-05-10T04:02:31.641328+0000","last_unstale":"2024-05-10T04:02:35.635041+0000","last_undegraded":"2024-05-10T04:02:35.635041+0000","last_fullsized":"2024-05-10T04:02:35.635041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:10:55.920245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.631051+0000","last_change":"2024-05-10T04:02:31.631919+0000","last_active":"2024-05-10T04:02:35.631051+0000","last_peered":"2024-05-10T04:02:35.631051+0000","last_clean":"2024-05-10T04:02:35.631051+0000","last_became_active":"2024-05-10T04:02:31.631592+0000","last_became_peered":"2024-05-10T04:02:31.631592+0000","last_unstale":"2024-05-10T04:02:35.631051+0000","last_undegraded":"2024-05-10T04:02:35.631051+0000","last_fullsized":"2024-05-10T04:02:35.631051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:59:03.792090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636548+0000","last_change":"2024-05-10T04:02:31.633288+0000","last_active":"2024-05-10T04:02:35.636548+0000","last_peered":"2024-05-10T04:02:35.636548+0000","last_clean":"2024-05-10T04:02:35.636548+0000","last_became_active":"2024-05-10T04:02:31.632582+0000","last_became_peered":"2024-05-10T04:02:31.632582+0000","last_unstale":"2024-05-10T04:02:35.636548+0000","last_undegraded":"2024-05-10T04:02:35.636548+0000","last_fullsized":"2024-05-10T04:02:35.636548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:27.991439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636432+0000","last_change":"2024-05-10T04:02:31.635833+0000","last_active":"2024-05-10T04:02:35.636432+0000","last_peered":"2024-05-10T04:02:35.636432+0000","last_clean":"2024-05-10T04:02:35.636432+0000","last_became_active":"2024-05-10T04:02:31.635520+0000","last_became_peered":"2024-05-10T04:02:31.635520+0000","last_unstale":"2024-05-10T04:02:35.636432+0000","last_undegraded":"2024-05-10T04:02:35.636432+0000","last_fullsized":"2024-05-10T04:02:35.636432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:40:06.312819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.639301+0000","last_change":"2024-05-10T04:02:31.631734+0000","last_active":"2024-05-10T04:02:35.639301+0000","last_peered":"2024-05-10T04:02:35.639301+0000","last_clean":"2024-05-10T04:02:35.639301+0000","last_became_active":"2024-05-10T04:02:31.631375+0000","last_became_peered":"2024-05-10T04:02:31.631375+0000","last_unstale":"2024-05-10T04:02:35.639301+0000","last_undegraded":"2024-05-10T04:02:35.639301+0000","last_fullsized":"2024-05-10T04:02:35.639301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:25:13.815406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640254+0000","last_change":"2024-05-10T04:02:31.642292+0000","last_active":"2024-05-10T04:02:35.640254+0000","last_peered":"2024-05-10T04:02:35.640254+0000","last_clean":"2024-05-10T04:02:35.640254+0000","last_became_active":"2024-05-10T04:02:31.641570+0000","last_became_peered":"2024-05-10T04:02:31.641570+0000","last_unstale":"2024-05-10T04:02:35.640254+0000","last_undegraded":"2024-05-10T04:02:35.640254+0000","last_fullsized":"2024-05-10T04:02:35.640254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:16:33.704809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.646273+0000","last_change":"2024-05-10T04:02:31.625695+0000","last_active":"2024-05-10T04:02:35.646273+0000","last_peered":"2024-05-10T04:02:35.646273+0000","last_clean":"2024-05-10T04:02:35.646273+0000","last_became_active":"2024-05-10T04:02:31.625393+0000","last_became_peered":"2024-05-10T04:02:31.625393+0000","last_unstale":"2024-05-10T04:02:35.646273+0000","last_undegraded":"2024-05-10T04:02:35.646273+0000","last_fullsized":"2024-05-10T04:02:35.646273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:53.955921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.633141+0000","last_change":"2024-05-10T04:02:31.629152+0000","last_active":"2024-05-10T04:02:35.633141+0000","last_peered":"2024-05-10T04:02:35.633141+0000","last_clean":"2024-05-10T04:02:35.633141+0000","last_became_active":"2024-05-10T04:02:31.628778+0000","last_became_peered":"2024-05-10T04:02:31.628778+0000","last_unstale":"2024-05-10T04:02:35.633141+0000","last_undegraded":"2024-05-10T04:02:35.633141+0000","last_fullsized":"2024-05-10T04:02:35.633141+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:39.772500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.409997+0000","last_change":"2024-05-10T04:02:35.650536+0000","last_active":"2024-05-10T04:02:36.409997+0000","last_peered":"2024-05-10T04:02:36.409997+0000","last_clean":"2024-05-10T04:02:36.409997+0000","last_became_active":"2024-05-10T04:02:35.650102+0000","last_became_peered":"2024-05-10T04:02:35.650102+0000","last_unstale":"2024-05-10T04:02:36.409997+0000","last_undegraded":"2024-05-10T04:02:36.409997+0000","last_fullsized":"2024-05-10T04:02:36.409997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:19:35.605844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182721+0000","last_change":"2024-05-10T04:02:31.186846+0000","last_active":"2024-05-10T04:02:56.182721+0000","last_peered":"2024-05-10T04:02:56.182721+0000","last_clean":"2024-05-10T04:02:56.182721+0000","last_became_active":"2024-05-10T04:02:31.186497+0000","last_became_peered":"2024-05-10T04:02:31.186497+0000","last_unstale":"2024-05-10T04:02:56.182721+0000","last_undegraded":"2024-05-10T04:02:56.182721+0000","last_fullsized":"2024-05-10T04:02:56.182721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:15:40.832455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956915+0000","last_change":"2024-05-10T04:02:31.631278+0000","last_active":"2024-05-10T04:02:35.956915+0000","last_peered":"2024-05-10T04:02:35.956915+0000","last_clean":"2024-05-10T04:02:35.956915+0000","last_became_active":"2024-05-10T04:02:31.630979+0000","last_became_peered":"2024-05-10T04:02:31.630979+0000","last_unstale":"2024-05-10T04:02:35.956915+0000","last_undegraded":"2024-05-10T04:02:35.956915+0000","last_fullsized":"2024-05-10T04:02:35.956915+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:50.227115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641619+0000","last_change":"2024-05-10T04:02:35.651622+0000","last_active":"2024-05-10T04:02:36.641619+0000","last_peered":"2024-05-10T04:02:36.641619+0000","last_clean":"2024-05-10T04:02:36.641619+0000","last_became_active":"2024-05-10T04:02:35.651148+0000","last_became_peered":"2024-05-10T04:02:35.651148+0000","last_unstale":"2024-05-10T04:02:36.641619+0000","last_undegraded":"2024-05-10T04:02:36.641619+0000","last_fullsized":"2024-05-10T04:02:36.641619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:40:09.019286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"80'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.644547+0000","last_change":"2024-05-10T04:02:31.187785+0000","last_active":"2024-05-10T04:02:35.644547+0000","last_peered":"2024-05-10T04:02:35.644547+0000","last_clean":"2024-05-10T04:02:35.644547+0000","last_became_active":"2024-05-10T04:02:31.187499+0000","last_became_peered":"2024-05-10T04:02:31.187499+0000","last_unstale":"2024-05-10T04:02:35.644547+0000","last_undegraded":"2024-05-10T04:02:35.644547+0000","last_fullsized":"2024-05-10T04:02:35.644547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:05:29.015767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636966+0000","last_change":"2024-05-10T04:02:31.624433+0000","last_active":"2024-05-10T04:02:35.636966+0000","last_peered":"2024-05-10T04:02:35.636966+0000","last_clean":"2024-05-10T04:02:35.636966+0000","last_became_active":"2024-05-10T04:02:31.624115+0000","last_became_peered":"2024-05-10T04:02:31.624115+0000","last_unstale":"2024-05-10T04:02:35.636966+0000","last_undegraded":"2024-05-10T04:02:35.636966+0000","last_fullsized":"2024-05-10T04:02:35.636966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:54.584681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641726+0000","last_change":"2024-05-10T04:02:35.651535+0000","last_active":"2024-05-10T04:02:36.641726+0000","last_peered":"2024-05-10T04:02:36.641726+0000","last_clean":"2024-05-10T04:02:36.641726+0000","last_became_active":"2024-05-10T04:02:35.651077+0000","last_became_peered":"2024-05-10T04:02:35.651077+0000","last_unstale":"2024-05-10T04:02:36.641726+0000","last_undegraded":"2024-05-10T04:02:36.641726+0000","last_fullsized":"2024-05-10T04:02:36.641726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:21:04.647730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.180078+0000","last_change":"2024-05-10T04:02:31.195666+0000","last_active":"2024-05-10T04:02:56.180078+0000","last_peered":"2024-05-10T04:02:56.180078+0000","last_clean":"2024-05-10T04:02:56.180078+0000","last_became_active":"2024-05-10T04:02:31.195285+0000","last_became_peered":"2024-05-10T04:02:31.195285+0000","last_unstale":"2024-05-10T04:02:56.180078+0000","last_undegraded":"2024-05-10T04:02:56.180078+0000","last_fullsized":"2024-05-10T04:02:56.180078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:02.906701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640666+0000","last_change":"2024-05-10T04:02:31.628124+0000","last_active":"2024-05-10T04:02:35.640666+0000","last_peered":"2024-05-10T04:02:35.640666+0000","last_clean":"2024-05-10T04:02:35.640666+0000","last_became_active":"2024-05-10T04:02:31.627789+0000","last_became_peered":"2024-05-10T04:02:31.627789+0000","last_unstale":"2024-05-10T04:02:35.640666+0000","last_undegraded":"2024-05-10T04:02:35.640666+0000","last_fullsized":"2024-05-10T04:02:35.640666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:40:11.240726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641791+0000","last_change":"2024-05-10T04:02:35.649536+0000","last_active":"2024-05-10T04:02:36.641791+0000","last_peered":"2024-05-10T04:02:36.641791+0000","last_clean":"2024-05-10T04:02:36.641791+0000","last_became_active":"2024-05-10T04:02:35.649130+0000","last_became_peered":"2024-05-10T04:02:35.649130+0000","last_unstale":"2024-05-10T04:02:36.641791+0000","last_undegraded":"2024-05-10T04:02:36.641791+0000","last_fullsized":"2024-05-10T04:02:36.641791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:03.870071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637193+0000","last_change":"2024-05-10T04:02:31.192627+0000","last_active":"2024-05-10T04:02:35.637193+0000","last_peered":"2024-05-10T04:02:35.637193+0000","last_clean":"2024-05-10T04:02:35.637193+0000","last_became_active":"2024-05-10T04:02:31.191433+0000","last_became_peered":"2024-05-10T04:02:31.191433+0000","last_unstale":"2024-05-10T04:02:35.637193+0000","last_undegraded":"2024-05-10T04:02:35.637193+0000","last_fullsized":"2024-05-10T04:02:35.637193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:29.235483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641519+0000","last_change":"2024-05-10T04:02:31.641192+0000","last_active":"2024-05-10T04:02:35.641519+0000","last_peered":"2024-05-10T04:02:35.641519+0000","last_clean":"2024-05-10T04:02:35.641519+0000","last_became_active":"2024-05-10T04:02:31.640540+0000","last_became_peered":"2024-05-10T04:02:31.640540+0000","last_unstale":"2024-05-10T04:02:35.641519+0000","last_undegraded":"2024-05-10T04:02:35.641519+0000","last_fullsized":"2024-05-10T04:02:35.641519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:36.734231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.410181+0000","last_change":"2024-05-10T04:02:35.644661+0000","last_active":"2024-05-10T04:02:36.410181+0000","last_peered":"2024-05-10T04:02:36.410181+0000","last_clean":"2024-05-10T04:02:36.410181+0000","last_became_active":"2024-05-10T04:02:35.644216+0000","last_became_peered":"2024-05-10T04:02:35.644216+0000","last_unstale":"2024-05-10T04:02:36.410181+0000","last_undegraded":"2024-05-10T04:02:36.410181+0000","last_fullsized":"2024-05-10T04:02:36.410181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:02.618922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.175086+0000","last_change":"2024-05-10T04:02:31.198037+0000","last_active":"2024-05-10T04:02:55.175086+0000","last_peered":"2024-05-10T04:02:55.175086+0000","last_clean":"2024-05-10T04:02:55.175086+0000","last_became_active":"2024-05-10T04:02:31.197679+0000","last_became_peered":"2024-05-10T04:02:31.197679+0000","last_unstale":"2024-05-10T04:02:55.175086+0000","last_undegraded":"2024-05-10T04:02:55.175086+0000","last_fullsized":"2024-05-10T04:02:55.175086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:19.059645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636057+0000","last_change":"2024-05-10T04:02:31.630020+0000","last_active":"2024-05-10T04:02:35.636057+0000","last_peered":"2024-05-10T04:02:35.636057+0000","last_clean":"2024-05-10T04:02:35.636057+0000","last_became_active":"2024-05-10T04:02:31.629738+0000","last_became_peered":"2024-05-10T04:02:31.629738+0000","last_unstale":"2024-05-10T04:02:35.636057+0000","last_undegraded":"2024-05-10T04:02:35.636057+0000","last_fullsized":"2024-05-10T04:02:35.636057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:58:42.317143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.410114+0000","last_change":"2024-05-10T04:02:35.647242+0000","last_active":"2024-05-10T04:02:36.410114+0000","last_peered":"2024-05-10T04:02:36.410114+0000","last_clean":"2024-05-10T04:02:36.410114+0000","last_became_active":"2024-05-10T04:02:35.646774+0000","last_became_peered":"2024-05-10T04:02:35.646774+0000","last_unstale":"2024-05-10T04:02:36.410114+0000","last_undegraded":"2024-05-10T04:02:36.410114+0000","last_fullsized":"2024-05-10T04:02:36.410114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:20:05.602820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"80'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.898649+0000","last_change":"2024-05-10T04:02:31.191981+0000","last_active":"2024-05-10T04:02:35.898649+0000","last_peered":"2024-05-10T04:02:35.898649+0000","last_clean":"2024-05-10T04:02:35.898649+0000","last_became_active":"2024-05-10T04:02:31.191632+0000","last_became_peered":"2024-05-10T04:02:31.191632+0000","last_unstale":"2024-05-10T04:02:35.898649+0000","last_undegraded":"2024-05-10T04:02:35.898649+0000","last_fullsized":"2024-05-10T04:02:35.898649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:44:52.744661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640083+0000","last_change":"2024-05-10T04:02:31.643240+0000","last_active":"2024-05-10T04:02:35.640083+0000","last_peered":"2024-05-10T04:02:35.640083+0000","last_clean":"2024-05-10T04:02:35.640083+0000","last_became_active":"2024-05-10T04:02:31.642891+0000","last_became_peered":"2024-05-10T04:02:31.642891+0000","last_unstale":"2024-05-10T04:02:35.640083+0000","last_undegraded":"2024-05-10T04:02:35.640083+0000","last_fullsized":"2024-05-10T04:02:35.640083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:41:42.901192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.639026+0000","last_change":"2024-05-10T04:02:35.652370+0000","last_active":"2024-05-10T04:02:36.639026+0000","last_peered":"2024-05-10T04:02:36.639026+0000","last_clean":"2024-05-10T04:02:36.639026+0000","last_became_active":"2024-05-10T04:02:35.651803+0000","last_became_peered":"2024-05-10T04:02:35.651803+0000","last_unstale":"2024-05-10T04:02:36.639026+0000","last_undegraded":"2024-05-10T04:02:36.639026+0000","last_fullsized":"2024-05-10T04:02:36.639026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:33:36.049311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630907+0000","last_change":"2024-05-10T04:02:31.189053+0000","last_active":"2024-05-10T04:02:35.630907+0000","last_peered":"2024-05-10T04:02:35.630907+0000","last_clean":"2024-05-10T04:02:35.630907+0000","last_became_active":"2024-05-10T04:02:31.188614+0000","last_became_peered":"2024-05-10T04:02:31.188614+0000","last_unstale":"2024-05-10T04:02:35.630907+0000","last_undegraded":"2024-05-10T04:02:35.630907+0000","last_fullsized":"2024-05-10T04:02:35.630907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:59:37.750507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956848+0000","last_change":"2024-05-10T04:02:31.630379+0000","last_active":"2024-05-10T04:02:35.956848+0000","last_peered":"2024-05-10T04:02:35.956848+0000","last_clean":"2024-05-10T04:02:35.956848+0000","last_became_active":"2024-05-10T04:02:31.630094+0000","last_became_peered":"2024-05-10T04:02:31.630094+0000","last_unstale":"2024-05-10T04:02:35.956848+0000","last_undegraded":"2024-05-10T04:02:35.956848+0000","last_fullsized":"2024-05-10T04:02:35.956848+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:05.699923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638588+0000","last_change":"2024-05-10T04:02:35.650770+0000","last_active":"2024-05-10T04:02:36.638588+0000","last_peered":"2024-05-10T04:02:36.638588+0000","last_clean":"2024-05-10T04:02:36.638588+0000","last_became_active":"2024-05-10T04:02:35.650351+0000","last_became_peered":"2024-05-10T04:02:35.650351+0000","last_unstale":"2024-05-10T04:02:36.638588+0000","last_undegraded":"2024-05-10T04:02:36.638588+0000","last_fullsized":"2024-05-10T04:02:36.638588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:41:06.728126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.068369+0000","last_change":"2024-05-10T04:02:31.189557+0000","last_active":"2024-05-10T04:02:36.068369+0000","last_peered":"2024-05-10T04:02:36.068369+0000","last_clean":"2024-05-10T04:02:36.068369+0000","last_became_active":"2024-05-10T04:02:31.189206+0000","last_became_peered":"2024-05-10T04:02:31.189206+0000","last_unstale":"2024-05-10T04:02:36.068369+0000","last_undegraded":"2024-05-10T04:02:36.068369+0000","last_fullsized":"2024-05-10T04:02:36.068369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:11:08.225965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.645298+0000","last_change":"2024-05-10T04:02:31.633561+0000","last_active":"2024-05-10T04:02:35.645298+0000","last_peered":"2024-05-10T04:02:35.645298+0000","last_clean":"2024-05-10T04:02:35.645298+0000","last_became_active":"2024-05-10T04:02:31.633224+0000","last_became_peered":"2024-05-10T04:02:31.633224+0000","last_unstale":"2024-05-10T04:02:35.645298+0000","last_undegraded":"2024-05-10T04:02:35.645298+0000","last_fullsized":"2024-05-10T04:02:35.645298+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:32.901146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956875+0000","last_change":"2024-05-10T04:02:35.649327+0000","last_active":"2024-05-10T04:02:35.956875+0000","last_peered":"2024-05-10T04:02:35.956875+0000","last_clean":"2024-05-10T04:02:35.956875+0000","last_became_active":"2024-05-10T04:02:35.648982+0000","last_became_peered":"2024-05-10T04:02:35.648982+0000","last_unstale":"2024-05-10T04:02:35.956875+0000","last_undegraded":"2024-05-10T04:02:35.956875+0000","last_fullsized":"2024-05-10T04:02:35.956875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:25.405748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.899395+0000","last_change":"2024-05-10T04:02:31.186447+0000","last_active":"2024-05-10T04:02:35.899395+0000","last_peered":"2024-05-10T04:02:35.899395+0000","last_clean":"2024-05-10T04:02:35.899395+0000","last_became_active":"2024-05-10T04:02:31.186073+0000","last_became_peered":"2024-05-10T04:02:31.186073+0000","last_unstale":"2024-05-10T04:02:35.899395+0000","last_undegraded":"2024-05-10T04:02:35.899395+0000","last_fullsized":"2024-05-10T04:02:35.899395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:53.381147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640992+0000","last_change":"2024-05-10T04:02:31.633772+0000","last_active":"2024-05-10T04:02:35.640992+0000","last_peered":"2024-05-10T04:02:35.640992+0000","last_clean":"2024-05-10T04:02:35.640992+0000","last_became_active":"2024-05-10T04:02:31.629592+0000","last_became_peered":"2024-05-10T04:02:31.629592+0000","last_unstale":"2024-05-10T04:02:35.640992+0000","last_undegraded":"2024-05-10T04:02:35.640992+0000","last_fullsized":"2024-05-10T04:02:35.640992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:38:37.342351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.088114+0000","last_change":"2024-05-10T04:02:35.646960+0000","last_active":"2024-05-10T04:02:36.088114+0000","last_peered":"2024-05-10T04:02:36.088114+0000","last_clean":"2024-05-10T04:02:36.088114+0000","last_became_active":"2024-05-10T04:02:35.646598+0000","last_became_peered":"2024-05-10T04:02:35.646598+0000","last_unstale":"2024-05-10T04:02:36.088114+0000","last_undegraded":"2024-05-10T04:02:36.088114+0000","last_fullsized":"2024-05-10T04:02:36.088114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:40:10.478745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.175169+0000","last_change":"2024-05-10T04:02:31.192846+0000","last_active":"2024-05-10T04:02:55.175169+0000","last_peered":"2024-05-10T04:02:55.175169+0000","last_clean":"2024-05-10T04:02:55.175169+0000","last_became_active":"2024-05-10T04:02:31.192514+0000","last_became_peered":"2024-05-10T04:02:31.192514+0000","last_unstale":"2024-05-10T04:02:55.175169+0000","last_undegraded":"2024-05-10T04:02:55.175169+0000","last_fullsized":"2024-05-10T04:02:55.175169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:50:32.649440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637130+0000","last_change":"2024-05-10T04:02:31.641578+0000","last_active":"2024-05-10T04:02:35.637130+0000","last_peered":"2024-05-10T04:02:35.637130+0000","last_clean":"2024-05-10T04:02:35.637130+0000","last_became_active":"2024-05-10T04:02:31.641268+0000","last_became_peered":"2024-05-10T04:02:31.641268+0000","last_unstale":"2024-05-10T04:02:35.637130+0000","last_undegraded":"2024-05-10T04:02:35.637130+0000","last_fullsized":"2024-05-10T04:02:35.637130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:31:00.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641920+0000","last_change":"2024-05-10T04:02:35.646754+0000","last_active":"2024-05-10T04:02:36.641920+0000","last_peered":"2024-05-10T04:02:36.641920+0000","last_clean":"2024-05-10T04:02:36.641920+0000","last_became_active":"2024-05-10T04:02:35.646342+0000","last_became_peered":"2024-05-10T04:02:35.646342+0000","last_unstale":"2024-05-10T04:02:36.641920+0000","last_undegraded":"2024-05-10T04:02:36.641920+0000","last_fullsized":"2024-05-10T04:02:36.641920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:28:33.373312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.068934+0000","last_change":"2024-05-10T04:02:31.186427+0000","last_active":"2024-05-10T04:02:36.068934+0000","last_peered":"2024-05-10T04:02:36.068934+0000","last_clean":"2024-05-10T04:02:36.068934+0000","last_became_active":"2024-05-10T04:02:31.185937+0000","last_became_peered":"2024-05-10T04:02:31.185937+0000","last_unstale":"2024-05-10T04:02:36.068934+0000","last_undegraded":"2024-05-10T04:02:36.068934+0000","last_fullsized":"2024-05-10T04:02:36.068934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.818126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640892+0000","last_change":"2024-05-10T04:02:31.645582+0000","last_active":"2024-05-10T04:02:35.640892+0000","last_peered":"2024-05-10T04:02:35.640892+0000","last_clean":"2024-05-10T04:02:35.640892+0000","last_became_active":"2024-05-10T04:02:31.645259+0000","last_became_peered":"2024-05-10T04:02:31.645259+0000","last_unstale":"2024-05-10T04:02:35.640892+0000","last_undegraded":"2024-05-10T04:02:35.640892+0000","last_fullsized":"2024-05-10T04:02:35.640892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:17:31.818116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.088108+0000","last_change":"2024-05-10T04:02:35.647341+0000","last_active":"2024-05-10T04:02:36.088108+0000","last_peered":"2024-05-10T04:02:36.088108+0000","last_clean":"2024-05-10T04:02:36.088108+0000","last_became_active":"2024-05-10T04:02:35.646806+0000","last_became_peered":"2024-05-10T04:02:35.646806+0000","last_unstale":"2024-05-10T04:02:36.088108+0000","last_undegraded":"2024-05-10T04:02:36.088108+0000","last_fullsized":"2024-05-10T04:02:36.088108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:01:51.252683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:46.792587+0000","last_change":"2024-05-10T04:02:31.195377+0000","last_active":"2024-05-10T04:02:46.792587+0000","last_peered":"2024-05-10T04:02:46.792587+0000","last_clean":"2024-05-10T04:02:46.792587+0000","last_became_active":"2024-05-10T04:02:31.194965+0000","last_became_peered":"2024-05-10T04:02:31.194965+0000","last_unstale":"2024-05-10T04:02:46.792587+0000","last_undegraded":"2024-05-10T04:02:46.792587+0000","last_fullsized":"2024-05-10T04:02:46.792587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:11:09.237525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636559+0000","last_change":"2024-05-10T04:02:31.629600+0000","last_active":"2024-05-10T04:02:35.636559+0000","last_peered":"2024-05-10T04:02:35.636559+0000","last_clean":"2024-05-10T04:02:35.636559+0000","last_became_active":"2024-05-10T04:02:31.629264+0000","last_became_peered":"2024-05-10T04:02:31.629264+0000","last_unstale":"2024-05-10T04:02:35.636559+0000","last_undegraded":"2024-05-10T04:02:35.636559+0000","last_fullsized":"2024-05-10T04:02:35.636559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:45:08.867783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.069212+0000","last_change":"2024-05-10T04:02:35.649468+0000","last_active":"2024-05-10T04:02:36.069212+0000","last_peered":"2024-05-10T04:02:36.069212+0000","last_clean":"2024-05-10T04:02:36.069212+0000","last_became_active":"2024-05-10T04:02:35.649053+0000","last_became_peered":"2024-05-10T04:02:35.649053+0000","last_unstale":"2024-05-10T04:02:36.069212+0000","last_undegraded":"2024-05-10T04:02:36.069212+0000","last_fullsized":"2024-05-10T04:02:36.069212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:01:05.761287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182116+0000","last_change":"2024-05-10T04:02:31.195169+0000","last_active":"2024-05-10T04:02:56.182116+0000","last_peered":"2024-05-10T04:02:56.182116+0000","last_clean":"2024-05-10T04:02:56.182116+0000","last_became_active":"2024-05-10T04:02:31.194720+0000","last_became_peered":"2024-05-10T04:02:31.194720+0000","last_unstale":"2024-05-10T04:02:56.182116+0000","last_undegraded":"2024-05-10T04:02:56.182116+0000","last_fullsized":"2024-05-10T04:02:56.182116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:42.727841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630936+0000","last_change":"2024-05-10T04:02:31.631739+0000","last_active":"2024-05-10T04:02:35.630936+0000","last_peered":"2024-05-10T04:02:35.630936+0000","last_clean":"2024-05-10T04:02:35.630936+0000","last_became_active":"2024-05-10T04:02:31.631401+0000","last_became_peered":"2024-05-10T04:02:31.631401+0000","last_unstale":"2024-05-10T04:02:35.630936+0000","last_undegraded":"2024-05-10T04:02:35.630936+0000","last_fullsized":"2024-05-10T04:02:35.630936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:06.958328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638793+0000","last_change":"2024-05-10T04:02:35.651484+0000","last_active":"2024-05-10T04:02:36.638793+0000","last_peered":"2024-05-10T04:02:36.638793+0000","last_clean":"2024-05-10T04:02:36.638793+0000","last_became_active":"2024-05-10T04:02:35.651069+0000","last_became_peered":"2024-05-10T04:02:35.651069+0000","last_unstale":"2024-05-10T04:02:36.638793+0000","last_undegraded":"2024-05-10T04:02:36.638793+0000","last_fullsized":"2024-05-10T04:02:36.638793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:26:45.229101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.172679+0000","last_change":"2024-05-10T04:02:31.186314+0000","last_active":"2024-05-10T04:02:55.172679+0000","last_peered":"2024-05-10T04:02:55.172679+0000","last_clean":"2024-05-10T04:02:55.172679+0000","last_became_active":"2024-05-10T04:02:31.185861+0000","last_became_peered":"2024-05-10T04:02:31.185861+0000","last_unstale":"2024-05-10T04:02:55.172679+0000","last_undegraded":"2024-05-10T04:02:55.172679+0000","last_fullsized":"2024-05-10T04:02:55.172679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:07.386978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640394+0000","last_change":"2024-05-10T04:02:31.637151+0000","last_active":"2024-05-10T04:02:35.640394+0000","last_peered":"2024-05-10T04:02:35.640394+0000","last_clean":"2024-05-10T04:02:35.640394+0000","last_became_active":"2024-05-10T04:02:31.636745+0000","last_became_peered":"2024-05-10T04:02:31.636745+0000","last_unstale":"2024-05-10T04:02:35.640394+0000","last_undegraded":"2024-05-10T04:02:35.640394+0000","last_fullsized":"2024-05-10T04:02:35.640394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:59.804391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.088363+0000","last_change":"2024-05-10T04:02:35.656108+0000","last_active":"2024-05-10T04:02:36.088363+0000","last_peered":"2024-05-10T04:02:36.088363+0000","last_clean":"2024-05-10T04:02:36.088363+0000","last_became_active":"2024-05-10T04:02:35.655664+0000","last_became_peered":"2024-05-10T04:02:35.655664+0000","last_unstale":"2024-05-10T04:02:36.088363+0000","last_undegraded":"2024-05-10T04:02:36.088363+0000","last_fullsized":"2024-05-10T04:02:36.088363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:10.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:58.198892+0000","last_change":"2024-05-10T04:02:31.189432+0000","last_active":"2024-05-10T04:02:58.198892+0000","last_peered":"2024-05-10T04:02:58.198892+0000","last_clean":"2024-05-10T04:02:58.198892+0000","last_became_active":"2024-05-10T04:02:31.189062+0000","last_became_peered":"2024-05-10T04:02:31.189062+0000","last_unstale":"2024-05-10T04:02:58.198892+0000","last_undegraded":"2024-05-10T04:02:58.198892+0000","last_fullsized":"2024-05-10T04:02:58.198892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:28.452100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.639024+0000","last_change":"2024-05-10T04:02:31.637517+0000","last_active":"2024-05-10T04:02:35.639024+0000","last_peered":"2024-05-10T04:02:35.639024+0000","last_clean":"2024-05-10T04:02:35.639024+0000","last_became_active":"2024-05-10T04:02:31.636518+0000","last_became_peered":"2024-05-10T04:02:31.636518+0000","last_unstale":"2024-05-10T04:02:35.639024+0000","last_undegraded":"2024-05-10T04:02:35.639024+0000","last_fullsized":"2024-05-10T04:02:35.639024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:07:09.732117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.067746+0000","last_change":"2024-05-10T04:02:35.649693+0000","last_active":"2024-05-10T04:02:36.067746+0000","last_peered":"2024-05-10T04:02:36.067746+0000","last_clean":"2024-05-10T04:02:36.067746+0000","last_became_active":"2024-05-10T04:02:35.649175+0000","last_became_peered":"2024-05-10T04:02:35.649175+0000","last_unstale":"2024-05-10T04:02:36.067746+0000","last_undegraded":"2024-05-10T04:02:36.067746+0000","last_fullsized":"2024-05-10T04:02:36.067746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:03:52.340444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.181556+0000","last_change":"2024-05-10T04:02:31.196909+0000","last_active":"2024-05-10T04:02:56.181556+0000","last_peered":"2024-05-10T04:02:56.181556+0000","last_clean":"2024-05-10T04:02:56.181556+0000","last_became_active":"2024-05-10T04:02:31.196599+0000","last_became_peered":"2024-05-10T04:02:31.196599+0000","last_unstale":"2024-05-10T04:02:56.181556+0000","last_undegraded":"2024-05-10T04:02:56.181556+0000","last_fullsized":"2024-05-10T04:02:56.181556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:16:51.483498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641842+0000","last_change":"2024-05-10T04:02:31.641426+0000","last_active":"2024-05-10T04:02:35.641842+0000","last_peered":"2024-05-10T04:02:35.641842+0000","last_clean":"2024-05-10T04:02:35.641842+0000","last_became_active":"2024-05-10T04:02:31.641054+0000","last_became_peered":"2024-05-10T04:02:31.641054+0000","last_unstale":"2024-05-10T04:02:35.641842+0000","last_undegraded":"2024-05-10T04:02:35.641842+0000","last_fullsized":"2024-05-10T04:02:35.641842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:12:48.472674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.068553+0000","last_change":"2024-05-10T04:02:35.653258+0000","last_active":"2024-05-10T04:02:36.068553+0000","last_peered":"2024-05-10T04:02:36.068553+0000","last_clean":"2024-05-10T04:02:36.068553+0000","last_became_active":"2024-05-10T04:02:35.652859+0000","last_became_peered":"2024-05-10T04:02:35.652859+0000","last_unstale":"2024-05-10T04:02:36.068553+0000","last_undegraded":"2024-05-10T04:02:36.068553+0000","last_fullsized":"2024-05-10T04:02:36.068553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:15:57.814963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:57.193415+0000","last_change":"2024-05-10T04:02:31.192231+0000","last_active":"2024-05-10T04:02:57.193415+0000","last_peered":"2024-05-10T04:02:57.193415+0000","last_clean":"2024-05-10T04:02:57.193415+0000","last_became_active":"2024-05-10T04:02:31.191902+0000","last_became_peered":"2024-05-10T04:02:31.191902+0000","last_unstale":"2024-05-10T04:02:57.193415+0000","last_undegraded":"2024-05-10T04:02:57.193415+0000","last_fullsized":"2024-05-10T04:02:57.193415+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:18.637112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.638702+0000","last_change":"2024-05-10T04:02:31.635514+0000","last_active":"2024-05-10T04:02:35.638702+0000","last_peered":"2024-05-10T04:02:35.638702+0000","last_clean":"2024-05-10T04:02:35.638702+0000","last_became_active":"2024-05-10T04:02:31.635171+0000","last_became_peered":"2024-05-10T04:02:31.635171+0000","last_unstale":"2024-05-10T04:02:35.638702+0000","last_undegraded":"2024-05-10T04:02:35.638702+0000","last_fullsized":"2024-05-10T04:02:35.638702+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:48.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.090937+0000","last_change":"2024-05-10T04:02:35.900174+0000","last_active":"2024-05-10T04:02:36.090937+0000","last_peered":"2024-05-10T04:02:36.090937+0000","last_clean":"2024-05-10T04:02:36.090937+0000","last_became_active":"2024-05-10T04:02:35.899653+0000","last_became_peered":"2024-05-10T04:02:35.899653+0000","last_unstale":"2024-05-10T04:02:36.090937+0000","last_undegraded":"2024-05-10T04:02:36.090937+0000","last_fullsized":"2024-05-10T04:02:36.090937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:32:56.962361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.645483+0000","last_change":"2024-05-10T04:02:31.190173+0000","last_active":"2024-05-10T04:02:35.645483+0000","last_peered":"2024-05-10T04:02:35.645483+0000","last_clean":"2024-05-10T04:02:35.645483+0000","last_became_active":"2024-05-10T04:02:31.189840+0000","last_became_peered":"2024-05-10T04:02:31.189840+0000","last_unstale":"2024-05-10T04:02:35.645483+0000","last_undegraded":"2024-05-10T04:02:35.645483+0000","last_fullsized":"2024-05-10T04:02:35.645483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:31:49.382482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.899188+0000","last_change":"2024-05-10T04:02:31.629497+0000","last_active":"2024-05-10T04:02:35.899188+0000","last_peered":"2024-05-10T04:02:35.899188+0000","last_clean":"2024-05-10T04:02:35.899188+0000","last_became_active":"2024-05-10T04:02:31.629182+0000","last_became_peered":"2024-05-10T04:02:31.629182+0000","last_unstale":"2024-05-10T04:02:35.899188+0000","last_undegraded":"2024-05-10T04:02:35.899188+0000","last_fullsized":"2024-05-10T04:02:35.899188+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:11:33.478737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.067644+0000","last_change":"2024-05-10T04:02:35.647249+0000","last_active":"2024-05-10T04:02:36.067644+0000","last_peered":"2024-05-10T04:02:36.067644+0000","last_clean":"2024-05-10T04:02:36.067644+0000","last_became_active":"2024-05-10T04:02:35.646716+0000","last_became_peered":"2024-05-10T04:02:35.646716+0000","last_unstale":"2024-05-10T04:02:36.067644+0000","last_undegraded":"2024-05-10T04:02:36.067644+0000","last_fullsized":"2024-05-10T04:02:36.067644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:44.651836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.639382+0000","last_change":"2024-05-10T04:02:31.191241+0000","last_active":"2024-05-10T04:02:36.639382+0000","last_peered":"2024-05-10T04:02:36.639382+0000","last_clean":"2024-05-10T04:02:36.639382+0000","last_became_active":"2024-05-10T04:02:31.190888+0000","last_became_peered":"2024-05-10T04:02:31.190888+0000","last_unstale":"2024-05-10T04:02:36.639382+0000","last_undegraded":"2024-05-10T04:02:36.639382+0000","last_fullsized":"2024-05-10T04:02:36.639382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:55.256119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.898849+0000","last_change":"2024-05-10T04:02:31.638394+0000","last_active":"2024-05-10T04:02:35.898849+0000","last_peered":"2024-05-10T04:02:35.898849+0000","last_clean":"2024-05-10T04:02:35.898849+0000","last_became_active":"2024-05-10T04:02:31.638025+0000","last_became_peered":"2024-05-10T04:02:31.638025+0000","last_unstale":"2024-05-10T04:02:35.898849+0000","last_undegraded":"2024-05-10T04:02:35.898849+0000","last_fullsized":"2024-05-10T04:02:35.898849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:13.088393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.067640+0000","last_change":"2024-05-10T04:02:35.899898+0000","last_active":"2024-05-10T04:02:36.067640+0000","last_peered":"2024-05-10T04:02:36.067640+0000","last_clean":"2024-05-10T04:02:36.067640+0000","last_became_active":"2024-05-10T04:02:35.899485+0000","last_became_peered":"2024-05-10T04:02:35.899485+0000","last_unstale":"2024-05-10T04:02:36.067640+0000","last_undegraded":"2024-05-10T04:02:36.067640+0000","last_fullsized":"2024-05-10T04:02:36.067640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:26:50.909122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640359+0000","last_change":"2024-05-10T04:02:31.640923+0000","last_active":"2024-05-10T04:02:35.640359+0000","last_peered":"2024-05-10T04:02:35.640359+0000","last_clean":"2024-05-10T04:02:35.640359+0000","last_became_active":"2024-05-10T04:02:31.640490+0000","last_became_peered":"2024-05-10T04:02:31.640490+0000","last_unstale":"2024-05-10T04:02:35.640359+0000","last_undegraded":"2024-05-10T04:02:35.640359+0000","last_fullsized":"2024-05-10T04:02:35.640359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:31:58.585789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630772+0000","last_change":"2024-05-10T04:02:31.191326+0000","last_active":"2024-05-10T04:02:35.630772+0000","last_peered":"2024-05-10T04:02:35.630772+0000","last_clean":"2024-05-10T04:02:35.630772+0000","last_became_active":"2024-05-10T04:02:31.190974+0000","last_became_peered":"2024-05-10T04:02:31.190974+0000","last_unstale":"2024-05-10T04:02:35.630772+0000","last_undegraded":"2024-05-10T04:02:35.630772+0000","last_fullsized":"2024-05-10T04:02:35.630772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:43.670619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.325315+0000","last_change":"2024-05-10T04:02:35.647526+0000","last_active":"2024-05-10T04:02:36.325315+0000","last_peered":"2024-05-10T04:02:36.325315+0000","last_clean":"2024-05-10T04:02:36.325315+0000","last_became_active":"2024-05-10T04:02:35.647018+0000","last_became_peered":"2024-05-10T04:02:35.647018+0000","last_unstale":"2024-05-10T04:02:36.325315+0000","last_undegraded":"2024-05-10T04:02:36.325315+0000","last_fullsized":"2024-05-10T04:02:36.325315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:17.173497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956894+0000","last_change":"2024-05-10T04:02:31.632263+0000","last_active":"2024-05-10T04:02:35.956894+0000","last_peered":"2024-05-10T04:02:35.956894+0000","last_clean":"2024-05-10T04:02:35.956894+0000","last_became_active":"2024-05-10T04:02:31.631785+0000","last_became_peered":"2024-05-10T04:02:31.631785+0000","last_unstale":"2024-05-10T04:02:35.956894+0000","last_undegraded":"2024-05-10T04:02:35.956894+0000","last_fullsized":"2024-05-10T04:02:35.956894+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:30.585862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:57.191865+0000","last_change":"2024-05-10T04:02:31.182262+0000","last_active":"2024-05-10T04:02:57.191865+0000","last_peered":"2024-05-10T04:02:57.191865+0000","last_clean":"2024-05-10T04:02:57.191865+0000","last_became_active":"2024-05-10T04:02:31.181802+0000","last_became_peered":"2024-05-10T04:02:31.181802+0000","last_unstale":"2024-05-10T04:02:57.191865+0000","last_undegraded":"2024-05-10T04:02:57.191865+0000","last_fullsized":"2024-05-10T04:02:57.191865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:45:40.696571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":325,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640698+0000","last_change":"2024-05-10T04:01:41.866365+0000","last_active":"2024-05-10T04:02:36.640698+0000","last_peered":"2024-05-10T04:02:36.640698+0000","last_clean":"2024-05-10T04:02:36.640698+0000","last_became_active":"2024-05-10T04:01:41.865825+0000","last_became_peered":"2024-05-10T04:01:41.865825+0000","last_unstale":"2024-05-10T04:02:36.640698+0000","last_undegraded":"2024-05-10T04:02:36.640698+0000","last_fullsized":"2024-05-10T04:02:36.640698+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_clean_scrub_stamp":"2024-05-10T03:59:19.853473+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:28:29.598339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.069016+0000","last_change":"2024-05-10T04:02:35.900841+0000","last_active":"2024-05-10T04:02:36.069016+0000","last_peered":"2024-05-10T04:02:36.069016+0000","last_clean":"2024-05-10T04:02:36.069016+0000","last_became_active":"2024-05-10T04:02:35.900323+0000","last_became_peered":"2024-05-10T04:02:35.900323+0000","last_unstale":"2024-05-10T04:02:36.069016+0000","last_undegraded":"2024-05-10T04:02:36.069016+0000","last_fullsized":"2024-05-10T04:02:36.069016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:00.554804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630799+0000","last_change":"2024-05-10T04:02:31.641167+0000","last_active":"2024-05-10T04:02:35.630799+0000","last_peered":"2024-05-10T04:02:35.630799+0000","last_clean":"2024-05-10T04:02:35.630799+0000","last_became_active":"2024-05-10T04:02:31.640824+0000","last_became_peered":"2024-05-10T04:02:31.640824+0000","last_unstale":"2024-05-10T04:02:35.630799+0000","last_undegraded":"2024-05-10T04:02:35.630799+0000","last_fullsized":"2024-05-10T04:02:35.630799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:59:48.531970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.180041+0000","last_change":"2024-05-10T04:02:31.187834+0000","last_active":"2024-05-10T04:02:56.180041+0000","last_peered":"2024-05-10T04:02:56.180041+0000","last_clean":"2024-05-10T04:02:56.180041+0000","last_became_active":"2024-05-10T04:02:31.187440+0000","last_became_peered":"2024-05-10T04:02:31.187440+0000","last_unstale":"2024-05-10T04:02:56.180041+0000","last_undegraded":"2024-05-10T04:02:56.180041+0000","last_fullsized":"2024-05-10T04:02:56.180041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:50:37.678245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638789+0000","last_change":"2024-05-10T04:02:35.649664+0000","last_active":"2024-05-10T04:02:36.638789+0000","last_peered":"2024-05-10T04:02:36.638789+0000","last_clean":"2024-05-10T04:02:36.638789+0000","last_became_active":"2024-05-10T04:02:35.649076+0000","last_became_peered":"2024-05-10T04:02:35.649076+0000","last_unstale":"2024-05-10T04:02:36.638789+0000","last_undegraded":"2024-05-10T04:02:36.638789+0000","last_fullsized":"2024-05-10T04:02:36.638789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:14:42.864845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.173775+0000","last_change":"2024-05-10T04:02:31.192126+0000","last_active":"2024-05-10T04:02:55.173775+0000","last_peered":"2024-05-10T04:02:55.173775+0000","last_clean":"2024-05-10T04:02:55.173775+0000","last_became_active":"2024-05-10T04:02:31.191771+0000","last_became_peered":"2024-05-10T04:02:31.191771+0000","last_unstale":"2024-05-10T04:02:55.173775+0000","last_undegraded":"2024-05-10T04:02:55.173775+0000","last_fullsized":"2024-05-10T04:02:55.173775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:43.550535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632220+0000","last_change":"2024-05-10T04:02:31.620788+0000","last_active":"2024-05-10T04:02:35.632220+0000","last_peered":"2024-05-10T04:02:35.632220+0000","last_clean":"2024-05-10T04:02:35.632220+0000","last_became_active":"2024-05-10T04:02:31.620456+0000","last_became_peered":"2024-05-10T04:02:31.620456+0000","last_unstale":"2024-05-10T04:02:35.632220+0000","last_undegraded":"2024-05-10T04:02:35.632220+0000","last_fullsized":"2024-05-10T04:02:35.632220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:02.956030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956837+0000","last_change":"2024-05-10T04:02:35.899935+0000","last_active":"2024-05-10T04:02:35.956837+0000","last_peered":"2024-05-10T04:02:35.956837+0000","last_clean":"2024-05-10T04:02:35.956837+0000","last_became_active":"2024-05-10T04:02:35.899488+0000","last_became_peered":"2024-05-10T04:02:35.899488+0000","last_unstale":"2024-05-10T04:02:35.956837+0000","last_undegraded":"2024-05-10T04:02:35.956837+0000","last_fullsized":"2024-05-10T04:02:35.956837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:28.600463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182368+0000","last_change":"2024-05-10T04:02:31.188605+0000","last_active":"2024-05-10T04:02:56.182368+0000","last_peered":"2024-05-10T04:02:56.182368+0000","last_clean":"2024-05-10T04:02:56.182368+0000","last_became_active":"2024-05-10T04:02:31.188283+0000","last_became_peered":"2024-05-10T04:02:31.188283+0000","last_unstale":"2024-05-10T04:02:56.182368+0000","last_undegraded":"2024-05-10T04:02:56.182368+0000","last_fullsized":"2024-05-10T04:02:56.182368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:40:03.940269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956931+0000","last_change":"2024-05-10T04:02:31.629476+0000","last_active":"2024-05-10T04:02:35.956931+0000","last_peered":"2024-05-10T04:02:35.956931+0000","last_clean":"2024-05-10T04:02:35.956931+0000","last_became_active":"2024-05-10T04:02:31.629104+0000","last_became_peered":"2024-05-10T04:02:31.629104+0000","last_unstale":"2024-05-10T04:02:35.956931+0000","last_undegraded":"2024-05-10T04:02:35.956931+0000","last_fullsized":"2024-05-10T04:02:35.956931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:57.873755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.409730+0000","last_change":"2024-05-10T04:02:35.644094+0000","last_active":"2024-05-10T04:02:36.409730+0000","last_peered":"2024-05-10T04:02:36.409730+0000","last_clean":"2024-05-10T04:02:36.409730+0000","last_became_active":"2024-05-10T04:02:35.643382+0000","last_became_peered":"2024-05-10T04:02:35.643382+0000","last_unstale":"2024-05-10T04:02:36.409730+0000","last_undegraded":"2024-05-10T04:02:36.409730+0000","last_fullsized":"2024-05-10T04:02:36.409730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:33.250052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.175394+0000","last_change":"2024-05-10T04:02:31.189302+0000","last_active":"2024-05-10T04:02:55.175394+0000","last_peered":"2024-05-10T04:02:55.175394+0000","last_clean":"2024-05-10T04:02:55.175394+0000","last_became_active":"2024-05-10T04:02:31.188323+0000","last_became_peered":"2024-05-10T04:02:31.188323+0000","last_unstale":"2024-05-10T04:02:55.175394+0000","last_undegraded":"2024-05-10T04:02:55.175394+0000","last_fullsized":"2024-05-10T04:02:55.175394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:11:02.922884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637085+0000","last_change":"2024-05-10T04:02:31.635864+0000","last_active":"2024-05-10T04:02:35.637085+0000","last_peered":"2024-05-10T04:02:35.637085+0000","last_clean":"2024-05-10T04:02:35.637085+0000","last_became_active":"2024-05-10T04:02:31.635512+0000","last_became_peered":"2024-05-10T04:02:31.635512+0000","last_unstale":"2024-05-10T04:02:35.637085+0000","last_undegraded":"2024-05-10T04:02:35.637085+0000","last_fullsized":"2024-05-10T04:02:35.637085+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:26:10.957848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.642076+0000","last_change":"2024-05-10T04:02:35.900489+0000","last_active":"2024-05-10T04:02:36.642076+0000","last_peered":"2024-05-10T04:02:36.642076+0000","last_clean":"2024-05-10T04:02:36.642076+0000","last_became_active":"2024-05-10T04:02:35.899908+0000","last_became_peered":"2024-05-10T04:02:35.899908+0000","last_unstale":"2024-05-10T04:02:36.642076+0000","last_undegraded":"2024-05-10T04:02:36.642076+0000","last_fullsized":"2024-05-10T04:02:36.642076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:08.386128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.173898+0000","last_change":"2024-05-10T04:02:31.193956+0000","last_active":"2024-05-10T04:02:55.173898+0000","last_peered":"2024-05-10T04:02:55.173898+0000","last_clean":"2024-05-10T04:02:55.173898+0000","last_became_active":"2024-05-10T04:02:31.193574+0000","last_became_peered":"2024-05-10T04:02:31.193574+0000","last_unstale":"2024-05-10T04:02:55.173898+0000","last_undegraded":"2024-05-10T04:02:55.173898+0000","last_fullsized":"2024-05-10T04:02:55.173898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:46.369152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635522+0000","last_change":"2024-05-10T04:02:31.633263+0000","last_active":"2024-05-10T04:02:35.635522+0000","last_peered":"2024-05-10T04:02:35.635522+0000","last_clean":"2024-05-10T04:02:35.635522+0000","last_became_active":"2024-05-10T04:02:31.632903+0000","last_became_peered":"2024-05-10T04:02:31.632903+0000","last_unstale":"2024-05-10T04:02:35.635522+0000","last_undegraded":"2024-05-10T04:02:35.635522+0000","last_fullsized":"2024-05-10T04:02:35.635522+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:28.047473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.088498+0000","last_change":"2024-05-10T04:02:35.656003+0000","last_active":"2024-05-10T04:02:36.088498+0000","last_peered":"2024-05-10T04:02:36.088498+0000","last_clean":"2024-05-10T04:02:36.088498+0000","last_became_active":"2024-05-10T04:02:35.655518+0000","last_became_peered":"2024-05-10T04:02:35.655518+0000","last_unstale":"2024-05-10T04:02:36.088498+0000","last_undegraded":"2024-05-10T04:02:36.088498+0000","last_fullsized":"2024-05-10T04:02:36.088498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:49.696699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.174999+0000","last_change":"2024-05-10T04:02:31.195297+0000","last_active":"2024-05-10T04:02:55.174999+0000","last_peered":"2024-05-10T04:02:55.174999+0000","last_clean":"2024-05-10T04:02:55.174999+0000","last_became_active":"2024-05-10T04:02:31.193960+0000","last_became_peered":"2024-05-10T04:02:31.193960+0000","last_unstale":"2024-05-10T04:02:55.174999+0000","last_undegraded":"2024-05-10T04:02:55.174999+0000","last_fullsized":"2024-05-10T04:02:55.174999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:04:10.314700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637944+0000","last_change":"2024-05-10T04:02:31.629247+0000","last_active":"2024-05-10T04:02:35.637944+0000","last_peered":"2024-05-10T04:02:35.637944+0000","last_clean":"2024-05-10T04:02:35.637944+0000","last_became_active":"2024-05-10T04:02:31.628838+0000","last_became_peered":"2024-05-10T04:02:31.628838+0000","last_unstale":"2024-05-10T04:02:35.637944+0000","last_undegraded":"2024-05-10T04:02:35.637944+0000","last_fullsized":"2024-05-10T04:02:35.637944+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:46:39.483264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641844+0000","last_change":"2024-05-10T04:02:35.644502+0000","last_active":"2024-05-10T04:02:36.641844+0000","last_peered":"2024-05-10T04:02:36.641844+0000","last_clean":"2024-05-10T04:02:36.641844+0000","last_became_active":"2024-05-10T04:02:35.643942+0000","last_became_peered":"2024-05-10T04:02:35.643942+0000","last_unstale":"2024-05-10T04:02:36.641844+0000","last_undegraded":"2024-05-10T04:02:36.641844+0000","last_fullsized":"2024-05-10T04:02:36.641844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:22:42.394568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632281+0000","last_change":"2024-05-10T04:02:31.193357+0000","last_active":"2024-05-10T04:02:35.632281+0000","last_peered":"2024-05-10T04:02:35.632281+0000","last_clean":"2024-05-10T04:02:35.632281+0000","last_became_active":"2024-05-10T04:02:31.190306+0000","last_became_peered":"2024-05-10T04:02:31.190306+0000","last_unstale":"2024-05-10T04:02:35.632281+0000","last_undegraded":"2024-05-10T04:02:35.632281+0000","last_fullsized":"2024-05-10T04:02:35.632281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:13.368960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956904+0000","last_change":"2024-05-10T04:02:31.628029+0000","last_active":"2024-05-10T04:02:35.956904+0000","last_peered":"2024-05-10T04:02:35.956904+0000","last_clean":"2024-05-10T04:02:35.956904+0000","last_became_active":"2024-05-10T04:02:31.627667+0000","last_became_peered":"2024-05-10T04:02:31.627667+0000","last_unstale":"2024-05-10T04:02:35.956904+0000","last_undegraded":"2024-05-10T04:02:35.956904+0000","last_fullsized":"2024-05-10T04:02:35.956904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:39:10.878894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641977+0000","last_change":"2024-05-10T04:02:35.900237+0000","last_active":"2024-05-10T04:02:36.641977+0000","last_peered":"2024-05-10T04:02:36.641977+0000","last_clean":"2024-05-10T04:02:36.641977+0000","last_became_active":"2024-05-10T04:02:35.899781+0000","last_became_peered":"2024-05-10T04:02:35.899781+0000","last_unstale":"2024-05-10T04:02:36.641977+0000","last_undegraded":"2024-05-10T04:02:36.641977+0000","last_fullsized":"2024-05-10T04:02:36.641977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:52:46.132557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.174762+0000","last_change":"2024-05-10T04:02:31.196920+0000","last_active":"2024-05-10T04:02:55.174762+0000","last_peered":"2024-05-10T04:02:55.174762+0000","last_clean":"2024-05-10T04:02:55.174762+0000","last_became_active":"2024-05-10T04:02:31.196191+0000","last_became_peered":"2024-05-10T04:02:31.196191+0000","last_unstale":"2024-05-10T04:02:55.174762+0000","last_undegraded":"2024-05-10T04:02:55.174762+0000","last_fullsized":"2024-05-10T04:02:55.174762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:45:17.980149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.639518+0000","last_change":"2024-05-10T04:02:31.623678+0000","last_active":"2024-05-10T04:02:35.639518+0000","last_peered":"2024-05-10T04:02:35.639518+0000","last_clean":"2024-05-10T04:02:35.639518+0000","last_became_active":"2024-05-10T04:02:31.623090+0000","last_became_peered":"2024-05-10T04:02:31.623090+0000","last_unstale":"2024-05-10T04:02:35.639518+0000","last_undegraded":"2024-05-10T04:02:35.639518+0000","last_fullsized":"2024-05-10T04:02:35.639518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:34.321959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.650242+0000","last_change":"2024-05-10T04:02:35.648833+0000","last_active":"2024-05-10T04:02:35.650242+0000","last_peered":"2024-05-10T04:02:35.650242+0000","last_clean":"2024-05-10T04:02:35.650242+0000","last_became_active":"2024-05-10T04:02:35.648391+0000","last_became_peered":"2024-05-10T04:02:35.648391+0000","last_unstale":"2024-05-10T04:02:35.650242+0000","last_undegraded":"2024-05-10T04:02:35.650242+0000","last_fullsized":"2024-05-10T04:02:35.650242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:12.773284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182221+0000","last_change":"2024-05-10T04:02:31.192900+0000","last_active":"2024-05-10T04:02:56.182221+0000","last_peered":"2024-05-10T04:02:56.182221+0000","last_clean":"2024-05-10T04:02:56.182221+0000","last_became_active":"2024-05-10T04:02:31.192481+0000","last_became_peered":"2024-05-10T04:02:31.192481+0000","last_unstale":"2024-05-10T04:02:56.182221+0000","last_undegraded":"2024-05-10T04:02:56.182221+0000","last_fullsized":"2024-05-10T04:02:56.182221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:17.202383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641659+0000","last_change":"2024-05-10T04:02:31.628490+0000","last_active":"2024-05-10T04:02:35.641659+0000","last_peered":"2024-05-10T04:02:35.641659+0000","last_clean":"2024-05-10T04:02:35.641659+0000","last_became_active":"2024-05-10T04:02:31.627889+0000","last_became_peered":"2024-05-10T04:02:31.627889+0000","last_unstale":"2024-05-10T04:02:35.641659+0000","last_undegraded":"2024-05-10T04:02:35.641659+0000","last_fullsized":"2024-05-10T04:02:35.641659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:32:55.366759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638326+0000","last_change":"2024-05-10T04:02:35.653530+0000","last_active":"2024-05-10T04:02:36.638326+0000","last_peered":"2024-05-10T04:02:36.638326+0000","last_clean":"2024-05-10T04:02:36.638326+0000","last_became_active":"2024-05-10T04:02:35.653095+0000","last_became_peered":"2024-05-10T04:02:35.653095+0000","last_unstale":"2024-05-10T04:02:36.638326+0000","last_undegraded":"2024-05-10T04:02:36.638326+0000","last_fullsized":"2024-05-10T04:02:36.638326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:09.921713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641619+0000","last_change":"2024-05-10T04:02:31.189434+0000","last_active":"2024-05-10T04:02:35.641619+0000","last_peered":"2024-05-10T04:02:35.641619+0000","last_clean":"2024-05-10T04:02:35.641619+0000","last_became_active":"2024-05-10T04:02:31.188843+0000","last_became_peered":"2024-05-10T04:02:31.188843+0000","last_unstale":"2024-05-10T04:02:35.641619+0000","last_undegraded":"2024-05-10T04:02:35.641619+0000","last_fullsized":"2024-05-10T04:02:35.641619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:31.508794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640750+0000","last_change":"2024-05-10T04:02:31.636775+0000","last_active":"2024-05-10T04:02:35.640750+0000","last_peered":"2024-05-10T04:02:35.640750+0000","last_clean":"2024-05-10T04:02:35.640750+0000","last_became_active":"2024-05-10T04:02:31.635946+0000","last_became_peered":"2024-05-10T04:02:31.635946+0000","last_unstale":"2024-05-10T04:02:35.640750+0000","last_undegraded":"2024-05-10T04:02:35.640750+0000","last_fullsized":"2024-05-10T04:02:35.640750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:48.830097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.068189+0000","last_change":"2024-05-10T04:02:35.646991+0000","last_active":"2024-05-10T04:02:36.068189+0000","last_peered":"2024-05-10T04:02:36.068189+0000","last_clean":"2024-05-10T04:02:36.068189+0000","last_became_active":"2024-05-10T04:02:35.646585+0000","last_became_peered":"2024-05-10T04:02:35.646585+0000","last_unstale":"2024-05-10T04:02:36.068189+0000","last_undegraded":"2024-05-10T04:02:36.068189+0000","last_fullsized":"2024-05-10T04:02:36.068189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:32:50.329746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.631077+0000","last_change":"2024-05-10T04:02:31.192103+0000","last_active":"2024-05-10T04:02:35.631077+0000","last_peered":"2024-05-10T04:02:35.631077+0000","last_clean":"2024-05-10T04:02:35.631077+0000","last_became_active":"2024-05-10T04:02:31.191378+0000","last_became_peered":"2024-05-10T04:02:31.191378+0000","last_unstale":"2024-05-10T04:02:35.631077+0000","last_undegraded":"2024-05-10T04:02:35.631077+0000","last_fullsized":"2024-05-10T04:02:35.631077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:55.298158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641428+0000","last_change":"2024-05-10T04:02:31.628587+0000","last_active":"2024-05-10T04:02:35.641428+0000","last_peered":"2024-05-10T04:02:35.641428+0000","last_clean":"2024-05-10T04:02:35.641428+0000","last_became_active":"2024-05-10T04:02:31.627296+0000","last_became_peered":"2024-05-10T04:02:31.627296+0000","last_unstale":"2024-05-10T04:02:35.641428+0000","last_undegraded":"2024-05-10T04:02:35.641428+0000","last_fullsized":"2024-05-10T04:02:35.641428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:04:09.783266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.638054+0000","last_change":"2024-05-10T04:02:31.634668+0000","last_active":"2024-05-10T04:02:35.638054+0000","last_peered":"2024-05-10T04:02:35.638054+0000","last_clean":"2024-05-10T04:02:35.638054+0000","last_became_active":"2024-05-10T04:02:31.634321+0000","last_became_peered":"2024-05-10T04:02:31.634321+0000","last_unstale":"2024-05-10T04:02:35.638054+0000","last_undegraded":"2024-05-10T04:02:35.638054+0000","last_fullsized":"2024-05-10T04:02:35.638054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:16:44.833804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.633396+0000","last_change":"2024-05-10T04:02:31.646002+0000","last_active":"2024-05-10T04:02:35.633396+0000","last_peered":"2024-05-10T04:02:35.633396+0000","last_clean":"2024-05-10T04:02:35.633396+0000","last_became_active":"2024-05-10T04:02:31.645673+0000","last_became_peered":"2024-05-10T04:02:31.645673+0000","last_unstale":"2024-05-10T04:02:35.633396+0000","last_undegraded":"2024-05-10T04:02:35.633396+0000","last_fullsized":"2024-05-10T04:02:35.633396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:26:26.814523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632295+0000","last_change":"2024-05-10T04:02:31.630606+0000","last_active":"2024-05-10T04:02:35.632295+0000","last_peered":"2024-05-10T04:02:35.632295+0000","last_clean":"2024-05-10T04:02:35.632295+0000","last_became_active":"2024-05-10T04:02:31.630313+0000","last_became_peered":"2024-05-10T04:02:31.630313+0000","last_unstale":"2024-05-10T04:02:35.632295+0000","last_undegraded":"2024-05-10T04:02:35.632295+0000","last_fullsized":"2024-05-10T04:02:35.632295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:39.876016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636526+0000","last_change":"2024-05-10T04:02:31.636239+0000","last_active":"2024-05-10T04:02:35.636526+0000","last_peered":"2024-05-10T04:02:35.636526+0000","last_clean":"2024-05-10T04:02:35.636526+0000","last_became_active":"2024-05-10T04:02:31.635377+0000","last_became_peered":"2024-05-10T04:02:31.635377+0000","last_unstale":"2024-05-10T04:02:35.636526+0000","last_undegraded":"2024-05-10T04:02:35.636526+0000","last_fullsized":"2024-05-10T04:02:35.636526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:02.612285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632276+0000","last_change":"2024-05-10T04:02:31.643231+0000","last_active":"2024-05-10T04:02:35.632276+0000","last_peered":"2024-05-10T04:02:35.632276+0000","last_clean":"2024-05-10T04:02:35.632276+0000","last_became_active":"2024-05-10T04:02:31.642797+0000","last_became_peered":"2024-05-10T04:02:31.642797+0000","last_unstale":"2024-05-10T04:02:35.632276+0000","last_undegraded":"2024-05-10T04:02:35.632276+0000","last_fullsized":"2024-05-10T04:02:35.632276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:01:57.365339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.638745+0000","last_change":"2024-05-10T04:02:31.633928+0000","last_active":"2024-05-10T04:02:35.638745+0000","last_peered":"2024-05-10T04:02:35.638745+0000","last_clean":"2024-05-10T04:02:35.638745+0000","last_became_active":"2024-05-10T04:02:31.633423+0000","last_became_peered":"2024-05-10T04:02:31.633423+0000","last_unstale":"2024-05-10T04:02:35.638745+0000","last_undegraded":"2024-05-10T04:02:35.638745+0000","last_fullsized":"2024-05-10T04:02:35.638745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:07.011354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632757+0000","last_change":"2024-05-10T04:02:31.627151+0000","last_active":"2024-05-10T04:02:35.632757+0000","last_peered":"2024-05-10T04:02:35.632757+0000","last_clean":"2024-05-10T04:02:35.632757+0000","last_became_active":"2024-05-10T04:02:31.626780+0000","last_became_peered":"2024-05-10T04:02:31.626780+0000","last_unstale":"2024-05-10T04:02:35.632757+0000","last_undegraded":"2024-05-10T04:02:35.632757+0000","last_fullsized":"2024-05-10T04:02:35.632757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:32.070530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956941+0000","last_change":"2024-05-10T04:02:31.640669+0000","last_active":"2024-05-10T04:02:35.956941+0000","last_peered":"2024-05-10T04:02:35.956941+0000","last_clean":"2024-05-10T04:02:35.956941+0000","last_became_active":"2024-05-10T04:02:31.640410+0000","last_became_peered":"2024-05-10T04:02:31.640410+0000","last_unstale":"2024-05-10T04:02:35.956941+0000","last_undegraded":"2024-05-10T04:02:35.956941+0000","last_fullsized":"2024-05-10T04:02:35.956941+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:02.235391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640243+0000","last_change":"2024-05-10T04:02:31.633624+0000","last_active":"2024-05-10T04:02:35.640243+0000","last_peered":"2024-05-10T04:02:35.640243+0000","last_clean":"2024-05-10T04:02:35.640243+0000","last_became_active":"2024-05-10T04:02:31.633241+0000","last_became_peered":"2024-05-10T04:02:31.633241+0000","last_unstale":"2024-05-10T04:02:35.640243+0000","last_undegraded":"2024-05-10T04:02:35.640243+0000","last_fullsized":"2024-05-10T04:02:35.640243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:58.750912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956858+0000","last_change":"2024-05-10T04:02:31.628632+0000","last_active":"2024-05-10T04:02:35.956858+0000","last_peered":"2024-05-10T04:02:35.956858+0000","last_clean":"2024-05-10T04:02:35.956858+0000","last_became_active":"2024-05-10T04:02:31.628349+0000","last_became_peered":"2024-05-10T04:02:31.628349+0000","last_unstale":"2024-05-10T04:02:35.956858+0000","last_undegraded":"2024-05-10T04:02:35.956858+0000","last_fullsized":"2024-05-10T04:02:35.956858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:32.575168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640841+0000","last_change":"2024-05-10T04:02:31.631328+0000","last_active":"2024-05-10T04:02:35.640841+0000","last_peered":"2024-05-10T04:02:35.640841+0000","last_clean":"2024-05-10T04:02:35.640841+0000","last_became_active":"2024-05-10T04:02:31.630957+0000","last_became_peered":"2024-05-10T04:02:31.630957+0000","last_unstale":"2024-05-10T04:02:35.640841+0000","last_undegraded":"2024-05-10T04:02:35.640841+0000","last_fullsized":"2024-05-10T04:02:35.640841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:40.400911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630956+0000","last_change":"2024-05-10T04:02:31.617972+0000","last_active":"2024-05-10T04:02:35.630956+0000","last_peered":"2024-05-10T04:02:35.630956+0000","last_clean":"2024-05-10T04:02:35.630956+0000","last_became_active":"2024-05-10T04:02:31.617663+0000","last_became_peered":"2024-05-10T04:02:31.617663+0000","last_unstale":"2024-05-10T04:02:35.630956+0000","last_undegraded":"2024-05-10T04:02:35.630956+0000","last_fullsized":"2024-05-10T04:02:35.630956+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:50.121955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641183+0000","last_change":"2024-05-10T04:02:31.644256+0000","last_active":"2024-05-10T04:02:35.641183+0000","last_peered":"2024-05-10T04:02:35.641183+0000","last_clean":"2024-05-10T04:02:35.641183+0000","last_became_active":"2024-05-10T04:02:31.643819+0000","last_became_peered":"2024-05-10T04:02:31.643819+0000","last_unstale":"2024-05-10T04:02:35.641183+0000","last_undegraded":"2024-05-10T04:02:35.641183+0000","last_fullsized":"2024-05-10T04:02:35.641183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:54.932547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279970,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7523,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":112,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":40501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514511,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":921600,"data_stored":457767,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":10,"up_from":68,"seq":292057776146,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":987136,"data_stored":1051262,"data_compressed":47273,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":1,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.35699999999999998}]},{"osd":2,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.71999999999999997}]},{"osd":3,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.59599999999999997}]},{"osd":4,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.498}]},{"osd":5,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76300000000000001}]},{"osd":6,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.629},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":8,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.70699999999999996}]},{"osd":9,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.66300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":9,"up_from":64,"seq":274877906966,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":937984,"data_stored":458003,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.307},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.34200000000000003}]},{"osd":1,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.502},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51600000000000001}]},{"osd":3,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.32500000000000001}]},{"osd":4,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.432},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.48799999999999999}]},{"osd":5,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.1699999999999999}]},{"osd":7,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.40200000000000002}]},{"osd":8,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.53000000000000003}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.61299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":8,"up_from":56,"seq":240518168602,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":937984,"data_stored":457923,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56299999999999994}]},{"osd":1,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.82099999999999995}]},{"osd":2,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.753},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.88}]},{"osd":6,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.90000000000000002}]},{"osd":7,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":1.0289999999999999}]},{"osd":9,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.97299999999999998}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.0509999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397533,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":978944,"data_stored":1048709,"data_compressed":47273,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.32100000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.65200000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.45800000000000002,"15min":0.45300000000000001},"min":{"1min":0.30499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.62}]},{"osd":2,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.93999999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75800000000000001}]},{"osd":3,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.443,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84799999999999998,"5min":1.143,"15min":1.143},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.63,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80900000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82199999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95799999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.69599999999999995,"15min":0.67400000000000004},"min":{"1min":0.438,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.5630000000000002,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.54600000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.88200000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64600000000000002}]},{"osd":8,"last update":"Fri May 10 04:02:03 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":15.521000000000001,"5min":15.521000000000001,"15min":15.521000000000001},"last":0.747},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":16.306999999999999,"5min":16.306999999999999,"15min":16.306999999999999},"last":0.85299999999999998}]},{"osd":9,"last update":"Fri May 10 04:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561055,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":917504,"data_stored":457185,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.38800000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.38900000000000001},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.48399999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71299999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.754,"5min":1.087,"15min":1.087},"last":0.64800000000000002}]},{"osd":3,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.47499999999999998,"15min":0.46700000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.36699999999999999}]},{"osd":4,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88500000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.63},"min":{"1min":0.44,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93700000000000006,"5min":1.018,"15min":1.018},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri May 10 04:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83299999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.67800000000000005,"15min":0.67300000000000004},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86399999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56499999999999995}]},{"osd":8,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.73199999999999998}]},{"osd":9,"last update":"Fri May 10 04:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.63}]},{"osd":10,"last update":"Fri May 10 04:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":929792,"data_stored":460981,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.32300000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.873,"15min":0.873},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri May 10 04:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.40400000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.52500000000000002,"15min":0.51100000000000001},"min":{"1min":0.38300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79800000000000004,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.70299999999999996}]},{"osd":3,"last update":"Fri May 10 04:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.53800000000000003,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.54800000000000004,"15min":0.53100000000000003},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri May 10 04:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.45700000000000002,"15min":0.44900000000000001},"min":{"1min":0.33800000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.36699999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.73899999999999999}]},{"osd":5,"last update":"Fri May 10 04:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.44900000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.54100000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.63200000000000001,"15min":0.61499999999999999},"min":{"1min":0.55500000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.84099999999999997}]},{"osd":7,"last update":"Fri May 10 04:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.58899999999999997,"15min":0.57399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.433,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.89700000000000002}]},{"osd":8,"last update":"Fri May 10 04:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.876}]},{"osd":9,"last update":"Fri May 10 04:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.94699999999999995}]},{"osd":10,"last update":"Fri May 10 04:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.97399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":458663,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.41899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51500000000000001},"min":{"1min":0.35899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri May 10 04:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54300000000000004,"15min":0.52500000000000002},"min":{"1min":0.40100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.77300000000000002}]},{"osd":3,"last update":"Fri May 10 04:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40799999999999997},"min":{"1min":0.26000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.58599999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.82699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri May 10 04:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri May 10 04:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59199999999999997,"15min":0.58099999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri May 10 04:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri May 10 04:01:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.69799999999999995}]},{"osd":8,"last update":"Fri May 10 04:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.84299999999999997}]},{"osd":9,"last update":"Fri May 10 04:02:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.751}]},{"osd":10,"last update":"Fri May 10 04:02:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.79100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":962560,"data_stored":469278,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.38800000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.39500000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72099999999999997,"5min":0.749,"15min":0.749},"last":0.74099999999999999}]},{"osd":1,"last update":"Fri May 10 04:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54400000000000004,"15min":0.52700000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79000000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57199999999999995,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.496}]},{"osd":3,"last update":"Fri May 10 04:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.47899999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.44400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.876}]},{"osd":4,"last update":"Fri May 10 04:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.41799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.318,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri May 10 04:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.501,"15min":0.5},"min":{"1min":0.29199999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.628},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80600000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.82799999999999996}]},{"osd":6,"last update":"Fri May 10 04:02:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.44400000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.86199999999999999}]},{"osd":7,"last update":"Fri May 10 04:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.93799999999999994}]},{"osd":8,"last update":"Fri May 10 04:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.95199999999999996}]},{"osd":9,"last update":"Fri May 10 04:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.91900000000000004}]},{"osd":10,"last update":"Fri May 10 04:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182443,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":457379,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.35399999999999998},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.46400000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36099999999999999},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.46800000000000003}]},{"osd":1,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.505,"15min":0.51100000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.59999999999999998,"5min":0.747,"15min":0.747},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69799999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69899999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78000000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.74199999999999999}]},{"osd":4,"last update":"Fri May 10 04:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58999999999999997,"15min":0.57999999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri May 10 04:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61899999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.51000000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri May 10 04:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri May 10 04:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.80500000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72799999999999998}]},{"osd":8,"last update":"Fri May 10 04:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.439}]},{"osd":9,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.75900000000000001}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.69899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953512,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":458174,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.33900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50900000000000001,"15min":0.502},"min":{"1min":0.371,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.79000000000000004}]},{"osd":1,"last update":"Fri May 10 04:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.39000000000000001},"min":{"1min":0.29599999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.66400000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.40899999999999997,"15min":0.38800000000000001},"min":{"1min":0.31900000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.65200000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri May 10 04:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60399999999999998,"15min":0.56399999999999995},"min":{"1min":0.46600000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.95099999999999996,"5min":1.046,"15min":1.046},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75900000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.65200000000000002}]},{"osd":3,"last update":"Fri May 10 04:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56999999999999995,"15min":0.54100000000000004},"min":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53700000000000003}]},{"osd":5,"last update":"Fri May 10 04:02:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.66400000000000003,"15min":0.65700000000000003},"min":{"1min":0.45500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.39700000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.81699999999999995}]},{"osd":6,"last update":"Fri May 10 04:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.47799999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70899999999999996,"15min":0.69999999999999996},"min":{"1min":0.503,"5min":0.439,"15min":0.439},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.89700000000000002}]},{"osd":7,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96499999999999997,"5min":1.01,"15min":1.01},"last":0.85099999999999998}]},{"osd":8,"last update":"Fri May 10 04:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.82999999999999996}]},{"osd":9,"last update":"Fri May 10 04:02:23 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.66900000000000004}]},{"osd":10,"last update":"Fri May 10 04:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.80200000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":5,"up_from":37,"seq":158913789987,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":991232,"data_stored":1048650,"data_compressed":47273,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.50600000000000001,"15min":0.49199999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39800000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79300000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri May 10 04:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.42199999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri May 10 04:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.25900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.39100000000000001}]},{"osd":3,"last update":"Fri May 10 04:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82499999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri May 10 04:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60699999999999998,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.58399999999999996},"min":{"1min":0.39000000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.439,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.48199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.69099999999999995,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.54400000000000004,"5min":0.505,"15min":0.505},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.72999999999999998}]},{"osd":8,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.64200000000000002}]},{"osd":9,"last update":"Fri May 10 04:02:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.78600000000000003}]},{"osd":10,"last update":"Fri May 10 04:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.82499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":24792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:03:00.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T04:03:00.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:00 smithi181 ceph-mon[46602]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:00.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:00 smithi181 ceph-mon[46602]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:00.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:00 smithi181 ceph-mon[46602]: pgmap v240: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-05-10T04:03:00.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:00 smithi181 ceph-mon[46602]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-10T04:03:00.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:00 smithi181 ceph-mon[46602]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:00.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:00 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1765318820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:00.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:00 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1639623044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:00.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:00 smithi161 ceph-mon[46918]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:00.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:00 smithi161 ceph-mon[46918]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:00.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:00 smithi161 ceph-mon[46918]: pgmap v240: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-05-10T04:03:00.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:00 smithi161 ceph-mon[46918]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-10T04:03:00.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:00 smithi161 ceph-mon[46918]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:00.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:00 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1765318820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:00.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:00 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1639623044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:00.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.566+0000 7f88983db700 1 -- 172.21.15.150:0/3398293715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8890102630 msgr2=0x7f8890108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:00.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.566+0000 7f88983db700 1 --2- 172.21.15.150:0/3398293715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8890102630 0x7f8890108ee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f888800b0a0 tx=0x7f888800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.566+0000 7f88983db700 1 -- 172.21.15.150:0/3398293715 shutdown_connections 2024-05-10T04:03:00.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.566+0000 7f88983db700 1 --2- 172.21.15.150:0/3398293715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8890102630 0x7f8890108ee0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.566+0000 7f88983db700 1 --2- 172.21.15.150:0/3398293715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8890100b70 0x7f8890100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.566+0000 7f88983db700 1 --2- 172.21.15.150:0/3398293715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8890101c90 0x7f88901020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.566+0000 7f88983db700 1 -- 172.21.15.150:0/3398293715 >> 172.21.15.150:0/3398293715 conn(0x7f88900fc180 msgr2=0x7f88900fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:00.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.566+0000 7f88983db700 1 -- 172.21.15.150:0/3398293715 shutdown_connections 2024-05-10T04:03:00.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.566+0000 7f88983db700 1 -- 172.21.15.150:0/3398293715 wait complete. 2024-05-10T04:03:00.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f88983db700 1 Processor -- start 2024-05-10T04:03:00.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f88983db700 1 -- start start 2024-05-10T04:03:00.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f88983db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8890100b70 0x7f8890196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:00.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f88983db700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8890101c90 0x7f88901974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:00.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f88983db700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8890102630 0x7f889019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:00.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f88983db700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889019ea50 con 0x7f8890102630 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f88983db700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889019eb90 con 0x7f8890101c90 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f88983db700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889019ed20 con 0x7f8890100b70 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f8896978700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8890102630 0x7f889019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f8896978700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8890102630 0x7f889019e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57644/0 (socket says 172.21.15.150:57644) 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f8896978700 1 -- 172.21.15.150:0/315023333 learned_addr learned my addr 172.21.15.150:0/315023333 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f8896177700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8890100b70 0x7f8890196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f8895976700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8890101c90 0x7f88901974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f8896978700 1 -- 172.21.15.150:0/315023333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8890100b70 msgr2=0x7f8890196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f8896978700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8890100b70 0x7f8890196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f8896978700 1 -- 172.21.15.150:0/315023333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8890101c90 msgr2=0x7f88901974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f8896978700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8890101c90 0x7f88901974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f8896978700 1 -- 172.21.15.150:0/315023333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8888009d00 con 0x7f8890102630 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f8895976700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8890101c90 0x7f88901974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:00.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.567+0000 7f8896978700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8890102630 0x7f889019e480 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f888800b070 tx=0x7f8888009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:00.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.568+0000 7f88877fe700 1 -- 172.21.15.150:0/315023333 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888800e040 con 0x7f8890102630 2024-05-10T04:03:00.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.568+0000 7f88877fe700 1 -- 172.21.15.150:0/315023333 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8888004d60 con 0x7f8890102630 2024-05-10T04:03:00.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.568+0000 7f88983db700 1 -- 172.21.15.150:0/315023333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889019efa0 con 0x7f8890102630 2024-05-10T04:03:00.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.568+0000 7f88877fe700 1 -- 172.21.15.150:0/315023333 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888801d480 con 0x7f8890102630 2024-05-10T04:03:00.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.568+0000 7f88983db700 1 -- 172.21.15.150:0/315023333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889019f3c0 con 0x7f8890102630 2024-05-10T04:03:00.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.569+0000 7f88983db700 1 -- 172.21.15.150:0/315023333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8890107d80 con 0x7f8890102630 2024-05-10T04:03:00.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.570+0000 7f88877fe700 1 -- 172.21.15.150:0/315023333 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f8888003a00 con 0x7f8890102630 2024-05-10T04:03:00.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.570+0000 7f88877fe700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f887c0a1230 0x7f887c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:00.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.570+0000 7f88877fe700 1 -- 172.21.15.150:0/315023333 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f88880ce310 con 0x7f8890102630 2024-05-10T04:03:00.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.570+0000 7f8896177700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f887c0a1230 0x7f887c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:00.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.571+0000 7f8896177700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f887c0a1230 0x7f887c0a3740 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8890101b90 tx=0x7f888c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:00.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.575+0000 7f88877fe700 1 -- 172.21.15.150:0/315023333 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f88880977e0 con 0x7f8890102630 2024-05-10T04:03:00.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.770+0000 7f88983db700 1 -- 172.21.15.150:0/315023333 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f889002cf80 con 0x7f887c0a1230 2024-05-10T04:03:00.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.782+0000 7f88877fe700 1 -- 172.21.15.150:0/315023333 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418643 (secure 0 0 0) 0x7f889002cf80 con 0x7f887c0a1230 2024-05-10T04:03:00.786 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:00.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.786+0000 7f88983db700 1 -- 172.21.15.150:0/315023333 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f887c0a1230 msgr2=0x7f887c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:00.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.786+0000 7f88983db700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f887c0a1230 0x7f887c0a3740 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8890101b90 tx=0x7f888c00a3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.786+0000 7f88983db700 1 -- 172.21.15.150:0/315023333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8890102630 msgr2=0x7f889019e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:00.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.786+0000 7f88983db700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8890102630 0x7f889019e480 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f888800b070 tx=0x7f8888009440 comp rx=0 tx=0).stop 2024-05-10T04:03:00.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.786+0000 7f88983db700 1 -- 172.21.15.150:0/315023333 shutdown_connections 2024-05-10T04:03:00.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.786+0000 7f88983db700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f887c0a1230 0x7f887c0a3740 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.786+0000 7f88983db700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8890102630 0x7f889019e480 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.786+0000 7f88983db700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8890100b70 0x7f8890196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.786+0000 7f88983db700 1 --2- 172.21.15.150:0/315023333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8890101c90 0x7f88901974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:00.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.786+0000 7f88983db700 1 -- 172.21.15.150:0/315023333 >> 172.21.15.150:0/315023333 conn(0x7f88900fc180 msgr2=0x7f8890104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:00.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.786+0000 7f88983db700 1 -- 172.21.15.150:0/315023333 shutdown_connections 2024-05-10T04:03:00.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:00.786+0000 7f88983db700 1 -- 172.21.15.150:0/315023333 wait complete. 2024-05-10T04:03:00.797 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:03:00.801 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-05-10T04:02:59.085237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597891,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":142,"num_read_kb":123,"num_write":293,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372176,"kb_used_data":11104,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569648,"statfs":{"total":1151940427776,"available":1151559319552,"internally_reserved":0,"allocated":11370496,"data_stored":7283974,"data_compressed":246795,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3070,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":65,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003642"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640492+0000","last_change":"2024-05-10T04:02:31.642089+0000","last_active":"2024-05-10T04:02:35.640492+0000","last_peered":"2024-05-10T04:02:35.640492+0000","last_clean":"2024-05-10T04:02:35.640492+0000","last_became_active":"2024-05-10T04:02:31.641080+0000","last_became_peered":"2024-05-10T04:02:31.641080+0000","last_unstale":"2024-05-10T04:02:35.640492+0000","last_undegraded":"2024-05-10T04:02:35.640492+0000","last_fullsized":"2024-05-10T04:02:35.640492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:29.693139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636966+0000","last_change":"2024-05-10T04:02:31.633056+0000","last_active":"2024-05-10T04:02:35.636966+0000","last_peered":"2024-05-10T04:02:35.636966+0000","last_clean":"2024-05-10T04:02:35.636966+0000","last_became_active":"2024-05-10T04:02:31.632725+0000","last_became_peered":"2024-05-10T04:02:31.632725+0000","last_unstale":"2024-05-10T04:02:35.636966+0000","last_undegraded":"2024-05-10T04:02:35.636966+0000","last_fullsized":"2024-05-10T04:02:35.636966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:08:40.365433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636999+0000","last_change":"2024-05-10T04:02:31.625219+0000","last_active":"2024-05-10T04:02:35.636999+0000","last_peered":"2024-05-10T04:02:35.636999+0000","last_clean":"2024-05-10T04:02:35.636999+0000","last_became_active":"2024-05-10T04:02:31.624889+0000","last_became_peered":"2024-05-10T04:02:31.624889+0000","last_unstale":"2024-05-10T04:02:35.636999+0000","last_undegraded":"2024-05-10T04:02:35.636999+0000","last_fullsized":"2024-05-10T04:02:35.636999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:10:59.081031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635079+0000","last_change":"2024-05-10T04:02:31.640651+0000","last_active":"2024-05-10T04:02:35.635079+0000","last_peered":"2024-05-10T04:02:35.635079+0000","last_clean":"2024-05-10T04:02:35.635079+0000","last_became_active":"2024-05-10T04:02:31.640242+0000","last_became_peered":"2024-05-10T04:02:31.640242+0000","last_unstale":"2024-05-10T04:02:35.635079+0000","last_undegraded":"2024-05-10T04:02:35.635079+0000","last_fullsized":"2024-05-10T04:02:35.635079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:07.562099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635834+0000","last_change":"2024-05-10T04:02:31.630688+0000","last_active":"2024-05-10T04:02:35.635834+0000","last_peered":"2024-05-10T04:02:35.635834+0000","last_clean":"2024-05-10T04:02:35.635834+0000","last_became_active":"2024-05-10T04:02:31.629252+0000","last_became_peered":"2024-05-10T04:02:31.629252+0000","last_unstale":"2024-05-10T04:02:35.635834+0000","last_undegraded":"2024-05-10T04:02:35.635834+0000","last_fullsized":"2024-05-10T04:02:35.635834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:37.647516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.898651+0000","last_change":"2024-05-10T04:02:31.642586+0000","last_active":"2024-05-10T04:02:35.898651+0000","last_peered":"2024-05-10T04:02:35.898651+0000","last_clean":"2024-05-10T04:02:35.898651+0000","last_became_active":"2024-05-10T04:02:31.642244+0000","last_became_peered":"2024-05-10T04:02:31.642244+0000","last_unstale":"2024-05-10T04:02:35.898651+0000","last_undegraded":"2024-05-10T04:02:35.898651+0000","last_fullsized":"2024-05-10T04:02:35.898651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:28.020151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.638641+0000","last_change":"2024-05-10T04:02:31.636706+0000","last_active":"2024-05-10T04:02:35.638641+0000","last_peered":"2024-05-10T04:02:35.638641+0000","last_clean":"2024-05-10T04:02:35.638641+0000","last_became_active":"2024-05-10T04:02:31.636303+0000","last_became_peered":"2024-05-10T04:02:31.636303+0000","last_unstale":"2024-05-10T04:02:35.638641+0000","last_undegraded":"2024-05-10T04:02:35.638641+0000","last_fullsized":"2024-05-10T04:02:35.638641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:16.941400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956821+0000","last_change":"2024-05-10T04:02:31.640032+0000","last_active":"2024-05-10T04:02:35.956821+0000","last_peered":"2024-05-10T04:02:35.956821+0000","last_clean":"2024-05-10T04:02:35.956821+0000","last_became_active":"2024-05-10T04:02:31.639733+0000","last_became_peered":"2024-05-10T04:02:31.639733+0000","last_unstale":"2024-05-10T04:02:35.956821+0000","last_undegraded":"2024-05-10T04:02:35.956821+0000","last_fullsized":"2024-05-10T04:02:35.956821+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:02:36.963414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641239+0000","last_change":"2024-05-10T04:02:31.638940+0000","last_active":"2024-05-10T04:02:35.641239+0000","last_peered":"2024-05-10T04:02:35.641239+0000","last_clean":"2024-05-10T04:02:35.641239+0000","last_became_active":"2024-05-10T04:02:31.638617+0000","last_became_peered":"2024-05-10T04:02:31.638617+0000","last_unstale":"2024-05-10T04:02:35.641239+0000","last_undegraded":"2024-05-10T04:02:35.641239+0000","last_fullsized":"2024-05-10T04:02:35.641239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:10:36.348117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.899440+0000","last_change":"2024-05-10T04:02:31.635702+0000","last_active":"2024-05-10T04:02:35.899440+0000","last_peered":"2024-05-10T04:02:35.899440+0000","last_clean":"2024-05-10T04:02:35.899440+0000","last_became_active":"2024-05-10T04:02:31.635330+0000","last_became_peered":"2024-05-10T04:02:31.635330+0000","last_unstale":"2024-05-10T04:02:35.899440+0000","last_undegraded":"2024-05-10T04:02:35.899440+0000","last_fullsized":"2024-05-10T04:02:35.899440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:21:15.160340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641553+0000","last_change":"2024-05-10T04:02:31.628252+0000","last_active":"2024-05-10T04:02:35.641553+0000","last_peered":"2024-05-10T04:02:35.641553+0000","last_clean":"2024-05-10T04:02:35.641553+0000","last_became_active":"2024-05-10T04:02:31.627394+0000","last_became_peered":"2024-05-10T04:02:31.627394+0000","last_unstale":"2024-05-10T04:02:35.641553+0000","last_undegraded":"2024-05-10T04:02:35.641553+0000","last_fullsized":"2024-05-10T04:02:35.641553+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:54.569786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635041+0000","last_change":"2024-05-10T04:02:31.641788+0000","last_active":"2024-05-10T04:02:35.635041+0000","last_peered":"2024-05-10T04:02:35.635041+0000","last_clean":"2024-05-10T04:02:35.635041+0000","last_became_active":"2024-05-10T04:02:31.641328+0000","last_became_peered":"2024-05-10T04:02:31.641328+0000","last_unstale":"2024-05-10T04:02:35.635041+0000","last_undegraded":"2024-05-10T04:02:35.635041+0000","last_fullsized":"2024-05-10T04:02:35.635041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:10:55.920245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.631051+0000","last_change":"2024-05-10T04:02:31.631919+0000","last_active":"2024-05-10T04:02:35.631051+0000","last_peered":"2024-05-10T04:02:35.631051+0000","last_clean":"2024-05-10T04:02:35.631051+0000","last_became_active":"2024-05-10T04:02:31.631592+0000","last_became_peered":"2024-05-10T04:02:31.631592+0000","last_unstale":"2024-05-10T04:02:35.631051+0000","last_undegraded":"2024-05-10T04:02:35.631051+0000","last_fullsized":"2024-05-10T04:02:35.631051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:59:03.792090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636548+0000","last_change":"2024-05-10T04:02:31.633288+0000","last_active":"2024-05-10T04:02:35.636548+0000","last_peered":"2024-05-10T04:02:35.636548+0000","last_clean":"2024-05-10T04:02:35.636548+0000","last_became_active":"2024-05-10T04:02:31.632582+0000","last_became_peered":"2024-05-10T04:02:31.632582+0000","last_unstale":"2024-05-10T04:02:35.636548+0000","last_undegraded":"2024-05-10T04:02:35.636548+0000","last_fullsized":"2024-05-10T04:02:35.636548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:27.991439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636432+0000","last_change":"2024-05-10T04:02:31.635833+0000","last_active":"2024-05-10T04:02:35.636432+0000","last_peered":"2024-05-10T04:02:35.636432+0000","last_clean":"2024-05-10T04:02:35.636432+0000","last_became_active":"2024-05-10T04:02:31.635520+0000","last_became_peered":"2024-05-10T04:02:31.635520+0000","last_unstale":"2024-05-10T04:02:35.636432+0000","last_undegraded":"2024-05-10T04:02:35.636432+0000","last_fullsized":"2024-05-10T04:02:35.636432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:40:06.312819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.639301+0000","last_change":"2024-05-10T04:02:31.631734+0000","last_active":"2024-05-10T04:02:35.639301+0000","last_peered":"2024-05-10T04:02:35.639301+0000","last_clean":"2024-05-10T04:02:35.639301+0000","last_became_active":"2024-05-10T04:02:31.631375+0000","last_became_peered":"2024-05-10T04:02:31.631375+0000","last_unstale":"2024-05-10T04:02:35.639301+0000","last_undegraded":"2024-05-10T04:02:35.639301+0000","last_fullsized":"2024-05-10T04:02:35.639301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:25:13.815406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640254+0000","last_change":"2024-05-10T04:02:31.642292+0000","last_active":"2024-05-10T04:02:35.640254+0000","last_peered":"2024-05-10T04:02:35.640254+0000","last_clean":"2024-05-10T04:02:35.640254+0000","last_became_active":"2024-05-10T04:02:31.641570+0000","last_became_peered":"2024-05-10T04:02:31.641570+0000","last_unstale":"2024-05-10T04:02:35.640254+0000","last_undegraded":"2024-05-10T04:02:35.640254+0000","last_fullsized":"2024-05-10T04:02:35.640254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:16:33.704809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.646273+0000","last_change":"2024-05-10T04:02:31.625695+0000","last_active":"2024-05-10T04:02:35.646273+0000","last_peered":"2024-05-10T04:02:35.646273+0000","last_clean":"2024-05-10T04:02:35.646273+0000","last_became_active":"2024-05-10T04:02:31.625393+0000","last_became_peered":"2024-05-10T04:02:31.625393+0000","last_unstale":"2024-05-10T04:02:35.646273+0000","last_undegraded":"2024-05-10T04:02:35.646273+0000","last_fullsized":"2024-05-10T04:02:35.646273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:53.955921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.633141+0000","last_change":"2024-05-10T04:02:31.629152+0000","last_active":"2024-05-10T04:02:35.633141+0000","last_peered":"2024-05-10T04:02:35.633141+0000","last_clean":"2024-05-10T04:02:35.633141+0000","last_became_active":"2024-05-10T04:02:31.628778+0000","last_became_peered":"2024-05-10T04:02:31.628778+0000","last_unstale":"2024-05-10T04:02:35.633141+0000","last_undegraded":"2024-05-10T04:02:35.633141+0000","last_fullsized":"2024-05-10T04:02:35.633141+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:39.772500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.409997+0000","last_change":"2024-05-10T04:02:35.650536+0000","last_active":"2024-05-10T04:02:36.409997+0000","last_peered":"2024-05-10T04:02:36.409997+0000","last_clean":"2024-05-10T04:02:36.409997+0000","last_became_active":"2024-05-10T04:02:35.650102+0000","last_became_peered":"2024-05-10T04:02:35.650102+0000","last_unstale":"2024-05-10T04:02:36.409997+0000","last_undegraded":"2024-05-10T04:02:36.409997+0000","last_fullsized":"2024-05-10T04:02:36.409997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:19:35.605844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182721+0000","last_change":"2024-05-10T04:02:31.186846+0000","last_active":"2024-05-10T04:02:56.182721+0000","last_peered":"2024-05-10T04:02:56.182721+0000","last_clean":"2024-05-10T04:02:56.182721+0000","last_became_active":"2024-05-10T04:02:31.186497+0000","last_became_peered":"2024-05-10T04:02:31.186497+0000","last_unstale":"2024-05-10T04:02:56.182721+0000","last_undegraded":"2024-05-10T04:02:56.182721+0000","last_fullsized":"2024-05-10T04:02:56.182721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:15:40.832455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956915+0000","last_change":"2024-05-10T04:02:31.631278+0000","last_active":"2024-05-10T04:02:35.956915+0000","last_peered":"2024-05-10T04:02:35.956915+0000","last_clean":"2024-05-10T04:02:35.956915+0000","last_became_active":"2024-05-10T04:02:31.630979+0000","last_became_peered":"2024-05-10T04:02:31.630979+0000","last_unstale":"2024-05-10T04:02:35.956915+0000","last_undegraded":"2024-05-10T04:02:35.956915+0000","last_fullsized":"2024-05-10T04:02:35.956915+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:50.227115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641619+0000","last_change":"2024-05-10T04:02:35.651622+0000","last_active":"2024-05-10T04:02:36.641619+0000","last_peered":"2024-05-10T04:02:36.641619+0000","last_clean":"2024-05-10T04:02:36.641619+0000","last_became_active":"2024-05-10T04:02:35.651148+0000","last_became_peered":"2024-05-10T04:02:35.651148+0000","last_unstale":"2024-05-10T04:02:36.641619+0000","last_undegraded":"2024-05-10T04:02:36.641619+0000","last_fullsized":"2024-05-10T04:02:36.641619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:40:09.019286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"80'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.644547+0000","last_change":"2024-05-10T04:02:31.187785+0000","last_active":"2024-05-10T04:02:35.644547+0000","last_peered":"2024-05-10T04:02:35.644547+0000","last_clean":"2024-05-10T04:02:35.644547+0000","last_became_active":"2024-05-10T04:02:31.187499+0000","last_became_peered":"2024-05-10T04:02:31.187499+0000","last_unstale":"2024-05-10T04:02:35.644547+0000","last_undegraded":"2024-05-10T04:02:35.644547+0000","last_fullsized":"2024-05-10T04:02:35.644547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:05:29.015767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636966+0000","last_change":"2024-05-10T04:02:31.624433+0000","last_active":"2024-05-10T04:02:35.636966+0000","last_peered":"2024-05-10T04:02:35.636966+0000","last_clean":"2024-05-10T04:02:35.636966+0000","last_became_active":"2024-05-10T04:02:31.624115+0000","last_became_peered":"2024-05-10T04:02:31.624115+0000","last_unstale":"2024-05-10T04:02:35.636966+0000","last_undegraded":"2024-05-10T04:02:35.636966+0000","last_fullsized":"2024-05-10T04:02:35.636966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:54.584681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641726+0000","last_change":"2024-05-10T04:02:35.651535+0000","last_active":"2024-05-10T04:02:36.641726+0000","last_peered":"2024-05-10T04:02:36.641726+0000","last_clean":"2024-05-10T04:02:36.641726+0000","last_became_active":"2024-05-10T04:02:35.651077+0000","last_became_peered":"2024-05-10T04:02:35.651077+0000","last_unstale":"2024-05-10T04:02:36.641726+0000","last_undegraded":"2024-05-10T04:02:36.641726+0000","last_fullsized":"2024-05-10T04:02:36.641726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:21:04.647730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.180078+0000","last_change":"2024-05-10T04:02:31.195666+0000","last_active":"2024-05-10T04:02:56.180078+0000","last_peered":"2024-05-10T04:02:56.180078+0000","last_clean":"2024-05-10T04:02:56.180078+0000","last_became_active":"2024-05-10T04:02:31.195285+0000","last_became_peered":"2024-05-10T04:02:31.195285+0000","last_unstale":"2024-05-10T04:02:56.180078+0000","last_undegraded":"2024-05-10T04:02:56.180078+0000","last_fullsized":"2024-05-10T04:02:56.180078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:02.906701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640666+0000","last_change":"2024-05-10T04:02:31.628124+0000","last_active":"2024-05-10T04:02:35.640666+0000","last_peered":"2024-05-10T04:02:35.640666+0000","last_clean":"2024-05-10T04:02:35.640666+0000","last_became_active":"2024-05-10T04:02:31.627789+0000","last_became_peered":"2024-05-10T04:02:31.627789+0000","last_unstale":"2024-05-10T04:02:35.640666+0000","last_undegraded":"2024-05-10T04:02:35.640666+0000","last_fullsized":"2024-05-10T04:02:35.640666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:40:11.240726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641791+0000","last_change":"2024-05-10T04:02:35.649536+0000","last_active":"2024-05-10T04:02:36.641791+0000","last_peered":"2024-05-10T04:02:36.641791+0000","last_clean":"2024-05-10T04:02:36.641791+0000","last_became_active":"2024-05-10T04:02:35.649130+0000","last_became_peered":"2024-05-10T04:02:35.649130+0000","last_unstale":"2024-05-10T04:02:36.641791+0000","last_undegraded":"2024-05-10T04:02:36.641791+0000","last_fullsized":"2024-05-10T04:02:36.641791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:03.870071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637193+0000","last_change":"2024-05-10T04:02:31.192627+0000","last_active":"2024-05-10T04:02:35.637193+0000","last_peered":"2024-05-10T04:02:35.637193+0000","last_clean":"2024-05-10T04:02:35.637193+0000","last_became_active":"2024-05-10T04:02:31.191433+0000","last_became_peered":"2024-05-10T04:02:31.191433+0000","last_unstale":"2024-05-10T04:02:35.637193+0000","last_undegraded":"2024-05-10T04:02:35.637193+0000","last_fullsized":"2024-05-10T04:02:35.637193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:29.235483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641519+0000","last_change":"2024-05-10T04:02:31.641192+0000","last_active":"2024-05-10T04:02:35.641519+0000","last_peered":"2024-05-10T04:02:35.641519+0000","last_clean":"2024-05-10T04:02:35.641519+0000","last_became_active":"2024-05-10T04:02:31.640540+0000","last_became_peered":"2024-05-10T04:02:31.640540+0000","last_unstale":"2024-05-10T04:02:35.641519+0000","last_undegraded":"2024-05-10T04:02:35.641519+0000","last_fullsized":"2024-05-10T04:02:35.641519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:36.734231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.410181+0000","last_change":"2024-05-10T04:02:35.644661+0000","last_active":"2024-05-10T04:02:36.410181+0000","last_peered":"2024-05-10T04:02:36.410181+0000","last_clean":"2024-05-10T04:02:36.410181+0000","last_became_active":"2024-05-10T04:02:35.644216+0000","last_became_peered":"2024-05-10T04:02:35.644216+0000","last_unstale":"2024-05-10T04:02:36.410181+0000","last_undegraded":"2024-05-10T04:02:36.410181+0000","last_fullsized":"2024-05-10T04:02:36.410181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:02.618922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.175086+0000","last_change":"2024-05-10T04:02:31.198037+0000","last_active":"2024-05-10T04:02:55.175086+0000","last_peered":"2024-05-10T04:02:55.175086+0000","last_clean":"2024-05-10T04:02:55.175086+0000","last_became_active":"2024-05-10T04:02:31.197679+0000","last_became_peered":"2024-05-10T04:02:31.197679+0000","last_unstale":"2024-05-10T04:02:55.175086+0000","last_undegraded":"2024-05-10T04:02:55.175086+0000","last_fullsized":"2024-05-10T04:02:55.175086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:19.059645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636057+0000","last_change":"2024-05-10T04:02:31.630020+0000","last_active":"2024-05-10T04:02:35.636057+0000","last_peered":"2024-05-10T04:02:35.636057+0000","last_clean":"2024-05-10T04:02:35.636057+0000","last_became_active":"2024-05-10T04:02:31.629738+0000","last_became_peered":"2024-05-10T04:02:31.629738+0000","last_unstale":"2024-05-10T04:02:35.636057+0000","last_undegraded":"2024-05-10T04:02:35.636057+0000","last_fullsized":"2024-05-10T04:02:35.636057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:58:42.317143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.410114+0000","last_change":"2024-05-10T04:02:35.647242+0000","last_active":"2024-05-10T04:02:36.410114+0000","last_peered":"2024-05-10T04:02:36.410114+0000","last_clean":"2024-05-10T04:02:36.410114+0000","last_became_active":"2024-05-10T04:02:35.646774+0000","last_became_peered":"2024-05-10T04:02:35.646774+0000","last_unstale":"2024-05-10T04:02:36.410114+0000","last_undegraded":"2024-05-10T04:02:36.410114+0000","last_fullsized":"2024-05-10T04:02:36.410114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:20:05.602820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"80'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.898649+0000","last_change":"2024-05-10T04:02:31.191981+0000","last_active":"2024-05-10T04:02:35.898649+0000","last_peered":"2024-05-10T04:02:35.898649+0000","last_clean":"2024-05-10T04:02:35.898649+0000","last_became_active":"2024-05-10T04:02:31.191632+0000","last_became_peered":"2024-05-10T04:02:31.191632+0000","last_unstale":"2024-05-10T04:02:35.898649+0000","last_undegraded":"2024-05-10T04:02:35.898649+0000","last_fullsized":"2024-05-10T04:02:35.898649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:44:52.744661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640083+0000","last_change":"2024-05-10T04:02:31.643240+0000","last_active":"2024-05-10T04:02:35.640083+0000","last_peered":"2024-05-10T04:02:35.640083+0000","last_clean":"2024-05-10T04:02:35.640083+0000","last_became_active":"2024-05-10T04:02:31.642891+0000","last_became_peered":"2024-05-10T04:02:31.642891+0000","last_unstale":"2024-05-10T04:02:35.640083+0000","last_undegraded":"2024-05-10T04:02:35.640083+0000","last_fullsized":"2024-05-10T04:02:35.640083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:41:42.901192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.639026+0000","last_change":"2024-05-10T04:02:35.652370+0000","last_active":"2024-05-10T04:02:36.639026+0000","last_peered":"2024-05-10T04:02:36.639026+0000","last_clean":"2024-05-10T04:02:36.639026+0000","last_became_active":"2024-05-10T04:02:35.651803+0000","last_became_peered":"2024-05-10T04:02:35.651803+0000","last_unstale":"2024-05-10T04:02:36.639026+0000","last_undegraded":"2024-05-10T04:02:36.639026+0000","last_fullsized":"2024-05-10T04:02:36.639026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:33:36.049311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630907+0000","last_change":"2024-05-10T04:02:31.189053+0000","last_active":"2024-05-10T04:02:35.630907+0000","last_peered":"2024-05-10T04:02:35.630907+0000","last_clean":"2024-05-10T04:02:35.630907+0000","last_became_active":"2024-05-10T04:02:31.188614+0000","last_became_peered":"2024-05-10T04:02:31.188614+0000","last_unstale":"2024-05-10T04:02:35.630907+0000","last_undegraded":"2024-05-10T04:02:35.630907+0000","last_fullsized":"2024-05-10T04:02:35.630907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:59:37.750507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956848+0000","last_change":"2024-05-10T04:02:31.630379+0000","last_active":"2024-05-10T04:02:35.956848+0000","last_peered":"2024-05-10T04:02:35.956848+0000","last_clean":"2024-05-10T04:02:35.956848+0000","last_became_active":"2024-05-10T04:02:31.630094+0000","last_became_peered":"2024-05-10T04:02:31.630094+0000","last_unstale":"2024-05-10T04:02:35.956848+0000","last_undegraded":"2024-05-10T04:02:35.956848+0000","last_fullsized":"2024-05-10T04:02:35.956848+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:05.699923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638588+0000","last_change":"2024-05-10T04:02:35.650770+0000","last_active":"2024-05-10T04:02:36.638588+0000","last_peered":"2024-05-10T04:02:36.638588+0000","last_clean":"2024-05-10T04:02:36.638588+0000","last_became_active":"2024-05-10T04:02:35.650351+0000","last_became_peered":"2024-05-10T04:02:35.650351+0000","last_unstale":"2024-05-10T04:02:36.638588+0000","last_undegraded":"2024-05-10T04:02:36.638588+0000","last_fullsized":"2024-05-10T04:02:36.638588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:41:06.728126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.068369+0000","last_change":"2024-05-10T04:02:31.189557+0000","last_active":"2024-05-10T04:02:36.068369+0000","last_peered":"2024-05-10T04:02:36.068369+0000","last_clean":"2024-05-10T04:02:36.068369+0000","last_became_active":"2024-05-10T04:02:31.189206+0000","last_became_peered":"2024-05-10T04:02:31.189206+0000","last_unstale":"2024-05-10T04:02:36.068369+0000","last_undegraded":"2024-05-10T04:02:36.068369+0000","last_fullsized":"2024-05-10T04:02:36.068369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:11:08.225965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.645298+0000","last_change":"2024-05-10T04:02:31.633561+0000","last_active":"2024-05-10T04:02:35.645298+0000","last_peered":"2024-05-10T04:02:35.645298+0000","last_clean":"2024-05-10T04:02:35.645298+0000","last_became_active":"2024-05-10T04:02:31.633224+0000","last_became_peered":"2024-05-10T04:02:31.633224+0000","last_unstale":"2024-05-10T04:02:35.645298+0000","last_undegraded":"2024-05-10T04:02:35.645298+0000","last_fullsized":"2024-05-10T04:02:35.645298+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:32.901146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956875+0000","last_change":"2024-05-10T04:02:35.649327+0000","last_active":"2024-05-10T04:02:35.956875+0000","last_peered":"2024-05-10T04:02:35.956875+0000","last_clean":"2024-05-10T04:02:35.956875+0000","last_became_active":"2024-05-10T04:02:35.648982+0000","last_became_peered":"2024-05-10T04:02:35.648982+0000","last_unstale":"2024-05-10T04:02:35.956875+0000","last_undegraded":"2024-05-10T04:02:35.956875+0000","last_fullsized":"2024-05-10T04:02:35.956875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:25.405748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.899395+0000","last_change":"2024-05-10T04:02:31.186447+0000","last_active":"2024-05-10T04:02:35.899395+0000","last_peered":"2024-05-10T04:02:35.899395+0000","last_clean":"2024-05-10T04:02:35.899395+0000","last_became_active":"2024-05-10T04:02:31.186073+0000","last_became_peered":"2024-05-10T04:02:31.186073+0000","last_unstale":"2024-05-10T04:02:35.899395+0000","last_undegraded":"2024-05-10T04:02:35.899395+0000","last_fullsized":"2024-05-10T04:02:35.899395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:53.381147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640992+0000","last_change":"2024-05-10T04:02:31.633772+0000","last_active":"2024-05-10T04:02:35.640992+0000","last_peered":"2024-05-10T04:02:35.640992+0000","last_clean":"2024-05-10T04:02:35.640992+0000","last_became_active":"2024-05-10T04:02:31.629592+0000","last_became_peered":"2024-05-10T04:02:31.629592+0000","last_unstale":"2024-05-10T04:02:35.640992+0000","last_undegraded":"2024-05-10T04:02:35.640992+0000","last_fullsized":"2024-05-10T04:02:35.640992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:38:37.342351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.088114+0000","last_change":"2024-05-10T04:02:35.646960+0000","last_active":"2024-05-10T04:02:36.088114+0000","last_peered":"2024-05-10T04:02:36.088114+0000","last_clean":"2024-05-10T04:02:36.088114+0000","last_became_active":"2024-05-10T04:02:35.646598+0000","last_became_peered":"2024-05-10T04:02:35.646598+0000","last_unstale":"2024-05-10T04:02:36.088114+0000","last_undegraded":"2024-05-10T04:02:36.088114+0000","last_fullsized":"2024-05-10T04:02:36.088114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:40:10.478745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.175169+0000","last_change":"2024-05-10T04:02:31.192846+0000","last_active":"2024-05-10T04:02:55.175169+0000","last_peered":"2024-05-10T04:02:55.175169+0000","last_clean":"2024-05-10T04:02:55.175169+0000","last_became_active":"2024-05-10T04:02:31.192514+0000","last_became_peered":"2024-05-10T04:02:31.192514+0000","last_unstale":"2024-05-10T04:02:55.175169+0000","last_undegraded":"2024-05-10T04:02:55.175169+0000","last_fullsized":"2024-05-10T04:02:55.175169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:50:32.649440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637130+0000","last_change":"2024-05-10T04:02:31.641578+0000","last_active":"2024-05-10T04:02:35.637130+0000","last_peered":"2024-05-10T04:02:35.637130+0000","last_clean":"2024-05-10T04:02:35.637130+0000","last_became_active":"2024-05-10T04:02:31.641268+0000","last_became_peered":"2024-05-10T04:02:31.641268+0000","last_unstale":"2024-05-10T04:02:35.637130+0000","last_undegraded":"2024-05-10T04:02:35.637130+0000","last_fullsized":"2024-05-10T04:02:35.637130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:31:00.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641920+0000","last_change":"2024-05-10T04:02:35.646754+0000","last_active":"2024-05-10T04:02:36.641920+0000","last_peered":"2024-05-10T04:02:36.641920+0000","last_clean":"2024-05-10T04:02:36.641920+0000","last_became_active":"2024-05-10T04:02:35.646342+0000","last_became_peered":"2024-05-10T04:02:35.646342+0000","last_unstale":"2024-05-10T04:02:36.641920+0000","last_undegraded":"2024-05-10T04:02:36.641920+0000","last_fullsized":"2024-05-10T04:02:36.641920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:28:33.373312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.068934+0000","last_change":"2024-05-10T04:02:31.186427+0000","last_active":"2024-05-10T04:02:36.068934+0000","last_peered":"2024-05-10T04:02:36.068934+0000","last_clean":"2024-05-10T04:02:36.068934+0000","last_became_active":"2024-05-10T04:02:31.185937+0000","last_became_peered":"2024-05-10T04:02:31.185937+0000","last_unstale":"2024-05-10T04:02:36.068934+0000","last_undegraded":"2024-05-10T04:02:36.068934+0000","last_fullsized":"2024-05-10T04:02:36.068934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.818126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640892+0000","last_change":"2024-05-10T04:02:31.645582+0000","last_active":"2024-05-10T04:02:35.640892+0000","last_peered":"2024-05-10T04:02:35.640892+0000","last_clean":"2024-05-10T04:02:35.640892+0000","last_became_active":"2024-05-10T04:02:31.645259+0000","last_became_peered":"2024-05-10T04:02:31.645259+0000","last_unstale":"2024-05-10T04:02:35.640892+0000","last_undegraded":"2024-05-10T04:02:35.640892+0000","last_fullsized":"2024-05-10T04:02:35.640892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:17:31.818116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.088108+0000","last_change":"2024-05-10T04:02:35.647341+0000","last_active":"2024-05-10T04:02:36.088108+0000","last_peered":"2024-05-10T04:02:36.088108+0000","last_clean":"2024-05-10T04:02:36.088108+0000","last_became_active":"2024-05-10T04:02:35.646806+0000","last_became_peered":"2024-05-10T04:02:35.646806+0000","last_unstale":"2024-05-10T04:02:36.088108+0000","last_undegraded":"2024-05-10T04:02:36.088108+0000","last_fullsized":"2024-05-10T04:02:36.088108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:01:51.252683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:46.792587+0000","last_change":"2024-05-10T04:02:31.195377+0000","last_active":"2024-05-10T04:02:46.792587+0000","last_peered":"2024-05-10T04:02:46.792587+0000","last_clean":"2024-05-10T04:02:46.792587+0000","last_became_active":"2024-05-10T04:02:31.194965+0000","last_became_peered":"2024-05-10T04:02:31.194965+0000","last_unstale":"2024-05-10T04:02:46.792587+0000","last_undegraded":"2024-05-10T04:02:46.792587+0000","last_fullsized":"2024-05-10T04:02:46.792587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:11:09.237525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636559+0000","last_change":"2024-05-10T04:02:31.629600+0000","last_active":"2024-05-10T04:02:35.636559+0000","last_peered":"2024-05-10T04:02:35.636559+0000","last_clean":"2024-05-10T04:02:35.636559+0000","last_became_active":"2024-05-10T04:02:31.629264+0000","last_became_peered":"2024-05-10T04:02:31.629264+0000","last_unstale":"2024-05-10T04:02:35.636559+0000","last_undegraded":"2024-05-10T04:02:35.636559+0000","last_fullsized":"2024-05-10T04:02:35.636559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:45:08.867783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.069212+0000","last_change":"2024-05-10T04:02:35.649468+0000","last_active":"2024-05-10T04:02:36.069212+0000","last_peered":"2024-05-10T04:02:36.069212+0000","last_clean":"2024-05-10T04:02:36.069212+0000","last_became_active":"2024-05-10T04:02:35.649053+0000","last_became_peered":"2024-05-10T04:02:35.649053+0000","last_unstale":"2024-05-10T04:02:36.069212+0000","last_undegraded":"2024-05-10T04:02:36.069212+0000","last_fullsized":"2024-05-10T04:02:36.069212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:01:05.761287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182116+0000","last_change":"2024-05-10T04:02:31.195169+0000","last_active":"2024-05-10T04:02:56.182116+0000","last_peered":"2024-05-10T04:02:56.182116+0000","last_clean":"2024-05-10T04:02:56.182116+0000","last_became_active":"2024-05-10T04:02:31.194720+0000","last_became_peered":"2024-05-10T04:02:31.194720+0000","last_unstale":"2024-05-10T04:02:56.182116+0000","last_undegraded":"2024-05-10T04:02:56.182116+0000","last_fullsized":"2024-05-10T04:02:56.182116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:42.727841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630936+0000","last_change":"2024-05-10T04:02:31.631739+0000","last_active":"2024-05-10T04:02:35.630936+0000","last_peered":"2024-05-10T04:02:35.630936+0000","last_clean":"2024-05-10T04:02:35.630936+0000","last_became_active":"2024-05-10T04:02:31.631401+0000","last_became_peered":"2024-05-10T04:02:31.631401+0000","last_unstale":"2024-05-10T04:02:35.630936+0000","last_undegraded":"2024-05-10T04:02:35.630936+0000","last_fullsized":"2024-05-10T04:02:35.630936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:06.958328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638793+0000","last_change":"2024-05-10T04:02:35.651484+0000","last_active":"2024-05-10T04:02:36.638793+0000","last_peered":"2024-05-10T04:02:36.638793+0000","last_clean":"2024-05-10T04:02:36.638793+0000","last_became_active":"2024-05-10T04:02:35.651069+0000","last_became_peered":"2024-05-10T04:02:35.651069+0000","last_unstale":"2024-05-10T04:02:36.638793+0000","last_undegraded":"2024-05-10T04:02:36.638793+0000","last_fullsized":"2024-05-10T04:02:36.638793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:26:45.229101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.172679+0000","last_change":"2024-05-10T04:02:31.186314+0000","last_active":"2024-05-10T04:02:55.172679+0000","last_peered":"2024-05-10T04:02:55.172679+0000","last_clean":"2024-05-10T04:02:55.172679+0000","last_became_active":"2024-05-10T04:02:31.185861+0000","last_became_peered":"2024-05-10T04:02:31.185861+0000","last_unstale":"2024-05-10T04:02:55.172679+0000","last_undegraded":"2024-05-10T04:02:55.172679+0000","last_fullsized":"2024-05-10T04:02:55.172679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:07.386978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640394+0000","last_change":"2024-05-10T04:02:31.637151+0000","last_active":"2024-05-10T04:02:35.640394+0000","last_peered":"2024-05-10T04:02:35.640394+0000","last_clean":"2024-05-10T04:02:35.640394+0000","last_became_active":"2024-05-10T04:02:31.636745+0000","last_became_peered":"2024-05-10T04:02:31.636745+0000","last_unstale":"2024-05-10T04:02:35.640394+0000","last_undegraded":"2024-05-10T04:02:35.640394+0000","last_fullsized":"2024-05-10T04:02:35.640394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:59.804391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.088363+0000","last_change":"2024-05-10T04:02:35.656108+0000","last_active":"2024-05-10T04:02:36.088363+0000","last_peered":"2024-05-10T04:02:36.088363+0000","last_clean":"2024-05-10T04:02:36.088363+0000","last_became_active":"2024-05-10T04:02:35.655664+0000","last_became_peered":"2024-05-10T04:02:35.655664+0000","last_unstale":"2024-05-10T04:02:36.088363+0000","last_undegraded":"2024-05-10T04:02:36.088363+0000","last_fullsized":"2024-05-10T04:02:36.088363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:10.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:58.198892+0000","last_change":"2024-05-10T04:02:31.189432+0000","last_active":"2024-05-10T04:02:58.198892+0000","last_peered":"2024-05-10T04:02:58.198892+0000","last_clean":"2024-05-10T04:02:58.198892+0000","last_became_active":"2024-05-10T04:02:31.189062+0000","last_became_peered":"2024-05-10T04:02:31.189062+0000","last_unstale":"2024-05-10T04:02:58.198892+0000","last_undegraded":"2024-05-10T04:02:58.198892+0000","last_fullsized":"2024-05-10T04:02:58.198892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:28.452100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.639024+0000","last_change":"2024-05-10T04:02:31.637517+0000","last_active":"2024-05-10T04:02:35.639024+0000","last_peered":"2024-05-10T04:02:35.639024+0000","last_clean":"2024-05-10T04:02:35.639024+0000","last_became_active":"2024-05-10T04:02:31.636518+0000","last_became_peered":"2024-05-10T04:02:31.636518+0000","last_unstale":"2024-05-10T04:02:35.639024+0000","last_undegraded":"2024-05-10T04:02:35.639024+0000","last_fullsized":"2024-05-10T04:02:35.639024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:07:09.732117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.067746+0000","last_change":"2024-05-10T04:02:35.649693+0000","last_active":"2024-05-10T04:02:36.067746+0000","last_peered":"2024-05-10T04:02:36.067746+0000","last_clean":"2024-05-10T04:02:36.067746+0000","last_became_active":"2024-05-10T04:02:35.649175+0000","last_became_peered":"2024-05-10T04:02:35.649175+0000","last_unstale":"2024-05-10T04:02:36.067746+0000","last_undegraded":"2024-05-10T04:02:36.067746+0000","last_fullsized":"2024-05-10T04:02:36.067746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:03:52.340444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.181556+0000","last_change":"2024-05-10T04:02:31.196909+0000","last_active":"2024-05-10T04:02:56.181556+0000","last_peered":"2024-05-10T04:02:56.181556+0000","last_clean":"2024-05-10T04:02:56.181556+0000","last_became_active":"2024-05-10T04:02:31.196599+0000","last_became_peered":"2024-05-10T04:02:31.196599+0000","last_unstale":"2024-05-10T04:02:56.181556+0000","last_undegraded":"2024-05-10T04:02:56.181556+0000","last_fullsized":"2024-05-10T04:02:56.181556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:16:51.483498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641842+0000","last_change":"2024-05-10T04:02:31.641426+0000","last_active":"2024-05-10T04:02:35.641842+0000","last_peered":"2024-05-10T04:02:35.641842+0000","last_clean":"2024-05-10T04:02:35.641842+0000","last_became_active":"2024-05-10T04:02:31.641054+0000","last_became_peered":"2024-05-10T04:02:31.641054+0000","last_unstale":"2024-05-10T04:02:35.641842+0000","last_undegraded":"2024-05-10T04:02:35.641842+0000","last_fullsized":"2024-05-10T04:02:35.641842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:12:48.472674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.068553+0000","last_change":"2024-05-10T04:02:35.653258+0000","last_active":"2024-05-10T04:02:36.068553+0000","last_peered":"2024-05-10T04:02:36.068553+0000","last_clean":"2024-05-10T04:02:36.068553+0000","last_became_active":"2024-05-10T04:02:35.652859+0000","last_became_peered":"2024-05-10T04:02:35.652859+0000","last_unstale":"2024-05-10T04:02:36.068553+0000","last_undegraded":"2024-05-10T04:02:36.068553+0000","last_fullsized":"2024-05-10T04:02:36.068553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:15:57.814963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:57.193415+0000","last_change":"2024-05-10T04:02:31.192231+0000","last_active":"2024-05-10T04:02:57.193415+0000","last_peered":"2024-05-10T04:02:57.193415+0000","last_clean":"2024-05-10T04:02:57.193415+0000","last_became_active":"2024-05-10T04:02:31.191902+0000","last_became_peered":"2024-05-10T04:02:31.191902+0000","last_unstale":"2024-05-10T04:02:57.193415+0000","last_undegraded":"2024-05-10T04:02:57.193415+0000","last_fullsized":"2024-05-10T04:02:57.193415+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:18.637112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.638702+0000","last_change":"2024-05-10T04:02:31.635514+0000","last_active":"2024-05-10T04:02:35.638702+0000","last_peered":"2024-05-10T04:02:35.638702+0000","last_clean":"2024-05-10T04:02:35.638702+0000","last_became_active":"2024-05-10T04:02:31.635171+0000","last_became_peered":"2024-05-10T04:02:31.635171+0000","last_unstale":"2024-05-10T04:02:35.638702+0000","last_undegraded":"2024-05-10T04:02:35.638702+0000","last_fullsized":"2024-05-10T04:02:35.638702+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:48.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.090937+0000","last_change":"2024-05-10T04:02:35.900174+0000","last_active":"2024-05-10T04:02:36.090937+0000","last_peered":"2024-05-10T04:02:36.090937+0000","last_clean":"2024-05-10T04:02:36.090937+0000","last_became_active":"2024-05-10T04:02:35.899653+0000","last_became_peered":"2024-05-10T04:02:35.899653+0000","last_unstale":"2024-05-10T04:02:36.090937+0000","last_undegraded":"2024-05-10T04:02:36.090937+0000","last_fullsized":"2024-05-10T04:02:36.090937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:32:56.962361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.645483+0000","last_change":"2024-05-10T04:02:31.190173+0000","last_active":"2024-05-10T04:02:35.645483+0000","last_peered":"2024-05-10T04:02:35.645483+0000","last_clean":"2024-05-10T04:02:35.645483+0000","last_became_active":"2024-05-10T04:02:31.189840+0000","last_became_peered":"2024-05-10T04:02:31.189840+0000","last_unstale":"2024-05-10T04:02:35.645483+0000","last_undegraded":"2024-05-10T04:02:35.645483+0000","last_fullsized":"2024-05-10T04:02:35.645483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:31:49.382482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.899188+0000","last_change":"2024-05-10T04:02:31.629497+0000","last_active":"2024-05-10T04:02:35.899188+0000","last_peered":"2024-05-10T04:02:35.899188+0000","last_clean":"2024-05-10T04:02:35.899188+0000","last_became_active":"2024-05-10T04:02:31.629182+0000","last_became_peered":"2024-05-10T04:02:31.629182+0000","last_unstale":"2024-05-10T04:02:35.899188+0000","last_undegraded":"2024-05-10T04:02:35.899188+0000","last_fullsized":"2024-05-10T04:02:35.899188+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:11:33.478737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.067644+0000","last_change":"2024-05-10T04:02:35.647249+0000","last_active":"2024-05-10T04:02:36.067644+0000","last_peered":"2024-05-10T04:02:36.067644+0000","last_clean":"2024-05-10T04:02:36.067644+0000","last_became_active":"2024-05-10T04:02:35.646716+0000","last_became_peered":"2024-05-10T04:02:35.646716+0000","last_unstale":"2024-05-10T04:02:36.067644+0000","last_undegraded":"2024-05-10T04:02:36.067644+0000","last_fullsized":"2024-05-10T04:02:36.067644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:44.651836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.639382+0000","last_change":"2024-05-10T04:02:31.191241+0000","last_active":"2024-05-10T04:02:36.639382+0000","last_peered":"2024-05-10T04:02:36.639382+0000","last_clean":"2024-05-10T04:02:36.639382+0000","last_became_active":"2024-05-10T04:02:31.190888+0000","last_became_peered":"2024-05-10T04:02:31.190888+0000","last_unstale":"2024-05-10T04:02:36.639382+0000","last_undegraded":"2024-05-10T04:02:36.639382+0000","last_fullsized":"2024-05-10T04:02:36.639382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:55.256119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.898849+0000","last_change":"2024-05-10T04:02:31.638394+0000","last_active":"2024-05-10T04:02:35.898849+0000","last_peered":"2024-05-10T04:02:35.898849+0000","last_clean":"2024-05-10T04:02:35.898849+0000","last_became_active":"2024-05-10T04:02:31.638025+0000","last_became_peered":"2024-05-10T04:02:31.638025+0000","last_unstale":"2024-05-10T04:02:35.898849+0000","last_undegraded":"2024-05-10T04:02:35.898849+0000","last_fullsized":"2024-05-10T04:02:35.898849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:13.088393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.067640+0000","last_change":"2024-05-10T04:02:35.899898+0000","last_active":"2024-05-10T04:02:36.067640+0000","last_peered":"2024-05-10T04:02:36.067640+0000","last_clean":"2024-05-10T04:02:36.067640+0000","last_became_active":"2024-05-10T04:02:35.899485+0000","last_became_peered":"2024-05-10T04:02:35.899485+0000","last_unstale":"2024-05-10T04:02:36.067640+0000","last_undegraded":"2024-05-10T04:02:36.067640+0000","last_fullsized":"2024-05-10T04:02:36.067640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:26:50.909122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640359+0000","last_change":"2024-05-10T04:02:31.640923+0000","last_active":"2024-05-10T04:02:35.640359+0000","last_peered":"2024-05-10T04:02:35.640359+0000","last_clean":"2024-05-10T04:02:35.640359+0000","last_became_active":"2024-05-10T04:02:31.640490+0000","last_became_peered":"2024-05-10T04:02:31.640490+0000","last_unstale":"2024-05-10T04:02:35.640359+0000","last_undegraded":"2024-05-10T04:02:35.640359+0000","last_fullsized":"2024-05-10T04:02:35.640359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:31:58.585789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630772+0000","last_change":"2024-05-10T04:02:31.191326+0000","last_active":"2024-05-10T04:02:35.630772+0000","last_peered":"2024-05-10T04:02:35.630772+0000","last_clean":"2024-05-10T04:02:35.630772+0000","last_became_active":"2024-05-10T04:02:31.190974+0000","last_became_peered":"2024-05-10T04:02:31.190974+0000","last_unstale":"2024-05-10T04:02:35.630772+0000","last_undegraded":"2024-05-10T04:02:35.630772+0000","last_fullsized":"2024-05-10T04:02:35.630772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:43.670619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.325315+0000","last_change":"2024-05-10T04:02:35.647526+0000","last_active":"2024-05-10T04:02:36.325315+0000","last_peered":"2024-05-10T04:02:36.325315+0000","last_clean":"2024-05-10T04:02:36.325315+0000","last_became_active":"2024-05-10T04:02:35.647018+0000","last_became_peered":"2024-05-10T04:02:35.647018+0000","last_unstale":"2024-05-10T04:02:36.325315+0000","last_undegraded":"2024-05-10T04:02:36.325315+0000","last_fullsized":"2024-05-10T04:02:36.325315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:17.173497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956894+0000","last_change":"2024-05-10T04:02:31.632263+0000","last_active":"2024-05-10T04:02:35.956894+0000","last_peered":"2024-05-10T04:02:35.956894+0000","last_clean":"2024-05-10T04:02:35.956894+0000","last_became_active":"2024-05-10T04:02:31.631785+0000","last_became_peered":"2024-05-10T04:02:31.631785+0000","last_unstale":"2024-05-10T04:02:35.956894+0000","last_undegraded":"2024-05-10T04:02:35.956894+0000","last_fullsized":"2024-05-10T04:02:35.956894+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:30.585862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:57.191865+0000","last_change":"2024-05-10T04:02:31.182262+0000","last_active":"2024-05-10T04:02:57.191865+0000","last_peered":"2024-05-10T04:02:57.191865+0000","last_clean":"2024-05-10T04:02:57.191865+0000","last_became_active":"2024-05-10T04:02:31.181802+0000","last_became_peered":"2024-05-10T04:02:31.181802+0000","last_unstale":"2024-05-10T04:02:57.191865+0000","last_undegraded":"2024-05-10T04:02:57.191865+0000","last_fullsized":"2024-05-10T04:02:57.191865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:45:40.696571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":325,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640698+0000","last_change":"2024-05-10T04:01:41.866365+0000","last_active":"2024-05-10T04:02:36.640698+0000","last_peered":"2024-05-10T04:02:36.640698+0000","last_clean":"2024-05-10T04:02:36.640698+0000","last_became_active":"2024-05-10T04:01:41.865825+0000","last_became_peered":"2024-05-10T04:01:41.865825+0000","last_unstale":"2024-05-10T04:02:36.640698+0000","last_undegraded":"2024-05-10T04:02:36.640698+0000","last_fullsized":"2024-05-10T04:02:36.640698+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_clean_scrub_stamp":"2024-05-10T03:59:19.853473+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:28:29.598339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.069016+0000","last_change":"2024-05-10T04:02:35.900841+0000","last_active":"2024-05-10T04:02:36.069016+0000","last_peered":"2024-05-10T04:02:36.069016+0000","last_clean":"2024-05-10T04:02:36.069016+0000","last_became_active":"2024-05-10T04:02:35.900323+0000","last_became_peered":"2024-05-10T04:02:35.900323+0000","last_unstale":"2024-05-10T04:02:36.069016+0000","last_undegraded":"2024-05-10T04:02:36.069016+0000","last_fullsized":"2024-05-10T04:02:36.069016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:00.554804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630799+0000","last_change":"2024-05-10T04:02:31.641167+0000","last_active":"2024-05-10T04:02:35.630799+0000","last_peered":"2024-05-10T04:02:35.630799+0000","last_clean":"2024-05-10T04:02:35.630799+0000","last_became_active":"2024-05-10T04:02:31.640824+0000","last_became_peered":"2024-05-10T04:02:31.640824+0000","last_unstale":"2024-05-10T04:02:35.630799+0000","last_undegraded":"2024-05-10T04:02:35.630799+0000","last_fullsized":"2024-05-10T04:02:35.630799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:59:48.531970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.180041+0000","last_change":"2024-05-10T04:02:31.187834+0000","last_active":"2024-05-10T04:02:56.180041+0000","last_peered":"2024-05-10T04:02:56.180041+0000","last_clean":"2024-05-10T04:02:56.180041+0000","last_became_active":"2024-05-10T04:02:31.187440+0000","last_became_peered":"2024-05-10T04:02:31.187440+0000","last_unstale":"2024-05-10T04:02:56.180041+0000","last_undegraded":"2024-05-10T04:02:56.180041+0000","last_fullsized":"2024-05-10T04:02:56.180041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:50:37.678245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638789+0000","last_change":"2024-05-10T04:02:35.649664+0000","last_active":"2024-05-10T04:02:36.638789+0000","last_peered":"2024-05-10T04:02:36.638789+0000","last_clean":"2024-05-10T04:02:36.638789+0000","last_became_active":"2024-05-10T04:02:35.649076+0000","last_became_peered":"2024-05-10T04:02:35.649076+0000","last_unstale":"2024-05-10T04:02:36.638789+0000","last_undegraded":"2024-05-10T04:02:36.638789+0000","last_fullsized":"2024-05-10T04:02:36.638789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:14:42.864845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.173775+0000","last_change":"2024-05-10T04:02:31.192126+0000","last_active":"2024-05-10T04:02:55.173775+0000","last_peered":"2024-05-10T04:02:55.173775+0000","last_clean":"2024-05-10T04:02:55.173775+0000","last_became_active":"2024-05-10T04:02:31.191771+0000","last_became_peered":"2024-05-10T04:02:31.191771+0000","last_unstale":"2024-05-10T04:02:55.173775+0000","last_undegraded":"2024-05-10T04:02:55.173775+0000","last_fullsized":"2024-05-10T04:02:55.173775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:43.550535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632220+0000","last_change":"2024-05-10T04:02:31.620788+0000","last_active":"2024-05-10T04:02:35.632220+0000","last_peered":"2024-05-10T04:02:35.632220+0000","last_clean":"2024-05-10T04:02:35.632220+0000","last_became_active":"2024-05-10T04:02:31.620456+0000","last_became_peered":"2024-05-10T04:02:31.620456+0000","last_unstale":"2024-05-10T04:02:35.632220+0000","last_undegraded":"2024-05-10T04:02:35.632220+0000","last_fullsized":"2024-05-10T04:02:35.632220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:02.956030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956837+0000","last_change":"2024-05-10T04:02:35.899935+0000","last_active":"2024-05-10T04:02:35.956837+0000","last_peered":"2024-05-10T04:02:35.956837+0000","last_clean":"2024-05-10T04:02:35.956837+0000","last_became_active":"2024-05-10T04:02:35.899488+0000","last_became_peered":"2024-05-10T04:02:35.899488+0000","last_unstale":"2024-05-10T04:02:35.956837+0000","last_undegraded":"2024-05-10T04:02:35.956837+0000","last_fullsized":"2024-05-10T04:02:35.956837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:28.600463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182368+0000","last_change":"2024-05-10T04:02:31.188605+0000","last_active":"2024-05-10T04:02:56.182368+0000","last_peered":"2024-05-10T04:02:56.182368+0000","last_clean":"2024-05-10T04:02:56.182368+0000","last_became_active":"2024-05-10T04:02:31.188283+0000","last_became_peered":"2024-05-10T04:02:31.188283+0000","last_unstale":"2024-05-10T04:02:56.182368+0000","last_undegraded":"2024-05-10T04:02:56.182368+0000","last_fullsized":"2024-05-10T04:02:56.182368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:40:03.940269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956931+0000","last_change":"2024-05-10T04:02:31.629476+0000","last_active":"2024-05-10T04:02:35.956931+0000","last_peered":"2024-05-10T04:02:35.956931+0000","last_clean":"2024-05-10T04:02:35.956931+0000","last_became_active":"2024-05-10T04:02:31.629104+0000","last_became_peered":"2024-05-10T04:02:31.629104+0000","last_unstale":"2024-05-10T04:02:35.956931+0000","last_undegraded":"2024-05-10T04:02:35.956931+0000","last_fullsized":"2024-05-10T04:02:35.956931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:57.873755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.409730+0000","last_change":"2024-05-10T04:02:35.644094+0000","last_active":"2024-05-10T04:02:36.409730+0000","last_peered":"2024-05-10T04:02:36.409730+0000","last_clean":"2024-05-10T04:02:36.409730+0000","last_became_active":"2024-05-10T04:02:35.643382+0000","last_became_peered":"2024-05-10T04:02:35.643382+0000","last_unstale":"2024-05-10T04:02:36.409730+0000","last_undegraded":"2024-05-10T04:02:36.409730+0000","last_fullsized":"2024-05-10T04:02:36.409730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:33.250052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.175394+0000","last_change":"2024-05-10T04:02:31.189302+0000","last_active":"2024-05-10T04:02:55.175394+0000","last_peered":"2024-05-10T04:02:55.175394+0000","last_clean":"2024-05-10T04:02:55.175394+0000","last_became_active":"2024-05-10T04:02:31.188323+0000","last_became_peered":"2024-05-10T04:02:31.188323+0000","last_unstale":"2024-05-10T04:02:55.175394+0000","last_undegraded":"2024-05-10T04:02:55.175394+0000","last_fullsized":"2024-05-10T04:02:55.175394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:11:02.922884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637085+0000","last_change":"2024-05-10T04:02:31.635864+0000","last_active":"2024-05-10T04:02:35.637085+0000","last_peered":"2024-05-10T04:02:35.637085+0000","last_clean":"2024-05-10T04:02:35.637085+0000","last_became_active":"2024-05-10T04:02:31.635512+0000","last_became_peered":"2024-05-10T04:02:31.635512+0000","last_unstale":"2024-05-10T04:02:35.637085+0000","last_undegraded":"2024-05-10T04:02:35.637085+0000","last_fullsized":"2024-05-10T04:02:35.637085+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:26:10.957848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.642076+0000","last_change":"2024-05-10T04:02:35.900489+0000","last_active":"2024-05-10T04:02:36.642076+0000","last_peered":"2024-05-10T04:02:36.642076+0000","last_clean":"2024-05-10T04:02:36.642076+0000","last_became_active":"2024-05-10T04:02:35.899908+0000","last_became_peered":"2024-05-10T04:02:35.899908+0000","last_unstale":"2024-05-10T04:02:36.642076+0000","last_undegraded":"2024-05-10T04:02:36.642076+0000","last_fullsized":"2024-05-10T04:02:36.642076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:08.386128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.173898+0000","last_change":"2024-05-10T04:02:31.193956+0000","last_active":"2024-05-10T04:02:55.173898+0000","last_peered":"2024-05-10T04:02:55.173898+0000","last_clean":"2024-05-10T04:02:55.173898+0000","last_became_active":"2024-05-10T04:02:31.193574+0000","last_became_peered":"2024-05-10T04:02:31.193574+0000","last_unstale":"2024-05-10T04:02:55.173898+0000","last_undegraded":"2024-05-10T04:02:55.173898+0000","last_fullsized":"2024-05-10T04:02:55.173898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:46.369152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635522+0000","last_change":"2024-05-10T04:02:31.633263+0000","last_active":"2024-05-10T04:02:35.635522+0000","last_peered":"2024-05-10T04:02:35.635522+0000","last_clean":"2024-05-10T04:02:35.635522+0000","last_became_active":"2024-05-10T04:02:31.632903+0000","last_became_peered":"2024-05-10T04:02:31.632903+0000","last_unstale":"2024-05-10T04:02:35.635522+0000","last_undegraded":"2024-05-10T04:02:35.635522+0000","last_fullsized":"2024-05-10T04:02:35.635522+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:28.047473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.088498+0000","last_change":"2024-05-10T04:02:35.656003+0000","last_active":"2024-05-10T04:02:36.088498+0000","last_peered":"2024-05-10T04:02:36.088498+0000","last_clean":"2024-05-10T04:02:36.088498+0000","last_became_active":"2024-05-10T04:02:35.655518+0000","last_became_peered":"2024-05-10T04:02:35.655518+0000","last_unstale":"2024-05-10T04:02:36.088498+0000","last_undegraded":"2024-05-10T04:02:36.088498+0000","last_fullsized":"2024-05-10T04:02:36.088498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:49.696699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.174999+0000","last_change":"2024-05-10T04:02:31.195297+0000","last_active":"2024-05-10T04:02:55.174999+0000","last_peered":"2024-05-10T04:02:55.174999+0000","last_clean":"2024-05-10T04:02:55.174999+0000","last_became_active":"2024-05-10T04:02:31.193960+0000","last_became_peered":"2024-05-10T04:02:31.193960+0000","last_unstale":"2024-05-10T04:02:55.174999+0000","last_undegraded":"2024-05-10T04:02:55.174999+0000","last_fullsized":"2024-05-10T04:02:55.174999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:04:10.314700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637944+0000","last_change":"2024-05-10T04:02:31.629247+0000","last_active":"2024-05-10T04:02:35.637944+0000","last_peered":"2024-05-10T04:02:35.637944+0000","last_clean":"2024-05-10T04:02:35.637944+0000","last_became_active":"2024-05-10T04:02:31.628838+0000","last_became_peered":"2024-05-10T04:02:31.628838+0000","last_unstale":"2024-05-10T04:02:35.637944+0000","last_undegraded":"2024-05-10T04:02:35.637944+0000","last_fullsized":"2024-05-10T04:02:35.637944+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:46:39.483264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641844+0000","last_change":"2024-05-10T04:02:35.644502+0000","last_active":"2024-05-10T04:02:36.641844+0000","last_peered":"2024-05-10T04:02:36.641844+0000","last_clean":"2024-05-10T04:02:36.641844+0000","last_became_active":"2024-05-10T04:02:35.643942+0000","last_became_peered":"2024-05-10T04:02:35.643942+0000","last_unstale":"2024-05-10T04:02:36.641844+0000","last_undegraded":"2024-05-10T04:02:36.641844+0000","last_fullsized":"2024-05-10T04:02:36.641844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:22:42.394568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632281+0000","last_change":"2024-05-10T04:02:31.193357+0000","last_active":"2024-05-10T04:02:35.632281+0000","last_peered":"2024-05-10T04:02:35.632281+0000","last_clean":"2024-05-10T04:02:35.632281+0000","last_became_active":"2024-05-10T04:02:31.190306+0000","last_became_peered":"2024-05-10T04:02:31.190306+0000","last_unstale":"2024-05-10T04:02:35.632281+0000","last_undegraded":"2024-05-10T04:02:35.632281+0000","last_fullsized":"2024-05-10T04:02:35.632281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:13.368960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956904+0000","last_change":"2024-05-10T04:02:31.628029+0000","last_active":"2024-05-10T04:02:35.956904+0000","last_peered":"2024-05-10T04:02:35.956904+0000","last_clean":"2024-05-10T04:02:35.956904+0000","last_became_active":"2024-05-10T04:02:31.627667+0000","last_became_peered":"2024-05-10T04:02:31.627667+0000","last_unstale":"2024-05-10T04:02:35.956904+0000","last_undegraded":"2024-05-10T04:02:35.956904+0000","last_fullsized":"2024-05-10T04:02:35.956904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:39:10.878894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641977+0000","last_change":"2024-05-10T04:02:35.900237+0000","last_active":"2024-05-10T04:02:36.641977+0000","last_peered":"2024-05-10T04:02:36.641977+0000","last_clean":"2024-05-10T04:02:36.641977+0000","last_became_active":"2024-05-10T04:02:35.899781+0000","last_became_peered":"2024-05-10T04:02:35.899781+0000","last_unstale":"2024-05-10T04:02:36.641977+0000","last_undegraded":"2024-05-10T04:02:36.641977+0000","last_fullsized":"2024-05-10T04:02:36.641977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:52:46.132557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.174762+0000","last_change":"2024-05-10T04:02:31.196920+0000","last_active":"2024-05-10T04:02:55.174762+0000","last_peered":"2024-05-10T04:02:55.174762+0000","last_clean":"2024-05-10T04:02:55.174762+0000","last_became_active":"2024-05-10T04:02:31.196191+0000","last_became_peered":"2024-05-10T04:02:31.196191+0000","last_unstale":"2024-05-10T04:02:55.174762+0000","last_undegraded":"2024-05-10T04:02:55.174762+0000","last_fullsized":"2024-05-10T04:02:55.174762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:45:17.980149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.639518+0000","last_change":"2024-05-10T04:02:31.623678+0000","last_active":"2024-05-10T04:02:35.639518+0000","last_peered":"2024-05-10T04:02:35.639518+0000","last_clean":"2024-05-10T04:02:35.639518+0000","last_became_active":"2024-05-10T04:02:31.623090+0000","last_became_peered":"2024-05-10T04:02:31.623090+0000","last_unstale":"2024-05-10T04:02:35.639518+0000","last_undegraded":"2024-05-10T04:02:35.639518+0000","last_fullsized":"2024-05-10T04:02:35.639518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:34.321959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.650242+0000","last_change":"2024-05-10T04:02:35.648833+0000","last_active":"2024-05-10T04:02:35.650242+0000","last_peered":"2024-05-10T04:02:35.650242+0000","last_clean":"2024-05-10T04:02:35.650242+0000","last_became_active":"2024-05-10T04:02:35.648391+0000","last_became_peered":"2024-05-10T04:02:35.648391+0000","last_unstale":"2024-05-10T04:02:35.650242+0000","last_undegraded":"2024-05-10T04:02:35.650242+0000","last_fullsized":"2024-05-10T04:02:35.650242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:12.773284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182221+0000","last_change":"2024-05-10T04:02:31.192900+0000","last_active":"2024-05-10T04:02:56.182221+0000","last_peered":"2024-05-10T04:02:56.182221+0000","last_clean":"2024-05-10T04:02:56.182221+0000","last_became_active":"2024-05-10T04:02:31.192481+0000","last_became_peered":"2024-05-10T04:02:31.192481+0000","last_unstale":"2024-05-10T04:02:56.182221+0000","last_undegraded":"2024-05-10T04:02:56.182221+0000","last_fullsized":"2024-05-10T04:02:56.182221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:17.202383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641659+0000","last_change":"2024-05-10T04:02:31.628490+0000","last_active":"2024-05-10T04:02:35.641659+0000","last_peered":"2024-05-10T04:02:35.641659+0000","last_clean":"2024-05-10T04:02:35.641659+0000","last_became_active":"2024-05-10T04:02:31.627889+0000","last_became_peered":"2024-05-10T04:02:31.627889+0000","last_unstale":"2024-05-10T04:02:35.641659+0000","last_undegraded":"2024-05-10T04:02:35.641659+0000","last_fullsized":"2024-05-10T04:02:35.641659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:32:55.366759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638326+0000","last_change":"2024-05-10T04:02:35.653530+0000","last_active":"2024-05-10T04:02:36.638326+0000","last_peered":"2024-05-10T04:02:36.638326+0000","last_clean":"2024-05-10T04:02:36.638326+0000","last_became_active":"2024-05-10T04:02:35.653095+0000","last_became_peered":"2024-05-10T04:02:35.653095+0000","last_unstale":"2024-05-10T04:02:36.638326+0000","last_undegraded":"2024-05-10T04:02:36.638326+0000","last_fullsized":"2024-05-10T04:02:36.638326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:09.921713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641619+0000","last_change":"2024-05-10T04:02:31.189434+0000","last_active":"2024-05-10T04:02:35.641619+0000","last_peered":"2024-05-10T04:02:35.641619+0000","last_clean":"2024-05-10T04:02:35.641619+0000","last_became_active":"2024-05-10T04:02:31.188843+0000","last_became_peered":"2024-05-10T04:02:31.188843+0000","last_unstale":"2024-05-10T04:02:35.641619+0000","last_undegraded":"2024-05-10T04:02:35.641619+0000","last_fullsized":"2024-05-10T04:02:35.641619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:31.508794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640750+0000","last_change":"2024-05-10T04:02:31.636775+0000","last_active":"2024-05-10T04:02:35.640750+0000","last_peered":"2024-05-10T04:02:35.640750+0000","last_clean":"2024-05-10T04:02:35.640750+0000","last_became_active":"2024-05-10T04:02:31.635946+0000","last_became_peered":"2024-05-10T04:02:31.635946+0000","last_unstale":"2024-05-10T04:02:35.640750+0000","last_undegraded":"2024-05-10T04:02:35.640750+0000","last_fullsized":"2024-05-10T04:02:35.640750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:48.830097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.068189+0000","last_change":"2024-05-10T04:02:35.646991+0000","last_active":"2024-05-10T04:02:36.068189+0000","last_peered":"2024-05-10T04:02:36.068189+0000","last_clean":"2024-05-10T04:02:36.068189+0000","last_became_active":"2024-05-10T04:02:35.646585+0000","last_became_peered":"2024-05-10T04:02:35.646585+0000","last_unstale":"2024-05-10T04:02:36.068189+0000","last_undegraded":"2024-05-10T04:02:36.068189+0000","last_fullsized":"2024-05-10T04:02:36.068189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:32:50.329746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.631077+0000","last_change":"2024-05-10T04:02:31.192103+0000","last_active":"2024-05-10T04:02:35.631077+0000","last_peered":"2024-05-10T04:02:35.631077+0000","last_clean":"2024-05-10T04:02:35.631077+0000","last_became_active":"2024-05-10T04:02:31.191378+0000","last_became_peered":"2024-05-10T04:02:31.191378+0000","last_unstale":"2024-05-10T04:02:35.631077+0000","last_undegraded":"2024-05-10T04:02:35.631077+0000","last_fullsized":"2024-05-10T04:02:35.631077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:55.298158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641428+0000","last_change":"2024-05-10T04:02:31.628587+0000","last_active":"2024-05-10T04:02:35.641428+0000","last_peered":"2024-05-10T04:02:35.641428+0000","last_clean":"2024-05-10T04:02:35.641428+0000","last_became_active":"2024-05-10T04:02:31.627296+0000","last_became_peered":"2024-05-10T04:02:31.627296+0000","last_unstale":"2024-05-10T04:02:35.641428+0000","last_undegraded":"2024-05-10T04:02:35.641428+0000","last_fullsized":"2024-05-10T04:02:35.641428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:04:09.783266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.638054+0000","last_change":"2024-05-10T04:02:31.634668+0000","last_active":"2024-05-10T04:02:35.638054+0000","last_peered":"2024-05-10T04:02:35.638054+0000","last_clean":"2024-05-10T04:02:35.638054+0000","last_became_active":"2024-05-10T04:02:31.634321+0000","last_became_peered":"2024-05-10T04:02:31.634321+0000","last_unstale":"2024-05-10T04:02:35.638054+0000","last_undegraded":"2024-05-10T04:02:35.638054+0000","last_fullsized":"2024-05-10T04:02:35.638054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:16:44.833804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.633396+0000","last_change":"2024-05-10T04:02:31.646002+0000","last_active":"2024-05-10T04:02:35.633396+0000","last_peered":"2024-05-10T04:02:35.633396+0000","last_clean":"2024-05-10T04:02:35.633396+0000","last_became_active":"2024-05-10T04:02:31.645673+0000","last_became_peered":"2024-05-10T04:02:31.645673+0000","last_unstale":"2024-05-10T04:02:35.633396+0000","last_undegraded":"2024-05-10T04:02:35.633396+0000","last_fullsized":"2024-05-10T04:02:35.633396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:26:26.814523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632295+0000","last_change":"2024-05-10T04:02:31.630606+0000","last_active":"2024-05-10T04:02:35.632295+0000","last_peered":"2024-05-10T04:02:35.632295+0000","last_clean":"2024-05-10T04:02:35.632295+0000","last_became_active":"2024-05-10T04:02:31.630313+0000","last_became_peered":"2024-05-10T04:02:31.630313+0000","last_unstale":"2024-05-10T04:02:35.632295+0000","last_undegraded":"2024-05-10T04:02:35.632295+0000","last_fullsized":"2024-05-10T04:02:35.632295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:39.876016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636526+0000","last_change":"2024-05-10T04:02:31.636239+0000","last_active":"2024-05-10T04:02:35.636526+0000","last_peered":"2024-05-10T04:02:35.636526+0000","last_clean":"2024-05-10T04:02:35.636526+0000","last_became_active":"2024-05-10T04:02:31.635377+0000","last_became_peered":"2024-05-10T04:02:31.635377+0000","last_unstale":"2024-05-10T04:02:35.636526+0000","last_undegraded":"2024-05-10T04:02:35.636526+0000","last_fullsized":"2024-05-10T04:02:35.636526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:02.612285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632276+0000","last_change":"2024-05-10T04:02:31.643231+0000","last_active":"2024-05-10T04:02:35.632276+0000","last_peered":"2024-05-10T04:02:35.632276+0000","last_clean":"2024-05-10T04:02:35.632276+0000","last_became_active":"2024-05-10T04:02:31.642797+0000","last_became_peered":"2024-05-10T04:02:31.642797+0000","last_unstale":"2024-05-10T04:02:35.632276+0000","last_undegraded":"2024-05-10T04:02:35.632276+0000","last_fullsized":"2024-05-10T04:02:35.632276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:01:57.365339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.638745+0000","last_change":"2024-05-10T04:02:31.633928+0000","last_active":"2024-05-10T04:02:35.638745+0000","last_peered":"2024-05-10T04:02:35.638745+0000","last_clean":"2024-05-10T04:02:35.638745+0000","last_became_active":"2024-05-10T04:02:31.633423+0000","last_became_peered":"2024-05-10T04:02:31.633423+0000","last_unstale":"2024-05-10T04:02:35.638745+0000","last_undegraded":"2024-05-10T04:02:35.638745+0000","last_fullsized":"2024-05-10T04:02:35.638745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:07.011354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632757+0000","last_change":"2024-05-10T04:02:31.627151+0000","last_active":"2024-05-10T04:02:35.632757+0000","last_peered":"2024-05-10T04:02:35.632757+0000","last_clean":"2024-05-10T04:02:35.632757+0000","last_became_active":"2024-05-10T04:02:31.626780+0000","last_became_peered":"2024-05-10T04:02:31.626780+0000","last_unstale":"2024-05-10T04:02:35.632757+0000","last_undegraded":"2024-05-10T04:02:35.632757+0000","last_fullsized":"2024-05-10T04:02:35.632757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:32.070530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956941+0000","last_change":"2024-05-10T04:02:31.640669+0000","last_active":"2024-05-10T04:02:35.956941+0000","last_peered":"2024-05-10T04:02:35.956941+0000","last_clean":"2024-05-10T04:02:35.956941+0000","last_became_active":"2024-05-10T04:02:31.640410+0000","last_became_peered":"2024-05-10T04:02:31.640410+0000","last_unstale":"2024-05-10T04:02:35.956941+0000","last_undegraded":"2024-05-10T04:02:35.956941+0000","last_fullsized":"2024-05-10T04:02:35.956941+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:02.235391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640243+0000","last_change":"2024-05-10T04:02:31.633624+0000","last_active":"2024-05-10T04:02:35.640243+0000","last_peered":"2024-05-10T04:02:35.640243+0000","last_clean":"2024-05-10T04:02:35.640243+0000","last_became_active":"2024-05-10T04:02:31.633241+0000","last_became_peered":"2024-05-10T04:02:31.633241+0000","last_unstale":"2024-05-10T04:02:35.640243+0000","last_undegraded":"2024-05-10T04:02:35.640243+0000","last_fullsized":"2024-05-10T04:02:35.640243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:58.750912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956858+0000","last_change":"2024-05-10T04:02:31.628632+0000","last_active":"2024-05-10T04:02:35.956858+0000","last_peered":"2024-05-10T04:02:35.956858+0000","last_clean":"2024-05-10T04:02:35.956858+0000","last_became_active":"2024-05-10T04:02:31.628349+0000","last_became_peered":"2024-05-10T04:02:31.628349+0000","last_unstale":"2024-05-10T04:02:35.956858+0000","last_undegraded":"2024-05-10T04:02:35.956858+0000","last_fullsized":"2024-05-10T04:02:35.956858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:32.575168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640841+0000","last_change":"2024-05-10T04:02:31.631328+0000","last_active":"2024-05-10T04:02:35.640841+0000","last_peered":"2024-05-10T04:02:35.640841+0000","last_clean":"2024-05-10T04:02:35.640841+0000","last_became_active":"2024-05-10T04:02:31.630957+0000","last_became_peered":"2024-05-10T04:02:31.630957+0000","last_unstale":"2024-05-10T04:02:35.640841+0000","last_undegraded":"2024-05-10T04:02:35.640841+0000","last_fullsized":"2024-05-10T04:02:35.640841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:40.400911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630956+0000","last_change":"2024-05-10T04:02:31.617972+0000","last_active":"2024-05-10T04:02:35.630956+0000","last_peered":"2024-05-10T04:02:35.630956+0000","last_clean":"2024-05-10T04:02:35.630956+0000","last_became_active":"2024-05-10T04:02:31.617663+0000","last_became_peered":"2024-05-10T04:02:31.617663+0000","last_unstale":"2024-05-10T04:02:35.630956+0000","last_undegraded":"2024-05-10T04:02:35.630956+0000","last_fullsized":"2024-05-10T04:02:35.630956+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:50.121955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641183+0000","last_change":"2024-05-10T04:02:31.644256+0000","last_active":"2024-05-10T04:02:35.641183+0000","last_peered":"2024-05-10T04:02:35.641183+0000","last_clean":"2024-05-10T04:02:35.641183+0000","last_became_active":"2024-05-10T04:02:31.643819+0000","last_became_peered":"2024-05-10T04:02:31.643819+0000","last_unstale":"2024-05-10T04:02:35.641183+0000","last_undegraded":"2024-05-10T04:02:35.641183+0000","last_fullsized":"2024-05-10T04:02:35.641183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:54.932547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279970,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7523,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":112,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":40501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514511,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":921600,"data_stored":457767,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":10,"up_from":68,"seq":292057776146,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":987136,"data_stored":1051262,"data_compressed":47273,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":1,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.35699999999999998}]},{"osd":2,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.71999999999999997}]},{"osd":3,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.59599999999999997}]},{"osd":4,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.498}]},{"osd":5,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76300000000000001}]},{"osd":6,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.629},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":8,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.70699999999999996}]},{"osd":9,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.66300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":9,"up_from":64,"seq":274877906966,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":937984,"data_stored":458003,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.307},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.34200000000000003}]},{"osd":1,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.502},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51600000000000001}]},{"osd":3,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.32500000000000001}]},{"osd":4,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.432},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.48799999999999999}]},{"osd":5,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.1699999999999999}]},{"osd":7,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.40200000000000002}]},{"osd":8,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.53000000000000003}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.61299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":8,"up_from":56,"seq":240518168602,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":937984,"data_stored":457923,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56299999999999994}]},{"osd":1,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.82099999999999995}]},{"osd":2,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.753},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.88}]},{"osd":6,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.90000000000000002}]},{"osd":7,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":1.0289999999999999}]},{"osd":9,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.97299999999999998}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.0509999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397533,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":978944,"data_stored":1048709,"data_compressed":47273,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.32100000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.65200000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.45800000000000002,"15min":0.45300000000000001},"min":{"1min":0.30499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.62}]},{"osd":2,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.93999999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75800000000000001}]},{"osd":3,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.443,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84799999999999998,"5min":1.143,"15min":1.143},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.63,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80900000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82199999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95799999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.69599999999999995,"15min":0.67400000000000004},"min":{"1min":0.438,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.5630000000000002,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.54600000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.88200000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64600000000000002}]},{"osd":8,"last update":"Fri May 10 04:02:03 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":15.521000000000001,"5min":15.521000000000001,"15min":15.521000000000001},"last":0.747},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":16.306999999999999,"5min":16.306999999999999,"15min":16.306999999999999},"last":0.85299999999999998}]},{"osd":9,"last update":"Fri May 10 04:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561055,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":917504,"data_stored":457185,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.38800000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.38900000000000001},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.48399999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71299999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.754,"5min":1.087,"15min":1.087},"last":0.64800000000000002}]},{"osd":3,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.47499999999999998,"15min":0.46700000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.36699999999999999}]},{"osd":4,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88500000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.63},"min":{"1min":0.44,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93700000000000006,"5min":1.018,"15min":1.018},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri May 10 04:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83299999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.67800000000000005,"15min":0.67300000000000004},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86399999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56499999999999995}]},{"osd":8,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.73199999999999998}]},{"osd":9,"last update":"Fri May 10 04:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.63}]},{"osd":10,"last update":"Fri May 10 04:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":929792,"data_stored":460981,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.32300000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.873,"15min":0.873},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri May 10 04:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.40400000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.52500000000000002,"15min":0.51100000000000001},"min":{"1min":0.38300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79800000000000004,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.70299999999999996}]},{"osd":3,"last update":"Fri May 10 04:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.53800000000000003,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.54800000000000004,"15min":0.53100000000000003},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri May 10 04:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.45700000000000002,"15min":0.44900000000000001},"min":{"1min":0.33800000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.36699999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.73899999999999999}]},{"osd":5,"last update":"Fri May 10 04:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.44900000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.54100000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.63200000000000001,"15min":0.61499999999999999},"min":{"1min":0.55500000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.84099999999999997}]},{"osd":7,"last update":"Fri May 10 04:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.58899999999999997,"15min":0.57399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.433,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.89700000000000002}]},{"osd":8,"last update":"Fri May 10 04:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.876}]},{"osd":9,"last update":"Fri May 10 04:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.94699999999999995}]},{"osd":10,"last update":"Fri May 10 04:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.97399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":458663,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.41899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51500000000000001},"min":{"1min":0.35899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri May 10 04:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54300000000000004,"15min":0.52500000000000002},"min":{"1min":0.40100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.77300000000000002}]},{"osd":3,"last update":"Fri May 10 04:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40799999999999997},"min":{"1min":0.26000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.58599999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.82699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri May 10 04:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri May 10 04:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59199999999999997,"15min":0.58099999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri May 10 04:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri May 10 04:01:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.69799999999999995}]},{"osd":8,"last update":"Fri May 10 04:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.84299999999999997}]},{"osd":9,"last update":"Fri May 10 04:02:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.751}]},{"osd":10,"last update":"Fri May 10 04:02:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.79100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":962560,"data_stored":469278,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.38800000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.39500000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72099999999999997,"5min":0.749,"15min":0.749},"last":0.74099999999999999}]},{"osd":1,"last update":"Fri May 10 04:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54400000000000004,"15min":0.52700000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79000000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57199999999999995,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.496}]},{"osd":3,"last update":"Fri May 10 04:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.47899999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.44400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.876}]},{"osd":4,"last update":"Fri May 10 04:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.41799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.318,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri May 10 04:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.501,"15min":0.5},"min":{"1min":0.29199999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.628},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80600000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.82799999999999996}]},{"osd":6,"last update":"Fri May 10 04:02:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.44400000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.86199999999999999}]},{"osd":7,"last update":"Fri May 10 04:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.93799999999999994}]},{"osd":8,"last update":"Fri May 10 04:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.95199999999999996}]},{"osd":9,"last update":"Fri May 10 04:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.91900000000000004}]},{"osd":10,"last update":"Fri May 10 04:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182443,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":457379,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.35399999999999998},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.46400000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36099999999999999},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.46800000000000003}]},{"osd":1,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.505,"15min":0.51100000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.59999999999999998,"5min":0.747,"15min":0.747},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69799999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69899999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78000000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.74199999999999999}]},{"osd":4,"last update":"Fri May 10 04:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58999999999999997,"15min":0.57999999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri May 10 04:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61899999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.51000000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri May 10 04:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri May 10 04:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.80500000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72799999999999998}]},{"osd":8,"last update":"Fri May 10 04:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.439}]},{"osd":9,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.75900000000000001}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.69899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953512,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":458174,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.33900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50900000000000001,"15min":0.502},"min":{"1min":0.371,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.79000000000000004}]},{"osd":1,"last update":"Fri May 10 04:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.39000000000000001},"min":{"1min":0.29599999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.66400000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.40899999999999997,"15min":0.38800000000000001},"min":{"1min":0.31900000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.65200000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri May 10 04:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60399999999999998,"15min":0.56399999999999995},"min":{"1min":0.46600000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.95099999999999996,"5min":1.046,"15min":1.046},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75900000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.65200000000000002}]},{"osd":3,"last update":"Fri May 10 04:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56999999999999995,"15min":0.54100000000000004},"min":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53700000000000003}]},{"osd":5,"last update":"Fri May 10 04:02:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.66400000000000003,"15min":0.65700000000000003},"min":{"1min":0.45500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.39700000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.81699999999999995}]},{"osd":6,"last update":"Fri May 10 04:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.47799999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70899999999999996,"15min":0.69999999999999996},"min":{"1min":0.503,"5min":0.439,"15min":0.439},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.89700000000000002}]},{"osd":7,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96499999999999997,"5min":1.01,"15min":1.01},"last":0.85099999999999998}]},{"osd":8,"last update":"Fri May 10 04:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.82999999999999996}]},{"osd":9,"last update":"Fri May 10 04:02:23 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.66900000000000004}]},{"osd":10,"last update":"Fri May 10 04:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.80200000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":5,"up_from":37,"seq":158913789987,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":991232,"data_stored":1048650,"data_compressed":47273,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.50600000000000001,"15min":0.49199999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39800000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79300000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri May 10 04:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.42199999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri May 10 04:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.25900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.39100000000000001}]},{"osd":3,"last update":"Fri May 10 04:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82499999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri May 10 04:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60699999999999998,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.58399999999999996},"min":{"1min":0.39000000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.439,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.48199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.69099999999999995,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.54400000000000004,"5min":0.505,"15min":0.505},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.72999999999999998}]},{"osd":8,"last update":"Fri May 10 04:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.64200000000000002}]},{"osd":9,"last update":"Fri May 10 04:02:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.78600000000000003}]},{"osd":10,"last update":"Fri May 10 04:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.82499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":24792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:03:00.809 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-10T04:03:00.810 INFO:tasks.cephadm:Setting up client nodes... 2024-05-10T04:03:00.810 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-10T04:03:01.066 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:01.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:01 smithi150 ceph-mon[42477]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:03:01.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:01 smithi150 ceph-mon[42477]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:03:01.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:01 smithi150 ceph-mon[42477]: pgmap v241: 129 pgs: 129 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-10T04:03:01.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:01 smithi181 ceph-mon[46602]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:03:01.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:01 smithi181 ceph-mon[46602]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:03:01.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:01 smithi181 ceph-mon[46602]: pgmap v241: 129 pgs: 129 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-10T04:03:01.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:01 smithi161 ceph-mon[46918]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:03:01.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:01 smithi161 ceph-mon[46918]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:03:01.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:01 smithi161 ceph-mon[46918]: pgmap v241: 129 pgs: 129 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-10T04:03:02.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.037+0000 7f6076641700 1 -- 172.21.15.150:0/3574912435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6070100b80 msgr2=0x7f6070100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:02.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.037+0000 7f6076641700 1 --2- 172.21.15.150:0/3574912435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6070100b80 0x7f6070100f90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6058009990 tx=0x7f6058009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.037+0000 7f6076641700 1 -- 172.21.15.150:0/3574912435 shutdown_connections 2024-05-10T04:03:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.037+0000 7f6076641700 1 --2- 172.21.15.150:0/3574912435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6070102650 0x7f6070108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.037+0000 7f6076641700 1 --2- 172.21.15.150:0/3574912435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6070100b80 0x7f6070100f90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.037+0000 7f6076641700 1 --2- 172.21.15.150:0/3574912435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6070101cb0 0x7f60701020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.037+0000 7f6076641700 1 -- 172.21.15.150:0/3574912435 >> 172.21.15.150:0/3574912435 conn(0x7f60700fc1c0 msgr2=0x7f60700fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.038+0000 7f6076641700 1 -- 172.21.15.150:0/3574912435 shutdown_connections 2024-05-10T04:03:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.038+0000 7f6076641700 1 -- 172.21.15.150:0/3574912435 wait complete. 2024-05-10T04:03:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.038+0000 7f6076641700 1 Processor -- start 2024-05-10T04:03:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.038+0000 7f6076641700 1 -- start start 2024-05-10T04:03:02.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f6076641700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6070100b80 0x7f607010b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:02.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f6076641700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6070101cb0 0x7f6070109ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:02.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f6076641700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6070102650 0x7f607010a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:02.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f6076641700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607010a840 con 0x7f6070100b80 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f6076641700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60701a2130 con 0x7f6070101cb0 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f6076641700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60701a2270 con 0x7f6070102650 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f606ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6070100b80 0x7f607010b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f606ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6070100b80 0x7f607010b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57656/0 (socket says 172.21.15.150:57656) 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f606ffff700 1 -- 172.21.15.150:0/2673154351 learned_addr learned my addr 172.21.15.150:0/2673154351 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f6074bde700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6070102650 0x7f607010a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f606f7fe700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6070101cb0 0x7f6070109ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f606ffff700 1 -- 172.21.15.150:0/2673154351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6070102650 msgr2=0x7f607010a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f606ffff700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6070102650 0x7f607010a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f606ffff700 1 -- 172.21.15.150:0/2673154351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6070101cb0 msgr2=0x7f6070109ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f606ffff700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6070101cb0 0x7f6070109ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f606ffff700 1 -- 172.21.15.150:0/2673154351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60580095f0 con 0x7f6070100b80 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f606f7fe700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6070101cb0 0x7f6070109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f606ffff700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6070100b80 0x7f607010b5e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6058005f50 tx=0x7f605800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:02.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.039+0000 7f606d7fa700 1 -- 172.21.15.150:0/2673154351 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605801d070 con 0x7f6070100b80 2024-05-10T04:03:02.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.040+0000 7f606d7fa700 1 -- 172.21.15.150:0/2673154351 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60580058e0 con 0x7f6070100b80 2024-05-10T04:03:02.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.040+0000 7f6076641700 1 -- 172.21.15.150:0/2673154351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60701a2470 con 0x7f6070100b80 2024-05-10T04:03:02.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.040+0000 7f606d7fa700 1 -- 172.21.15.150:0/2673154351 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058004a90 con 0x7f6070100b80 2024-05-10T04:03:02.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.040+0000 7f6076641700 1 -- 172.21.15.150:0/2673154351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60701a2910 con 0x7f6070100b80 2024-05-10T04:03:02.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.040+0000 7f6076641700 1 -- 172.21.15.150:0/2673154351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6070107e10 con 0x7f6070100b80 2024-05-10T04:03:02.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.042+0000 7f606d7fa700 1 -- 172.21.15.150:0/2673154351 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6058004bf0 con 0x7f6070100b80 2024-05-10T04:03:02.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.042+0000 7f606d7fa700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f605c0a11b0 0x7f605c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:02.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.042+0000 7f606f7fe700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f605c0a11b0 0x7f605c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:02.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.042+0000 7f606d7fa700 1 -- 172.21.15.150:0/2673154351 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f60580cdb40 con 0x7f6070100b80 2024-05-10T04:03:02.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.043+0000 7f606f7fe700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f605c0a11b0 0x7f605c0a36c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6060005950 tx=0x7f60600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:02.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.047+0000 7f606d7fa700 1 -- 172.21.15.150:0/2673154351 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6058097010 con 0x7f6070100b80 2024-05-10T04:03:02.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.317+0000 7f6076641700 1 -- 172.21.15.150:0/2673154351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f607004edc0 con 0x7f6070100b80 2024-05-10T04:03:02.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.323+0000 7f606d7fa700 1 -- 172.21.15.150:0/2673154351 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f605809a630 con 0x7f6070100b80 2024-05-10T04:03:02.325 INFO:teuthology.orchestra.run.smithi150.stdout:[client.0] 2024-05-10T04:03:02.325 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQB2nD1mB5MKExAAvWmcBNnF0ozq4s3t3xexZQ== 2024-05-10T04:03:02.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.325+0000 7f6076641700 1 -- 172.21.15.150:0/2673154351 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f605c0a11b0 msgr2=0x7f605c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:02.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.325+0000 7f6076641700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f605c0a11b0 0x7f605c0a36c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6060005950 tx=0x7f60600058e0 comp rx=0 tx=0).stop 2024-05-10T04:03:02.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.325+0000 7f6076641700 1 -- 172.21.15.150:0/2673154351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6070100b80 msgr2=0x7f607010b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:02.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.325+0000 7f6076641700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6070100b80 0x7f607010b5e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6058005f50 tx=0x7f605800f710 comp rx=0 tx=0).stop 2024-05-10T04:03:02.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.325+0000 7f6076641700 1 -- 172.21.15.150:0/2673154351 shutdown_connections 2024-05-10T04:03:02.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.325+0000 7f6076641700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f605c0a11b0 0x7f605c0a36c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:02.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.325+0000 7f6076641700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6070102650 0x7f607010a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:02.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.325+0000 7f6076641700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6070100b80 0x7f607010b5e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:02.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.325+0000 7f6076641700 1 --2- 172.21.15.150:0/2673154351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6070101cb0 0x7f6070109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:02.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.325+0000 7f6076641700 1 -- 172.21.15.150:0/2673154351 >> 172.21.15.150:0/2673154351 conn(0x7f60700fc1c0 msgr2=0x7f6070104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:02.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.325+0000 7f6076641700 1 -- 172.21.15.150:0/2673154351 shutdown_connections 2024-05-10T04:03:02.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:02.325+0000 7f6076641700 1 -- 172.21.15.150:0/2673154351 wait complete. 2024-05-10T04:03:02.613 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:02 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2673154351' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T04:03:02.613 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:02 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2673154351' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T04:03:02.722 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:03:02.723 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-10T04:03:02.723 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-10T04:03:02.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:02 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2673154351' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T04:03:02.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:02 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2673154351' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T04:03:02.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:02 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2673154351' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T04:03:02.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:02 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2673154351' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T04:03:02.764 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-10T04:03:02.764 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-10T04:03:02.764 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-10T04:03:03.025 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:03.451 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:03 smithi150 ceph-mon[42477]: pgmap v242: 129 pgs: 129 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-05-10T04:03:03.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:03 smithi181 ceph-mon[46602]: pgmap v242: 129 pgs: 129 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-05-10T04:03:03.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:03 smithi161 ceph-mon[46918]: pgmap v242: 129 pgs: 129 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-05-10T04:03:03.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.976+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/2192951245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a34106880 msgr2=0x7f6a34106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:03.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.976+0000 7f6a3ac5c700 1 --2- 172.21.15.150:0/2192951245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a34106880 0x7f6a34106c90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009990 tx=0x7f6a20009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:03.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.977+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/2192951245 shutdown_connections 2024-05-10T04:03:03.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.977+0000 7f6a3ac5c700 1 --2- 172.21.15.150:0/2192951245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a341071d0 0x7f6a3410eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:03.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.977+0000 7f6a3ac5c700 1 --2- 172.21.15.150:0/2192951245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a34101fa0 0x7f6a341023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:03.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.977+0000 7f6a3ac5c700 1 --2- 172.21.15.150:0/2192951245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a34106880 0x7f6a34106c90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:03.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.977+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/2192951245 >> 172.21.15.150:0/2192951245 conn(0x7f6a340778b0 msgr2=0x7f6a34077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:03.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.977+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/2192951245 shutdown_connections 2024-05-10T04:03:03.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.977+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/2192951245 wait complete. 2024-05-10T04:03:03.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.977+0000 7f6a3ac5c700 1 Processor -- start 2024-05-10T04:03:03.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.977+0000 7f6a3ac5c700 1 -- start start 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.977+0000 7f6a3ac5c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a34101fa0 0x7f6a34198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.977+0000 7f6a3ac5c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a34106880 0x7f6a34198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.977+0000 7f6a3ac5c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a341071d0 0x7f6a3419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a3ac5c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3419d540 con 0x7f6a34101fa0 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a3ac5c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3419d6b0 con 0x7f6a341071d0 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a3ac5c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3419d870 con 0x7f6a34106880 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a389f8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a34101fa0 0x7f6a34198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a389f8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a34101fa0 0x7f6a34198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57676/0 (socket says 172.21.15.150:57676) 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a389f8700 1 -- 172.21.15.150:0/3226792439 learned_addr learned my addr 172.21.15.150:0/3226792439 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a391f9700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a341071d0 0x7f6a3419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a33fff700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a34106880 0x7f6a34198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a389f8700 1 -- 172.21.15.150:0/3226792439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a34106880 msgr2=0x7f6a34198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a389f8700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a34106880 0x7f6a34198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a389f8700 1 -- 172.21.15.150:0/3226792439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a341071d0 msgr2=0x7f6a3419cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:03.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a389f8700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a341071d0 0x7f6a3419cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:03.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a389f8700 1 -- 172.21.15.150:0/3226792439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a200095f0 con 0x7f6a34101fa0 2024-05-10T04:03:03.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a391f9700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a341071d0 0x7f6a3419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:03.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a33fff700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a34106880 0x7f6a34198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:03.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a389f8700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a34101fa0 0x7f6a34198520 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800d950 tx=0x7f6a2800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:03.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a31ffb700 1 -- 172.21.15.150:0/3226792439 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28009b40 con 0x7f6a34101fa0 2024-05-10T04:03:03.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/3226792439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3419db50 con 0x7f6a34101fa0 2024-05-10T04:03:03.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a31ffb700 1 -- 172.21.15.150:0/3226792439 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a28009ca0 con 0x7f6a34101fa0 2024-05-10T04:03:03.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.978+0000 7f6a31ffb700 1 -- 172.21.15.150:0/3226792439 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2800f620 con 0x7f6a34101fa0 2024-05-10T04:03:03.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.979+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/3226792439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a341a5450 con 0x7f6a34101fa0 2024-05-10T04:03:03.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.979+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/3226792439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3410a280 con 0x7f6a34101fa0 2024-05-10T04:03:03.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.981+0000 7f6a31ffb700 1 -- 172.21.15.150:0/3226792439 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6a28010460 con 0x7f6a34101fa0 2024-05-10T04:03:03.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.981+0000 7f6a31ffb700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6a240a11b0 0x7f6a240a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:03.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.982+0000 7f6a33fff700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6a240a11b0 0x7f6a240a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:03.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.982+0000 7f6a31ffb700 1 -- 172.21.15.150:0/3226792439 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6a280cd000 con 0x7f6a34101fa0 2024-05-10T04:03:03.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.982+0000 7f6a33fff700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6a240a11b0 0x7f6a240a36c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000b5c0 tx=0x7f6a20005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:03.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:03.986+0000 7f6a31ffb700 1 -- 172.21.15.150:0/3226792439 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6a28100d60 con 0x7f6a34101fa0 2024-05-10T04:03:04.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.235+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/3226792439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6a34103400 con 0x7f6a34101fa0 2024-05-10T04:03:04.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.240+0000 7f6a31ffb700 1 -- 172.21.15.150:0/3226792439 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260292 (secure 0 0 0) 0x7f6a281138e0 con 0x7f6a34101fa0 2024-05-10T04:03:04.245 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:04.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.243+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/3226792439 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6a240a11b0 msgr2=0x7f6a240a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:04.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.243+0000 7f6a3ac5c700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6a240a11b0 0x7f6a240a36c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000b5c0 tx=0x7f6a20005960 comp rx=0 tx=0).stop 2024-05-10T04:03:04.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.243+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/3226792439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a34101fa0 msgr2=0x7f6a34198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:04.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.243+0000 7f6a3ac5c700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a34101fa0 0x7f6a34198520 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800d950 tx=0x7f6a2800dd10 comp rx=0 tx=0).stop 2024-05-10T04:03:04.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.243+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/3226792439 shutdown_connections 2024-05-10T04:03:04.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.243+0000 7f6a3ac5c700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6a240a11b0 0x7f6a240a36c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:04.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.243+0000 7f6a3ac5c700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a341071d0 0x7f6a3419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:04.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.243+0000 7f6a3ac5c700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a34101fa0 0x7f6a34198520 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:04.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.243+0000 7f6a3ac5c700 1 --2- 172.21.15.150:0/3226792439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a34106880 0x7f6a34198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:04.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.243+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/3226792439 >> 172.21.15.150:0/3226792439 conn(0x7f6a340778b0 msgr2=0x7f6a340ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:04.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.243+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/3226792439 shutdown_connections 2024-05-10T04:03:04.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:04.243+0000 7f6a3ac5c700 1 -- 172.21.15.150:0/3226792439 wait complete. 2024-05-10T04:03:04.537 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:04 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3226792439' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T04:03:04.639 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1954703650},{"type":"v1","addr":"172.21.15.150:6801","nonce":1954703650}]},"active_addr":"172.21.15.150:6801/1954703650","active_change":"2024-05-10T03:57:29.013577+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.150:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":3012201187}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":661419065}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":1054942358}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":36425353}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":2602394562}]}]} 2024-05-10T04:03:04.644 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-10T04:03:04.644 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-10T04:03:04.644 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T04:03:04.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:04 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3226792439' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T04:03:04.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:04 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3226792439' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T04:03:04.868 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:05.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:05 smithi150 ceph-mon[42477]: pgmap v243: 129 pgs: 129 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-05-10T04:03:05.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:05 smithi181 ceph-mon[46602]: pgmap v243: 129 pgs: 129 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-05-10T04:03:05.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:05 smithi161 ceph-mon[46918]: pgmap v243: 129 pgs: 129 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-05-10T04:03:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.844+0000 7feb87acf700 1 -- 172.21.15.150:0/3911485497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb80105730 msgr2=0x7feb80105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.844+0000 7feb87acf700 1 --2- 172.21.15.150:0/3911485497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb80105730 0x7feb80105b40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feb70009990 tx=0x7feb70009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:05.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.844+0000 7feb87acf700 1 -- 172.21.15.150:0/3911485497 shutdown_connections 2024-05-10T04:03:05.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.844+0000 7feb87acf700 1 --2- 172.21.15.150:0/3911485497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb80106080 0x7feb8010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:05.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.844+0000 7feb87acf700 1 --2- 172.21.15.150:0/3911485497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb80100e00 0x7feb80101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:05.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.844+0000 7feb87acf700 1 --2- 172.21.15.150:0/3911485497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb80105730 0x7feb80105b40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:05.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.844+0000 7feb87acf700 1 -- 172.21.15.150:0/3911485497 >> 172.21.15.150:0/3911485497 conn(0x7feb800fc7a0 msgr2=0x7feb800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:05.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.844+0000 7feb87acf700 1 -- 172.21.15.150:0/3911485497 shutdown_connections 2024-05-10T04:03:05.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.844+0000 7feb87acf700 1 -- 172.21.15.150:0/3911485497 wait complete. 2024-05-10T04:03:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.845+0000 7feb87acf700 1 Processor -- start 2024-05-10T04:03:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.845+0000 7feb87acf700 1 -- start start 2024-05-10T04:03:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.845+0000 7feb87acf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb80100e00 0x7feb801973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.845+0000 7feb87acf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb80105730 0x7feb80197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.845+0000 7feb87acf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb80106080 0x7feb8019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.845+0000 7feb87acf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8019c3d0 con 0x7feb80105730 2024-05-10T04:03:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.845+0000 7feb87acf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8019c540 con 0x7feb80100e00 2024-05-10T04:03:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.845+0000 7feb87acf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8019c700 con 0x7feb80106080 2024-05-10T04:03:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.845+0000 7feb8506a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb80105730 0x7feb80197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.845+0000 7feb8506a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb80105730 0x7feb80197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57696/0 (socket says 172.21.15.150:57696) 2024-05-10T04:03:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.845+0000 7feb8506a700 1 -- 172.21.15.150:0/283896311 learned_addr learned my addr 172.21.15.150:0/283896311 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.845+0000 7feb8606c700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb80106080 0x7feb8019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.845+0000 7feb8586b700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb80100e00 0x7feb801973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.846+0000 7feb8506a700 1 -- 172.21.15.150:0/283896311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb80106080 msgr2=0x7feb8019be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.846+0000 7feb8506a700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb80106080 0x7feb8019be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.846+0000 7feb8506a700 1 -- 172.21.15.150:0/283896311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb80100e00 msgr2=0x7feb801973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.846+0000 7feb8506a700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb80100e00 0x7feb801973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.846+0000 7feb8506a700 1 -- 172.21.15.150:0/283896311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb700095f0 con 0x7feb80105730 2024-05-10T04:03:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.846+0000 7feb8506a700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb80105730 0x7feb80197950 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7feb700098e0 tx=0x7feb7000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:05.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.846+0000 7feb76ffd700 1 -- 172.21.15.150:0/283896311 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7001d070 con 0x7feb80105730 2024-05-10T04:03:05.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.846+0000 7feb76ffd700 1 -- 172.21.15.150:0/283896311 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb70022470 con 0x7feb80105730 2024-05-10T04:03:05.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.846+0000 7feb87acf700 1 -- 172.21.15.150:0/283896311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8019c980 con 0x7feb80105730 2024-05-10T04:03:05.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.846+0000 7feb76ffd700 1 -- 172.21.15.150:0/283896311 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70017650 con 0x7feb80105730 2024-05-10T04:03:05.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.846+0000 7feb87acf700 1 -- 172.21.15.150:0/283896311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb801a4110 con 0x7feb80105730 2024-05-10T04:03:05.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.847+0000 7feb87acf700 1 -- 172.21.15.150:0/283896311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb80109130 con 0x7feb80105730 2024-05-10T04:03:05.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.849+0000 7feb76ffd700 1 -- 172.21.15.150:0/283896311 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7feb700225e0 con 0x7feb80105730 2024-05-10T04:03:05.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.850+0000 7feb76ffd700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb6c0a11b0 0x7feb6c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:05.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.850+0000 7feb8586b700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb6c0a11b0 0x7feb6c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:05.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.850+0000 7feb76ffd700 1 -- 172.21.15.150:0/283896311 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7feb700ce100 con 0x7feb80105730 2024-05-10T04:03:05.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.850+0000 7feb8586b700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb6c0a11b0 0x7feb6c0a36c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb7c005fd0 tx=0x7feb7c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:05.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:05.853+0000 7feb76ffd700 1 -- 172.21.15.150:0/283896311 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feb7009b120 con 0x7feb80105730 2024-05-10T04:03:06.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.050+0000 7feb87acf700 1 -- 172.21.15.150:0/283896311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb801022b0 con 0x7feb80105730 2024-05-10T04:03:06.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.052+0000 7feb76ffd700 1 -- 172.21.15.150:0/283896311 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24027 (secure 0 0 0) 0x7feb7009acb0 con 0x7feb80105730 2024-05-10T04:03:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:06.058 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":85,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:02:36.063310+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:03:06.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.054+0000 7feb87acf700 1 -- 172.21.15.150:0/283896311 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb6c0a11b0 msgr2=0x7feb6c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:06.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.054+0000 7feb87acf700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb6c0a11b0 0x7feb6c0a36c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb7c005fd0 tx=0x7feb7c00a560 comp rx=0 tx=0).stop 2024-05-10T04:03:06.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.054+0000 7feb87acf700 1 -- 172.21.15.150:0/283896311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb80105730 msgr2=0x7feb80197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:06.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.054+0000 7feb87acf700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb80105730 0x7feb80197950 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7feb700098e0 tx=0x7feb7000f670 comp rx=0 tx=0).stop 2024-05-10T04:03:06.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.054+0000 7feb87acf700 1 -- 172.21.15.150:0/283896311 shutdown_connections 2024-05-10T04:03:06.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.054+0000 7feb87acf700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb6c0a11b0 0x7feb6c0a36c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:06.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.054+0000 7feb87acf700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb80106080 0x7feb8019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:06.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.054+0000 7feb87acf700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb80100e00 0x7feb801973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:06.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.054+0000 7feb87acf700 1 --2- 172.21.15.150:0/283896311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb80105730 0x7feb80197950 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:06.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.054+0000 7feb87acf700 1 -- 172.21.15.150:0/283896311 >> 172.21.15.150:0/283896311 conn(0x7feb800fc7a0 msgr2=0x7feb80106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:06.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.054+0000 7feb87acf700 1 -- 172.21.15.150:0/283896311 shutdown_connections 2024-05-10T04:03:06.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:06.054+0000 7feb87acf700 1 -- 172.21.15.150:0/283896311 wait complete. 2024-05-10T04:03:06.415 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-10T04:03:06.415 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T04:03:06.641 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:06.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:06 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/283896311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:06.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:06 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/283896311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:06.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:06 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/283896311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:07.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.653+0000 7f22a907d700 1 -- 172.21.15.150:0/74966429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4100e00 msgr2=0x7f22a4101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:07.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.653+0000 7f22a907d700 1 --2- 172.21.15.150:0/74966429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4100e00 0x7f22a4101210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2294009990 tx=0x7f2294009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:07.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.653+0000 7f22a907d700 1 -- 172.21.15.150:0/74966429 shutdown_connections 2024-05-10T04:03:07.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.653+0000 7f22a907d700 1 --2- 172.21.15.150:0/74966429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4106080 0x7f22a410d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:07.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.653+0000 7f22a907d700 1 --2- 172.21.15.150:0/74966429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4100e00 0x7f22a4101210 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:07.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.653+0000 7f22a907d700 1 --2- 172.21.15.150:0/74966429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4105730 0x7f22a4105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:07.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.653+0000 7f22a907d700 1 -- 172.21.15.150:0/74966429 >> 172.21.15.150:0/74966429 conn(0x7f22a40fc7a0 msgr2=0x7f22a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:07.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.653+0000 7f22a907d700 1 -- 172.21.15.150:0/74966429 shutdown_connections 2024-05-10T04:03:07.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.653+0000 7f22a907d700 1 -- 172.21.15.150:0/74966429 wait complete. 2024-05-10T04:03:07.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.654+0000 7f22a907d700 1 Processor -- start 2024-05-10T04:03:07.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.654+0000 7f22a907d700 1 -- start start 2024-05-10T04:03:07.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.654+0000 7f22a907d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4100e00 0x7f22a41994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:07.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.654+0000 7f22a907d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4105730 0x7f22a4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:07.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.654+0000 7f22a907d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4106080 0x7f22a41a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:07.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.654+0000 7f22a907d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a419a1a0 con 0x7f22a4105730 2024-05-10T04:03:07.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.654+0000 7f22a907d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a41a48e0 con 0x7f22a4106080 2024-05-10T04:03:07.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.654+0000 7f22a907d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a41a4aa0 con 0x7f22a4100e00 2024-05-10T04:03:07.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4105730 0x7f22a4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:07.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4105730 0x7f22a4199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57722/0 (socket says 172.21.15.150:57722) 2024-05-10T04:03:07.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a259c700 1 -- 172.21.15.150:0/2110441202 learned_addr learned my addr 172.21.15.150:0/2110441202 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:07.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a2d9d700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4100e00 0x7f22a41994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:07.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a359e700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4106080 0x7f22a41a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:07.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a259c700 1 -- 172.21.15.150:0/2110441202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4100e00 msgr2=0x7f22a41994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:07.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a259c700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4100e00 0x7f22a41994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:07.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a259c700 1 -- 172.21.15.150:0/2110441202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4106080 msgr2=0x7f22a41a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:07.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a259c700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4106080 0x7f22a41a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:07.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a259c700 1 -- 172.21.15.150:0/2110441202 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22940095f0 con 0x7f22a4105730 2024-05-10T04:03:07.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a2d9d700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4100e00 0x7f22a41994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:03:07.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a359e700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4106080 0x7f22a41a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:03:07.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a259c700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4105730 0x7f22a4199a30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f229800e970 tx=0x7f229800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:07.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f228bfff700 1 -- 172.21.15.150:0/2110441202 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298009790 con 0x7f22a4105730 2024-05-10T04:03:07.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f22a907d700 1 -- 172.21.15.150:0/2110441202 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a4197170 con 0x7f22a4105730 2024-05-10T04:03:07.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f228bfff700 1 -- 172.21.15.150:0/2110441202 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f229800c7e0 con 0x7f22a4105730 2024-05-10T04:03:07.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.655+0000 7f228bfff700 1 -- 172.21.15.150:0/2110441202 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980054e0 con 0x7f22a4105730 2024-05-10T04:03:07.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.656+0000 7f22a907d700 1 -- 172.21.15.150:0/2110441202 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a41976c0 con 0x7f22a4105730 2024-05-10T04:03:07.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.656+0000 7f22a907d700 1 -- 172.21.15.150:0/2110441202 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a4109040 con 0x7f22a4105730 2024-05-10T04:03:07.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.658+0000 7f228bfff700 1 -- 172.21.15.150:0/2110441202 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f229800c950 con 0x7f22a4105730 2024-05-10T04:03:07.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.658+0000 7f228bfff700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f228c0a11f0 0x7f228c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:07.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.658+0000 7f22a2d9d700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f228c0a11f0 0x7f228c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:07.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.658+0000 7f228bfff700 1 -- 172.21.15.150:0/2110441202 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2298014070 con 0x7f22a4105730 2024-05-10T04:03:07.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.659+0000 7f22a2d9d700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f228c0a11f0 0x7f228c0a3700 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f229400b2e0 tx=0x7f229401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:07.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.662+0000 7f228bfff700 1 -- 172.21.15.150:0/2110441202 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2298096520 con 0x7f22a4105730 2024-05-10T04:03:07.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:07 smithi150 ceph-mon[42477]: pgmap v244: 129 pgs: 129 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-05-10T04:03:07.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:07 smithi181 ceph-mon[46602]: pgmap v244: 129 pgs: 129 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-05-10T04:03:07.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:07 smithi161 ceph-mon[46918]: pgmap v244: 129 pgs: 129 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-05-10T04:03:07.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.856+0000 7f22a907d700 1 -- 172.21.15.150:0/2110441202 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22a404edc0 con 0x7f22a4105730 2024-05-10T04:03:07.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.857+0000 7f228bfff700 1 -- 172.21.15.150:0/2110441202 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24027 (secure 0 0 0) 0x7f2298099b40 con 0x7f22a4105730 2024-05-10T04:03:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:07.861 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":85,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:02:36.063310+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:03:07.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.859+0000 7f22a907d700 1 -- 172.21.15.150:0/2110441202 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f228c0a11f0 msgr2=0x7f228c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:07.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.859+0000 7f22a907d700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f228c0a11f0 0x7f228c0a3700 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f229400b2e0 tx=0x7f229401a040 comp rx=0 tx=0).stop 2024-05-10T04:03:07.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.859+0000 7f22a907d700 1 -- 172.21.15.150:0/2110441202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4105730 msgr2=0x7f22a4199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:07.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.859+0000 7f22a907d700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4105730 0x7f22a4199a30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f229800e970 tx=0x7f229800ec80 comp rx=0 tx=0).stop 2024-05-10T04:03:07.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.859+0000 7f22a907d700 1 -- 172.21.15.150:0/2110441202 shutdown_connections 2024-05-10T04:03:07.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.859+0000 7f22a907d700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f228c0a11f0 0x7f228c0a3700 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:07.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.859+0000 7f22a907d700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4106080 0x7f22a41a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:07.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.859+0000 7f22a907d700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4100e00 0x7f22a41994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:07.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.859+0000 7f22a907d700 1 --2- 172.21.15.150:0/2110441202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4105730 0x7f22a4199a30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:07.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.859+0000 7f22a907d700 1 -- 172.21.15.150:0/2110441202 >> 172.21.15.150:0/2110441202 conn(0x7f22a40fc7a0 msgr2=0x7f22a4104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:07.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.860+0000 7f22a907d700 1 -- 172.21.15.150:0/2110441202 shutdown_connections 2024-05-10T04:03:07.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:07.860+0000 7f22a907d700 1 -- 172.21.15.150:0/2110441202 wait complete. 2024-05-10T04:03:08.248 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-10T04:03:08.248 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-10T04:03:08.249 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-10T04:03:08.249 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-10T04:03:08.249 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-10T04:03:08.250 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-10T04:03:08.250 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-10T04:03:08.251 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-10T04:03:08.251 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-10T04:03:08.251 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-10T04:03:08.252 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-10T04:03:08.252 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-10T04:03:08.722 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:08 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2110441202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:08.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:08 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2110441202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:08.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:08 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2110441202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:08.814 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:09.058 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:09.116 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:09.167 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:09.283 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:09.283 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:09.283 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:09.430 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:09 smithi150 ceph-mon[42477]: pgmap v245: 129 pgs: 129 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2024-05-10T04:03:09.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:09 smithi181 ceph-mon[46602]: pgmap v245: 129 pgs: 129 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2024-05-10T04:03:09.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:09 smithi161 ceph-mon[46918]: pgmap v245: 129 pgs: 129 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2024-05-10T04:03:10.579 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:10.631 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:10.632 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:11.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.382+0000 7fdee1938700 1 -- 172.21.15.150:0/1248982715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdedc103750 msgr2=0x7fdedc10bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:11.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.382+0000 7fdee1938700 1 --2- 172.21.15.150:0/1248982715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdedc103750 0x7fdedc10bca0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009990 tx=0x7fdec4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:11.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.382+0000 7fdee1938700 1 -- 172.21.15.150:0/1248982715 shutdown_connections 2024-05-10T04:03:11.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.382+0000 7fdee1938700 1 --2- 172.21.15.150:0/1248982715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdedc10c230 0x7fdedc10e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:11.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.382+0000 7fdee1938700 1 --2- 172.21.15.150:0/1248982715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdedc102e00 0x7fdedc103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:11.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.382+0000 7fdee1938700 1 --2- 172.21.15.150:0/1248982715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdedc103750 0x7fdedc10bca0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.382+0000 7fdee1938700 1 -- 172.21.15.150:0/1248982715 >> 172.21.15.150:0/1248982715 conn(0x7fdedc0fc7a0 msgr2=0x7fdedc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.382+0000 7fdee1938700 1 -- 172.21.15.150:0/1248982715 shutdown_connections 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.382+0000 7fdee1938700 1 -- 172.21.15.150:0/1248982715 wait complete. 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.383+0000 7fdee1938700 1 Processor -- start 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.383+0000 7fdee1938700 1 -- start start 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.383+0000 7fdee1938700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdedc102e00 0x7fdedc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.383+0000 7fdee1938700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdedc103750 0x7fdedc198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.383+0000 7fdee1938700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdedc10c230 0x7fdedc19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.383+0000 7fdee1938700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc19d4b0 con 0x7fdedc102e00 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.383+0000 7fdee1938700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc19d620 con 0x7fdedc10c230 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.383+0000 7fdee1938700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc19d7e0 con 0x7fdedc103750 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.383+0000 7fdedaffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdedc102e00 0x7fdedc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.383+0000 7fdedaffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdedc102e00 0x7fdedc198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54488/0 (socket says 172.21.15.150:54488) 2024-05-10T04:03:11.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.383+0000 7fdedaffd700 1 -- 172.21.15.150:0/440508715 learned_addr learned my addr 172.21.15.150:0/440508715 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdeda7fc700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdedc103750 0x7fdedc198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdedb7fe700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdedc10c230 0x7fdedc19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdedaffd700 1 -- 172.21.15.150:0/440508715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdedc103750 msgr2=0x7fdedc198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdedaffd700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdedc103750 0x7fdedc198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdedaffd700 1 -- 172.21.15.150:0/440508715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdedc10c230 msgr2=0x7fdedc19cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdedaffd700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdedc10c230 0x7fdedc19cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdedaffd700 1 -- 172.21.15.150:0/440508715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec40095f0 con 0x7fdedc102e00 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdeda7fc700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdedc103750 0x7fdedc198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdedaffd700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdedc102e00 0x7fdedc198490 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fded4007f10 tx=0x7fded400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdee0936700 1 -- 172.21.15.150:0/440508715 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded4017070 con 0x7fdedc102e00 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdee0936700 1 -- 172.21.15.150:0/440508715 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fded400f040 con 0x7fdedc102e00 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdee1938700 1 -- 172.21.15.150:0/440508715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdedc19dac0 con 0x7fdedc102e00 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdee0936700 1 -- 172.21.15.150:0/440508715 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded40046f0 con 0x7fdedc102e00 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.384+0000 7fdee1938700 1 -- 172.21.15.150:0/440508715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdedc1a53c0 con 0x7fdedc102e00 2024-05-10T04:03:11.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.385+0000 7fdee1938700 1 -- 172.21.15.150:0/440508715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdedc0695a0 con 0x7fdedc102e00 2024-05-10T04:03:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.387+0000 7fdee0936700 1 -- 172.21.15.150:0/440508715 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fded4003a00 con 0x7fdedc102e00 2024-05-10T04:03:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.387+0000 7fdee0936700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdec80a11b0 0x7fdec80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.387+0000 7fdeda7fc700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdec80a11b0 0x7fdec80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.387+0000 7fdee0936700 1 -- 172.21.15.150:0/440508715 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fded40ccea0 con 0x7fdedc102e00 2024-05-10T04:03:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.387+0000 7fdee0936700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215] conn(0x7fdec80abd80 0x7fdec80ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.387+0000 7fdee0936700 1 -- 172.21.15.150:0/440508715 --> [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdec80ae8e0 con 0x7fdec80abd80 2024-05-10T04:03:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.387+0000 7fdee0936700 1 -- 172.21.15.150:0/440508715 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fded40cd2b0 con 0x7fdedc102e00 2024-05-10T04:03:11.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.388+0000 7fdedb7fe700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215] conn(0x7fdec80abd80 0x7fdec80ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:11.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.388+0000 7fdeda7fc700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdec80a11b0 0x7fdec80a36c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0ffbd0 tx=0x7fdec400b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:11.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.388+0000 7fdedb7fe700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215] conn(0x7fdec80abd80 0x7fdec80ae210 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:11.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.389+0000 7fdee0936700 1 -- 172.21.15.150:0/440508715 <== osd.0 v2:172.21.15.150:6802/2214000215 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fdec80ae8e0 con 0x7fdec80abd80 2024-05-10T04:03:11.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.404+0000 7fdee1938700 1 -- 172.21.15.150:0/440508715 --> [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdedc0618d0 con 0x7fdec80abd80 2024-05-10T04:03:11.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.405+0000 7fdee0936700 1 -- 172.21.15.150:0/440508715 <== osd.0 v2:172.21.15.150:6802/2214000215 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdedc0618d0 con 0x7fdec80abd80 2024-05-10T04:03:11.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.405+0000 7fdee1938700 1 -- 172.21.15.150:0/440508715 >> [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215] conn(0x7fdec80abd80 msgr2=0x7fdec80ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:11.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.405+0000 7fdee1938700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215] conn(0x7fdec80abd80 0x7fdec80ae210 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:11.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.405+0000 7fdee1938700 1 -- 172.21.15.150:0/440508715 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdec80a11b0 msgr2=0x7fdec80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:11.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.405+0000 7fdee1938700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdec80a11b0 0x7fdec80a36c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0ffbd0 tx=0x7fdec400b580 comp rx=0 tx=0).stop 2024-05-10T04:03:11.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.405+0000 7fdee1938700 1 -- 172.21.15.150:0/440508715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdedc102e00 msgr2=0x7fdedc198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:11.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.405+0000 7fdee1938700 1 --2- 172.21.15.150:0/440508715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdedc102e00 0x7fdedc198490 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fded4007f10 tx=0x7fded400d3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:11.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.405+0000 7fdedb7fe700 1 -- 172.21.15.150:0/440508715 reap_dead start 2024-05-10T04:03:11.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.405+0000 7fdee1938700 1 -- 172.21.15.150:0/440508715 shutdown_connections 2024-05-10T04:03:11.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.405+0000 7fdee1938700 1 -- 172.21.15.150:0/440508715 >> 172.21.15.150:0/440508715 conn(0x7fdedc0fc7a0 msgr2=0x7fdedc0fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:11.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.405+0000 7fdee1938700 1 -- 172.21.15.150:0/440508715 shutdown_connections 2024-05-10T04:03:11.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:11.405+0000 7fdee1938700 1 -- 172.21.15.150:0/440508715 wait complete. 2024-05-10T04:03:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.243+0000 7fd57998a700 1 -- 172.21.15.150:0/534988144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd574105750 msgr2=0x7fd574105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.243+0000 7fd57998a700 1 --2- 172.21.15.150:0/534988144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd574105750 0x7fd574105b60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd568009990 tx=0x7fd568009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.243+0000 7fd57998a700 1 -- 172.21.15.150:0/534988144 shutdown_connections 2024-05-10T04:03:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.243+0000 7fd57998a700 1 --2- 172.21.15.150:0/534988144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5741060a0 0x7fd57410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.243+0000 7fd57998a700 1 --2- 172.21.15.150:0/534988144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd574100e20 0x7fd574101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.243+0000 7fd57998a700 1 --2- 172.21.15.150:0/534988144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd574105750 0x7fd574105b60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:12.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.243+0000 7fd57998a700 1 -- 172.21.15.150:0/534988144 >> 172.21.15.150:0/534988144 conn(0x7fd5740fc7a0 msgr2=0x7fd5740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:12.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.243+0000 7fd57998a700 1 -- 172.21.15.150:0/534988144 shutdown_connections 2024-05-10T04:03:12.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.243+0000 7fd57998a700 1 -- 172.21.15.150:0/534988144 wait complete. 2024-05-10T04:03:12.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.244+0000 7fd57998a700 1 Processor -- start 2024-05-10T04:03:12.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.244+0000 7fd57998a700 1 -- start start 2024-05-10T04:03:12.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd57998a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd574100e20 0x7fd5741984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:12.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd57998a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd574105750 0x7fd574198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:12.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd57998a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5741060a0 0x7fd57419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:12.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd57998a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57419d4c0 con 0x7fd574100e20 2024-05-10T04:03:12.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd57998a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57419d630 con 0x7fd574105750 2024-05-10T04:03:12.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd57998a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57419d7f0 con 0x7fd5741060a0 2024-05-10T04:03:12.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd572ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd574100e20 0x7fd5741984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:12.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd572ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd574100e20 0x7fd5741984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54504/0 (socket says 172.21.15.150:54504) 2024-05-10T04:03:12.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd572ffd700 1 -- 172.21.15.150:0/2351008512 learned_addr learned my addr 172.21.15.150:0/2351008512 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:12.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd5737fe700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5741060a0 0x7fd57419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:12.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd5727fc700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd574105750 0x7fd574198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:12.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd5727fc700 1 -- 172.21.15.150:0/2351008512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5741060a0 msgr2=0x7fd57419cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:12.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd5727fc700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5741060a0 0x7fd57419cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:12.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd5727fc700 1 -- 172.21.15.150:0/2351008512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd574100e20 msgr2=0x7fd5741984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:12.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd5727fc700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd574100e20 0x7fd5741984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:12.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd5727fc700 1 -- 172.21.15.150:0/2351008512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5680095f0 con 0x7fd574105750 2024-05-10T04:03:12.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.245+0000 7fd5737fe700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5741060a0 0x7fd57419cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:03:12.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.246+0000 7fd5727fc700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd574105750 0x7fd574198a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5680098e0 tx=0x7fd568004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:12.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.246+0000 7fd578988700 1 -- 172.21.15.150:0/2351008512 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568021070 con 0x7fd574105750 2024-05-10T04:03:12.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.246+0000 7fd57998a700 1 -- 172.21.15.150:0/2351008512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57419da70 con 0x7fd574105750 2024-05-10T04:03:12.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.246+0000 7fd578988700 1 -- 172.21.15.150:0/2351008512 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd56800bbf0 con 0x7fd574105750 2024-05-10T04:03:12.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.246+0000 7fd578988700 1 -- 172.21.15.150:0/2351008512 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568027420 con 0x7fd574105750 2024-05-10T04:03:12.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.246+0000 7fd57998a700 1 -- 172.21.15.150:0/2351008512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5740fcbb0 con 0x7fd574105750 2024-05-10T04:03:12.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.247+0000 7fd57998a700 1 -- 172.21.15.150:0/2351008512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5741022d0 con 0x7fd574105750 2024-05-10T04:03:12.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.248+0000 7fd578988700 1 -- 172.21.15.150:0/2351008512 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd568025070 con 0x7fd574105750 2024-05-10T04:03:12.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.249+0000 7fd578988700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd55c0a11b0 0x7fd55c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:12.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.249+0000 7fd578988700 1 -- 172.21.15.150:0/2351008512 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd5680d2d50 con 0x7fd574105750 2024-05-10T04:03:12.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.249+0000 7fd572ffd700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd55c0a11b0 0x7fd55c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:12.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.249+0000 7fd578988700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933] conn(0x7fd55c0abd80 0x7fd55c0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:12.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.249+0000 7fd578988700 1 -- 172.21.15.150:0/2351008512 --> [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd55c0ae8e0 con 0x7fd55c0abd80 2024-05-10T04:03:12.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.249+0000 7fd578988700 1 -- 172.21.15.150:0/2351008512 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5680d30c0 con 0x7fd574105750 2024-05-10T04:03:12.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.249+0000 7fd5737fe700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933] conn(0x7fd55c0abd80 0x7fd55c0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:12.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.249+0000 7fd572ffd700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd55c0a11b0 0x7fd55c0a36c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd564005950 tx=0x7fd56400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:12.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.250+0000 7fd5737fe700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933] conn(0x7fd55c0abd80 0x7fd55c0ae210 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:12.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.253+0000 7fd578988700 1 -- 172.21.15.150:0/2351008512 <== osd.10 v2:172.21.15.161:6824/1965594933 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd55c0ae8e0 con 0x7fd55c0abd80 2024-05-10T04:03:12.269 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:12 smithi150 ceph-mon[42477]: pgmap v246: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-05-10T04:03:12.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.269+0000 7fd57998a700 1 -- 172.21.15.150:0/2351008512 --> [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd574061890 con 0x7fd55c0abd80 2024-05-10T04:03:12.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.270+0000 7fd578988700 1 -- 172.21.15.150:0/2351008512 <== osd.10 v2:172.21.15.161:6824/1965594933 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd574061890 con 0x7fd55c0abd80 2024-05-10T04:03:12.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.270+0000 7fd57998a700 1 -- 172.21.15.150:0/2351008512 >> [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933] conn(0x7fd55c0abd80 msgr2=0x7fd55c0ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:12.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.270+0000 7fd57998a700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933] conn(0x7fd55c0abd80 0x7fd55c0ae210 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:12.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.270+0000 7fd57998a700 1 -- 172.21.15.150:0/2351008512 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd55c0a11b0 msgr2=0x7fd55c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:12.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.270+0000 7fd57998a700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd55c0a11b0 0x7fd55c0a36c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd564005950 tx=0x7fd56400b500 comp rx=0 tx=0).stop 2024-05-10T04:03:12.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.270+0000 7fd57998a700 1 -- 172.21.15.150:0/2351008512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd574105750 msgr2=0x7fd574198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:12.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.270+0000 7fd57998a700 1 --2- 172.21.15.150:0/2351008512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd574105750 0x7fd574198a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5680098e0 tx=0x7fd568004730 comp rx=0 tx=0).stop 2024-05-10T04:03:12.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.270+0000 7fd5737fe700 1 -- 172.21.15.150:0/2351008512 reap_dead start 2024-05-10T04:03:12.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.270+0000 7fd57998a700 1 -- 172.21.15.150:0/2351008512 shutdown_connections 2024-05-10T04:03:12.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.270+0000 7fd57998a700 1 -- 172.21.15.150:0/2351008512 >> 172.21.15.150:0/2351008512 conn(0x7fd5740fc7a0 msgr2=0x7fd574069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:12.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.271+0000 7fd57998a700 1 -- 172.21.15.150:0/2351008512 shutdown_connections 2024-05-10T04:03:12.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.271+0000 7fd57998a700 1 -- 172.21.15.150:0/2351008512 wait complete. 2024-05-10T04:03:12.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:12 smithi181 ceph-mon[46602]: pgmap v246: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-05-10T04:03:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:12 smithi161 ceph-mon[46918]: pgmap v246: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-05-10T04:03:13.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.975+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/1507480946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc10c260 msgr2=0x7fb9bc10e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:13.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.975+0000 7fb9c2d2a700 1 --2- 172.21.15.150:0/1507480946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc10c260 0x7fb9bc10e770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b0a0 tx=0x7fb9b400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:13.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.975+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/1507480946 shutdown_connections 2024-05-10T04:03:13.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.975+0000 7fb9c2d2a700 1 --2- 172.21.15.150:0/1507480946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc10c260 0x7fb9bc10e770 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:13.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.975+0000 7fb9c2d2a700 1 --2- 172.21.15.150:0/1507480946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9bc102df0 0x7fb9bc103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:13.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.975+0000 7fb9c2d2a700 1 --2- 172.21.15.150:0/1507480946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9bc103740 0x7fb9bc10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:13.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.975+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/1507480946 >> 172.21.15.150:0/1507480946 conn(0x7fb9bc0fc7a0 msgr2=0x7fb9bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:13.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.975+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/1507480946 shutdown_connections 2024-05-10T04:03:13.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.975+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/1507480946 wait complete. 2024-05-10T04:03:13.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c2d2a700 1 Processor -- start 2024-05-10T04:03:13.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c2d2a700 1 -- start start 2024-05-10T04:03:13.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c2d2a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9bc102df0 0x7fb9bc19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:13.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c2d2a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9bc103740 0x7fb9bc19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:13.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c2d2a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc10c260 0x7fb9bc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:13.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c2d2a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc198a00 con 0x7fb9bc10c260 2024-05-10T04:03:13.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c2d2a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc198b70 con 0x7fb9bc102df0 2024-05-10T04:03:13.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c2d2a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc198d30 con 0x7fb9bc103740 2024-05-10T04:03:13.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c12c7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc10c260 0x7fb9bc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:13.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c12c7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc10c260 0x7fb9bc198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54520/0 (socket says 172.21.15.150:54520) 2024-05-10T04:03:13.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c12c7700 1 -- 172.21.15.150:0/846093195 learned_addr learned my addr 172.21.15.150:0/846093195 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:13.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9bbfff700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9bc103740 0x7fb9bc19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c0ac6700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9bc102df0 0x7fb9bc19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c12c7700 1 -- 172.21.15.150:0/846093195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9bc103740 msgr2=0x7fb9bc19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c12c7700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9bc103740 0x7fb9bc19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c12c7700 1 -- 172.21.15.150:0/846093195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9bc102df0 msgr2=0x7fb9bc19e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c12c7700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9bc102df0 0x7fb9bc19e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.976+0000 7fb9c12c7700 1 -- 172.21.15.150:0/846093195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b4009d00 con 0x7fb9bc10c260 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.977+0000 7fb9c12c7700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc10c260 0x7fb9bc198430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b070 tx=0x7fb9b4007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.977+0000 7fb9b9ffb700 1 -- 172.21.15.150:0/846093195 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b400e040 con 0x7fb9bc10c260 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.977+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/846093195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9bc198fb0 con 0x7fb9bc10c260 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.977+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/846093195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9bc1a54a0 con 0x7fb9bc10c260 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.977+0000 7fb9b9ffb700 1 -- 172.21.15.150:0/846093195 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9b400be40 con 0x7fb9bc10c260 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.977+0000 7fb9b9ffb700 1 -- 172.21.15.150:0/846093195 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40052a0 con 0x7fb9bc10c260 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.978+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/846093195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb9a0000fc0 con 0x7fb9bc10c260 2024-05-10T04:03:13.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.979+0000 7fb9b9ffb700 1 -- 172.21.15.150:0/846093195 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb9b4004c50 con 0x7fb9bc10c260 2024-05-10T04:03:13.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.980+0000 7fb9b9ffb700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb9ac0a1230 0x7fb9ac0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:13.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.980+0000 7fb9c0ac6700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb9ac0a1230 0x7fb9ac0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:13.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.980+0000 7fb9b9ffb700 1 -- 172.21.15.150:0/846093195 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb9b40ce7e0 con 0x7fb9bc10c260 2024-05-10T04:03:13.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.980+0000 7fb9b9ffb700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813] conn(0x7fb9ac0abe00 0x7fb9ac0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:13.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.980+0000 7fb9b9ffb700 1 -- 172.21.15.150:0/846093195 --> [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9ac0ae960 con 0x7fb9ac0abe00 2024-05-10T04:03:13.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.980+0000 7fb9b9ffb700 1 -- 172.21.15.150:0/846093195 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9b40ceb50 con 0x7fb9bc10c260 2024-05-10T04:03:13.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.980+0000 7fb9c0ac6700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb9ac0a1230 0x7fb9ac0a3740 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc1998a0 tx=0x7fb9b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:13.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.981+0000 7fb9bbfff700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813] conn(0x7fb9ac0abe00 0x7fb9ac0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:13.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.981+0000 7fb9bbfff700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813] conn(0x7fb9ac0abe00 0x7fb9ac0ae290 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:13.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.982+0000 7fb9b9ffb700 1 -- 172.21.15.150:0/846093195 <== osd.8 v2:172.21.15.181:6816/3605634813 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb9ac0ae960 con 0x7fb9ac0abe00 2024-05-10T04:03:13.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.998+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/846093195 --> [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9a0002d40 con 0x7fb9ac0abe00 2024-05-10T04:03:13.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.998+0000 7fb9b9ffb700 1 -- 172.21.15.150:0/846093195 <== osd.8 v2:172.21.15.181:6816/3605634813 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb9a0002d40 con 0x7fb9ac0abe00 2024-05-10T04:03:13.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.999+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/846093195 >> [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813] conn(0x7fb9ac0abe00 msgr2=0x7fb9ac0ae290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:13.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.999+0000 7fb9c2d2a700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813] conn(0x7fb9ac0abe00 0x7fb9ac0ae290 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:13.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.999+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/846093195 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb9ac0a1230 msgr2=0x7fb9ac0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:13.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.999+0000 7fb9c2d2a700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb9ac0a1230 0x7fb9ac0a3740 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc1998a0 tx=0x7fb9b0009260 comp rx=0 tx=0).stop 2024-05-10T04:03:13.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.999+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/846093195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc10c260 msgr2=0x7fb9bc198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:13.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.999+0000 7fb9c2d2a700 1 --2- 172.21.15.150:0/846093195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc10c260 0x7fb9bc198430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b070 tx=0x7fb9b4007930 comp rx=0 tx=0).stop 2024-05-10T04:03:13.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.999+0000 7fb9c12c7700 1 -- 172.21.15.150:0/846093195 reap_dead start 2024-05-10T04:03:13.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.999+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/846093195 shutdown_connections 2024-05-10T04:03:13.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.999+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/846093195 >> 172.21.15.150:0/846093195 conn(0x7fb9bc0fc7a0 msgr2=0x7fb9bc0fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:13.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.999+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/846093195 shutdown_connections 2024-05-10T04:03:13.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:12.999+0000 7fb9c2d2a700 1 -- 172.21.15.150:0/846093195 wait complete. 2024-05-10T04:03:13.037 INFO:teuthology.orchestra.run.smithi150.stdout:34359738424 2024-05-10T04:03:13.038 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-10T04:03:13.351 INFO:teuthology.orchestra.run.smithi150.stdout:292057776150 2024-05-10T04:03:13.352 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-10T04:03:14.227 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:14.292 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:14 smithi150 ceph-mon[42477]: pgmap v247: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-10T04:03:14.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:14 smithi181 ceph-mon[46602]: pgmap v247: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-10T04:03:14.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:14 smithi161 ceph-mon[46918]: pgmap v247: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-10T04:03:14.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.933+0000 7f0ad758d700 1 -- 172.21.15.150:0/2399767466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0106c60 msgr2=0x7f0ad0107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:14.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.933+0000 7f0ad758d700 1 --2- 172.21.15.150:0/2399767466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0106c60 0x7f0ad0107070 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009990 tx=0x7f0acc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:14.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.933+0000 7f0ad758d700 1 -- 172.21.15.150:0/2399767466 shutdown_connections 2024-05-10T04:03:14.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.933+0000 7f0ad758d700 1 --2- 172.21.15.150:0/2399767466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ad0100a40 0x7f0ad010ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.933+0000 7f0ad758d700 1 --2- 172.21.15.150:0/2399767466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0106c60 0x7f0ad0107070 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.933+0000 7f0ad758d700 1 --2- 172.21.15.150:0/2399767466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad01000f0 0x7f0ad0100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.933+0000 7f0ad758d700 1 -- 172.21.15.150:0/2399767466 >> 172.21.15.150:0/2399767466 conn(0x7f0ad00fbed0 msgr2=0x7f0ad00fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.933+0000 7f0ad758d700 1 -- 172.21.15.150:0/2399767466 shutdown_connections 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.933+0000 7f0ad758d700 1 -- 172.21.15.150:0/2399767466 wait complete. 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.934+0000 7f0ad758d700 1 Processor -- start 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.934+0000 7f0ad758d700 1 -- start start 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.934+0000 7f0ad758d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad01000f0 0x7f0ad019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.934+0000 7f0ad758d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ad0100a40 0x7f0ad019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.934+0000 7f0ad758d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0106c60 0x7f0ad01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.934+0000 7f0ad758d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad01989a0 con 0x7f0ad01000f0 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.934+0000 7f0ad758d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad0198b10 con 0x7f0ad0100a40 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.934+0000 7f0ad758d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad0198cd0 con 0x7f0ad0106c60 2024-05-10T04:03:14.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.935+0000 7f0ad5329700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad01000f0 0x7f0ad019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.935+0000 7f0ad4b28700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ad0100a40 0x7f0ad019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.935+0000 7f0ad5329700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad01000f0 0x7f0ad019e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54540/0 (socket says 172.21.15.150:54540) 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.935+0000 7f0ad5329700 1 -- 172.21.15.150:0/2685220629 learned_addr learned my addr 172.21.15.150:0/2685220629 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.935+0000 7f0ad5b2a700 1 --2- 172.21.15.150:0/2685220629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0106c60 0x7f0ad01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.935+0000 7f0ad4b28700 1 -- 172.21.15.150:0/2685220629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0106c60 msgr2=0x7f0ad01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.935+0000 7f0ad4b28700 1 --2- 172.21.15.150:0/2685220629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0106c60 0x7f0ad01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.935+0000 7f0ad4b28700 1 -- 172.21.15.150:0/2685220629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad01000f0 msgr2=0x7f0ad019e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.935+0000 7f0ad4b28700 1 --2- 172.21.15.150:0/2685220629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad01000f0 0x7f0ad019e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.935+0000 7f0ad4b28700 1 -- 172.21.15.150:0/2685220629 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0acc0095f0 con 0x7f0ad0100a40 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.935+0000 7f0ad4b28700 1 --2- 172.21.15.150:0/2685220629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ad0100a40 0x7f0ad019e9b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000c940 tx=0x7f0ac000cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.936+0000 7f0ac67fc700 1 -- 172.21.15.150:0/2685220629 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0004a30 con 0x7f0ad0100a40 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.936+0000 7f0ad758d700 1 -- 172.21.15.150:0/2685220629 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad0198fb0 con 0x7f0ad0100a40 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.936+0000 7f0ac67fc700 1 -- 172.21.15.150:0/2685220629 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ac00054e0 con 0x7f0ad0100a40 2024-05-10T04:03:14.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.936+0000 7f0ac67fc700 1 -- 172.21.15.150:0/2685220629 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac000eac0 con 0x7f0ad0100a40 2024-05-10T04:03:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.937+0000 7f0ad758d700 1 -- 172.21.15.150:0/2685220629 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad01a5440 con 0x7f0ad0100a40 2024-05-10T04:03:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.937+0000 7f0ad758d700 1 -- 172.21.15.150:0/2685220629 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ad0111f30 con 0x7f0ad0100a40 2024-05-10T04:03:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.938+0000 7f0ac67fc700 1 -- 172.21.15.150:0/2685220629 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0ac0004b90 con 0x7f0ad0100a40 2024-05-10T04:03:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.939+0000 7f0ac67fc700 1 --2- 172.21.15.150:0/2685220629 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0abc0a11b0 0x7f0abc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.939+0000 7f0ac67fc700 1 -- 172.21.15.150:0/2685220629 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0ac00cc750 con 0x7f0ad0100a40 2024-05-10T04:03:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.939+0000 7f0ad5329700 1 --2- 172.21.15.150:0/2685220629 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0abc0a11b0 0x7f0abc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.939+0000 7f0ac67fc700 1 --2- 172.21.15.150:0/2685220629 >> [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921] conn(0x7f0abc0abd80 0x7f0abc0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.939+0000 7f0ac67fc700 1 -- 172.21.15.150:0/2685220629 --> [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0abc0ae8e0 con 0x7f0abc0abd80 2024-05-10T04:03:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.939+0000 7f0ac67fc700 1 -- 172.21.15.150:0/2685220629 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ac00cedd0 con 0x7f0ad0100a40 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.940+0000 7f0ad5b2a700 1 --2- 172.21.15.150:0/2685220629 >> [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921] conn(0x7f0abc0abd80 0x7f0abc0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.940+0000 7f0ad5329700 1 --2- 172.21.15.150:0/2685220629 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0abc0a11b0 0x7f0abc0a36c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0098e0 tx=0x7f0acc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.940+0000 7f0ad5b2a700 1 --2- 172.21.15.150:0/2685220629 >> [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921] conn(0x7f0abc0abd80 0x7f0abc0ae210 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.941+0000 7f0ac67fc700 1 -- 172.21.15.150:0/2685220629 <== osd.11 v2:172.21.15.181:6824/1224976921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f0abc0ae8e0 con 0x7f0abc0abd80 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.958+0000 7f0ad758d700 1 -- 172.21.15.150:0/2685220629 --> [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ad0061890 con 0x7f0abc0abd80 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.958+0000 7f0ac67fc700 1 -- 172.21.15.150:0/2685220629 <== osd.11 v2:172.21.15.181:6824/1224976921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0ad0061890 con 0x7f0abc0abd80 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.959+0000 7f0ad758d700 1 -- 172.21.15.150:0/2685220629 >> [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921] conn(0x7f0abc0abd80 msgr2=0x7f0abc0ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.959+0000 7f0ad758d700 1 --2- 172.21.15.150:0/2685220629 >> [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921] conn(0x7f0abc0abd80 0x7f0abc0ae210 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.959+0000 7f0ad758d700 1 -- 172.21.15.150:0/2685220629 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0abc0a11b0 msgr2=0x7f0abc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.959+0000 7f0ad758d700 1 --2- 172.21.15.150:0/2685220629 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0abc0a11b0 0x7f0abc0a36c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0098e0 tx=0x7f0acc005960 comp rx=0 tx=0).stop 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.959+0000 7f0ad758d700 1 -- 172.21.15.150:0/2685220629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ad0100a40 msgr2=0x7f0ad019e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.959+0000 7f0ad758d700 1 --2- 172.21.15.150:0/2685220629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ad0100a40 0x7f0ad019e9b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000c940 tx=0x7f0ac000cc50 comp rx=0 tx=0).stop 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.959+0000 7f0ad5b2a700 1 -- 172.21.15.150:0/2685220629 reap_dead start 2024-05-10T04:03:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.959+0000 7f0ad758d700 1 -- 172.21.15.150:0/2685220629 shutdown_connections 2024-05-10T04:03:14.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.959+0000 7f0ad758d700 1 -- 172.21.15.150:0/2685220629 >> 172.21.15.150:0/2685220629 conn(0x7f0ad00fbed0 msgr2=0x7f0ad01080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:14.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.959+0000 7f0ad758d700 1 -- 172.21.15.150:0/2685220629 shutdown_connections 2024-05-10T04:03:14.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:13.959+0000 7f0ad758d700 1 -- 172.21.15.150:0/2685220629 wait complete. 2024-05-10T04:03:14.930 INFO:teuthology.orchestra.run.smithi150.stdout:240518168605 2024-05-10T04:03:14.931 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-10T04:03:15.178 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:15.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.268+0000 7fe4e208a700 1 -- 172.21.15.150:0/1871189424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4dc100a40 msgr2=0x7fe4dc10ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.268+0000 7fe4e208a700 1 --2- 172.21.15.150:0/1871189424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4dc100a40 0x7fe4dc10ead0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400b0a0 tx=0x7fe4d400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.268+0000 7fe4e208a700 1 -- 172.21.15.150:0/1871189424 shutdown_connections 2024-05-10T04:03:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.268+0000 7fe4e208a700 1 --2- 172.21.15.150:0/1871189424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4dc100a40 0x7fe4dc10ead0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.268+0000 7fe4e208a700 1 --2- 172.21.15.150:0/1871189424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4dc106c60 0x7fe4dc107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.268+0000 7fe4e208a700 1 --2- 172.21.15.150:0/1871189424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4dc1000f0 0x7fe4dc100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.268+0000 7fe4e208a700 1 -- 172.21.15.150:0/1871189424 >> 172.21.15.150:0/1871189424 conn(0x7fe4dc0fbed0 msgr2=0x7fe4dc0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.268+0000 7fe4e208a700 1 -- 172.21.15.150:0/1871189424 shutdown_connections 2024-05-10T04:03:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.268+0000 7fe4e208a700 1 -- 172.21.15.150:0/1871189424 wait complete. 2024-05-10T04:03:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.269+0000 7fe4e208a700 1 Processor -- start 2024-05-10T04:03:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.269+0000 7fe4e208a700 1 -- start start 2024-05-10T04:03:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4e208a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4dc1000f0 0x7fe4dc19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4e208a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4dc100a40 0x7fe4dc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4e208a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4dc106c60 0x7fe4dc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4e208a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc198bb0 con 0x7fe4dc1000f0 2024-05-10T04:03:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4e208a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc198d20 con 0x7fe4dc100a40 2024-05-10T04:03:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4e208a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc198e90 con 0x7fe4dc106c60 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4db7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4dc1000f0 0x7fe4dc19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4db7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4dc1000f0 0x7fe4dc19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54546/0 (socket says 172.21.15.150:54546) 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4db7fe700 1 -- 172.21.15.150:0/114687263 learned_addr learned my addr 172.21.15.150:0/114687263 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4daffd700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4dc100a40 0x7fe4dc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4dbfff700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4dc106c60 0x7fe4dc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4db7fe700 1 -- 172.21.15.150:0/114687263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4dc106c60 msgr2=0x7fe4dc1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4db7fe700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4dc106c60 0x7fe4dc1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4db7fe700 1 -- 172.21.15.150:0/114687263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4dc100a40 msgr2=0x7fe4dc19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4db7fe700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4dc100a40 0x7fe4dc19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4db7fe700 1 -- 172.21.15.150:0/114687263 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d4009d00 con 0x7fe4dc1000f0 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4db7fe700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4dc1000f0 0x7fe4dc19e640 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400c970 tx=0x7fe4c400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4d8ff9700 1 -- 172.21.15.150:0/114687263 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4004220 con 0x7fe4dc1000f0 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.270+0000 7fe4d8ff9700 1 -- 172.21.15.150:0/114687263 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4c400ce60 con 0x7fe4dc1000f0 2024-05-10T04:03:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.271+0000 7fe4e208a700 1 -- 172.21.15.150:0/114687263 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4dc199170 con 0x7fe4dc1000f0 2024-05-10T04:03:15.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.271+0000 7fe4d8ff9700 1 -- 172.21.15.150:0/114687263 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4003c60 con 0x7fe4dc1000f0 2024-05-10T04:03:15.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.271+0000 7fe4e208a700 1 -- 172.21.15.150:0/114687263 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc1a56a0 con 0x7fe4dc1000f0 2024-05-10T04:03:15.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.271+0000 7fe4e208a700 1 -- 172.21.15.150:0/114687263 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe4dc111fc0 con 0x7fe4dc1000f0 2024-05-10T04:03:15.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.273+0000 7fe4d8ff9700 1 -- 172.21.15.150:0/114687263 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe4c4007900 con 0x7fe4dc1000f0 2024-05-10T04:03:15.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.273+0000 7fe4d8ff9700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe4c80a11b0 0x7fe4c80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:15.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.273+0000 7fe4daffd700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe4c80a11b0 0x7fe4c80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:15.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.273+0000 7fe4d8ff9700 1 -- 172.21.15.150:0/114687263 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe4c40ccd60 con 0x7fe4dc1000f0 2024-05-10T04:03:15.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.274+0000 7fe4d8ff9700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780] conn(0x7fe4c80abd80 0x7fe4c80ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:15.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.274+0000 7fe4d8ff9700 1 -- 172.21.15.150:0/114687263 --> [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe4c80ae8e0 con 0x7fe4c80abd80 2024-05-10T04:03:15.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.274+0000 7fe4d8ff9700 1 -- 172.21.15.150:0/114687263 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4c40cd170 con 0x7fe4dc1000f0 2024-05-10T04:03:15.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.274+0000 7fe4daffd700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe4c80a11b0 0x7fe4c80a36c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc1052a0 tx=0x7fe4cc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:15.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.274+0000 7fe4dbfff700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780] conn(0x7fe4c80abd80 0x7fe4c80ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:15.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.274+0000 7fe4dbfff700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780] conn(0x7fe4c80abd80 0x7fe4c80ae210 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:15.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.275+0000 7fe4d8ff9700 1 -- 172.21.15.150:0/114687263 <== osd.5 v2:172.21.15.181:6808/1962584780 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe4c80ae8e0 con 0x7fe4c80abd80 2024-05-10T04:03:15.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.291+0000 7fe4e208a700 1 -- 172.21.15.150:0/114687263 --> [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe4dc061890 con 0x7fe4c80abd80 2024-05-10T04:03:15.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.292+0000 7fe4d8ff9700 1 -- 172.21.15.150:0/114687263 <== osd.5 v2:172.21.15.181:6808/1962584780 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe4dc061890 con 0x7fe4c80abd80 2024-05-10T04:03:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.292+0000 7fe4e208a700 1 -- 172.21.15.150:0/114687263 >> [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780] conn(0x7fe4c80abd80 msgr2=0x7fe4c80ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.292+0000 7fe4e208a700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780] conn(0x7fe4c80abd80 0x7fe4c80ae210 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.292+0000 7fe4e208a700 1 -- 172.21.15.150:0/114687263 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe4c80a11b0 msgr2=0x7fe4c80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.292+0000 7fe4e208a700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe4c80a11b0 0x7fe4c80a36c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc1052a0 tx=0x7fe4cc009190 comp rx=0 tx=0).stop 2024-05-10T04:03:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.292+0000 7fe4e208a700 1 -- 172.21.15.150:0/114687263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4dc1000f0 msgr2=0x7fe4dc19e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.292+0000 7fe4e208a700 1 --2- 172.21.15.150:0/114687263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4dc1000f0 0x7fe4dc19e640 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400c970 tx=0x7fe4c400cc80 comp rx=0 tx=0).stop 2024-05-10T04:03:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.292+0000 7fe4dbfff700 1 -- 172.21.15.150:0/114687263 reap_dead start 2024-05-10T04:03:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.292+0000 7fe4e208a700 1 -- 172.21.15.150:0/114687263 shutdown_connections 2024-05-10T04:03:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.292+0000 7fe4e208a700 1 -- 172.21.15.150:0/114687263 >> 172.21.15.150:0/114687263 conn(0x7fe4dc0fbed0 msgr2=0x7fe4dc1080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.292+0000 7fe4e208a700 1 -- 172.21.15.150:0/114687263 shutdown_connections 2024-05-10T04:03:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.293+0000 7fe4e208a700 1 -- 172.21.15.150:0/114687263 wait complete. 2024-05-10T04:03:15.368 INFO:teuthology.orchestra.run.smithi150.stdout:326417514515 2024-05-10T04:03:15.369 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-10T04:03:15.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.522+0000 7f95f7117700 1 -- 172.21.15.150:0/1043953632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f010a000 msgr2=0x7f95f010a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.522+0000 7f95f7117700 1 --2- 172.21.15.150:0/1043953632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f010a000 0x7f95f010a410 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009a10 tx=0x7f95dc009d20 comp rx=0 tx=0).stop 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.522+0000 7f95f7117700 1 -- 172.21.15.150:0/1043953632 shutdown_connections 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.522+0000 7f95f7117700 1 --2- 172.21.15.150:0/1043953632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f010a950 0x7f95f0116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.522+0000 7f95f7117700 1 --2- 172.21.15.150:0/1043953632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f010e460 0x7f95f010e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.522+0000 7f95f7117700 1 --2- 172.21.15.150:0/1043953632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f010a000 0x7f95f010a410 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.522+0000 7f95f7117700 1 -- 172.21.15.150:0/1043953632 >> 172.21.15.150:0/1043953632 conn(0x7f95f006ce00 msgr2=0x7f95f006d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.523+0000 7f95f7117700 1 -- 172.21.15.150:0/1043953632 shutdown_connections 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.523+0000 7f95f7117700 1 -- 172.21.15.150:0/1043953632 wait complete. 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.524+0000 7f95f7117700 1 Processor -- start 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.524+0000 7f95f7117700 1 -- start start 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.524+0000 7f95f7117700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f010a000 0x7f95f01a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.524+0000 7f95f7117700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f010a950 0x7f95f01a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.524+0000 7f95f7117700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f010e460 0x7f95f019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:15.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.524+0000 7f95f7117700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f019f080 con 0x7f95f010e460 2024-05-10T04:03:15.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.524+0000 7f95f7117700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f019f1f0 con 0x7f95f010a000 2024-05-10T04:03:15.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.524+0000 7f95f7117700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f019f3b0 con 0x7f95f010a950 2024-05-10T04:03:15.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.524+0000 7f95f56b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f010e460 0x7f95f019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.524+0000 7f95f56b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f010e460 0x7f95f019ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54578/0 (socket says 172.21.15.150:54578) 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.524+0000 7f95f56b4700 1 -- 172.21.15.150:0/1998025707 learned_addr learned my addr 172.21.15.150:0/1998025707 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.525+0000 7f95effff700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f010a950 0x7f95f01a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.525+0000 7f95f4eb3700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f010a000 0x7f95f01a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.525+0000 7f95effff700 1 -- 172.21.15.150:0/1998025707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f010a000 msgr2=0x7f95f01a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.525+0000 7f95effff700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f010a000 0x7f95f01a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.525+0000 7f95effff700 1 -- 172.21.15.150:0/1998025707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f010e460 msgr2=0x7f95f019ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.525+0000 7f95effff700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f010e460 0x7f95f019ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.525+0000 7f95effff700 1 -- 172.21.15.150:0/1998025707 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e4009520 con 0x7f95f010a950 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.525+0000 7f95effff700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f010a950 0x7f95f01a4010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f95dc000c00 tx=0x7f95dc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.526+0000 7f95edffb700 1 -- 172.21.15.150:0/1998025707 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc01d070 con 0x7f95f010a950 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.526+0000 7f95edffb700 1 -- 172.21.15.150:0/1998025707 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95dc00fb30 con 0x7f95f010a950 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.526+0000 7f95f7117700 1 -- 172.21.15.150:0/1998025707 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc0095f0 con 0x7f95f010a950 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.526+0000 7f95edffb700 1 -- 172.21.15.150:0/1998025707 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc022ae0 con 0x7f95f010a950 2024-05-10T04:03:15.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.526+0000 7f95f7117700 1 -- 172.21.15.150:0/1998025707 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f019f860 con 0x7f95f010a950 2024-05-10T04:03:15.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.527+0000 7f95f7117700 1 -- 172.21.15.150:0/1998025707 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f95f01089c0 con 0x7f95f010a950 2024-05-10T04:03:15.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.529+0000 7f95edffb700 1 -- 172.21.15.150:0/1998025707 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f95dc00fca0 con 0x7f95f010a950 2024-05-10T04:03:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.529+0000 7f95edffb700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f95e00a11b0 0x7f95e00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.530+0000 7f95f4eb3700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f95e00a11b0 0x7f95e00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.530+0000 7f95edffb700 1 -- 172.21.15.150:0/1998025707 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f95dc0ce330 con 0x7f95f010a950 2024-05-10T04:03:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.530+0000 7f95edffb700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934] conn(0x7f95e00abd80 0x7f95e00ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.530+0000 7f95edffb700 1 -- 172.21.15.150:0/1998025707 --> [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95e00ae8e0 con 0x7f95e00abd80 2024-05-10T04:03:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.530+0000 7f95edffb700 1 -- 172.21.15.150:0/1998025707 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95dc0ce6a0 con 0x7f95f010a950 2024-05-10T04:03:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.530+0000 7f95f56b4700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934] conn(0x7f95e00abd80 0x7f95e00ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:15.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.530+0000 7f95f56b4700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934] conn(0x7f95e00abd80 0x7f95e00ae210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:15.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.531+0000 7f95edffb700 1 -- 172.21.15.150:0/1998025707 <== osd.9 v2:172.21.15.150:6826/383677934 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f95e00ae8e0 con 0x7f95e00abd80 2024-05-10T04:03:15.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.532+0000 7f95f4eb3700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f95e00a11b0 0x7f95e00a36c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f95f0109210 tx=0x7f95e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:15.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.555+0000 7f95f7117700 1 -- 172.21.15.150:0/1998025707 --> [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f95f010d340 con 0x7f95e00abd80 2024-05-10T04:03:15.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.555+0000 7f95edffb700 1 -- 172.21.15.150:0/1998025707 <== osd.9 v2:172.21.15.150:6826/383677934 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f95f010d340 con 0x7f95e00abd80 2024-05-10T04:03:15.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.556+0000 7f95f7117700 1 -- 172.21.15.150:0/1998025707 >> [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934] conn(0x7f95e00abd80 msgr2=0x7f95e00ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:15.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.556+0000 7f95f7117700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934] conn(0x7f95e00abd80 0x7f95e00ae210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:15.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.556+0000 7f95f7117700 1 -- 172.21.15.150:0/1998025707 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f95e00a11b0 msgr2=0x7f95e00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:15.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.556+0000 7f95f7117700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f95e00a11b0 0x7f95e00a36c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f95f0109210 tx=0x7f95e4009260 comp rx=0 tx=0).stop 2024-05-10T04:03:15.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.556+0000 7f95f7117700 1 -- 172.21.15.150:0/1998025707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f010a950 msgr2=0x7f95f01a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:15.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.556+0000 7f95f7117700 1 --2- 172.21.15.150:0/1998025707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f010a950 0x7f95f01a4010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f95dc000c00 tx=0x7f95dc005740 comp rx=0 tx=0).stop 2024-05-10T04:03:15.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.556+0000 7f95f56b4700 1 -- 172.21.15.150:0/1998025707 reap_dead start 2024-05-10T04:03:15.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.556+0000 7f95f7117700 1 -- 172.21.15.150:0/1998025707 shutdown_connections 2024-05-10T04:03:15.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.556+0000 7f95f7117700 1 -- 172.21.15.150:0/1998025707 >> 172.21.15.150:0/1998025707 conn(0x7f95f006ce00 msgr2=0x7f95f00715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:15.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.556+0000 7f95f7117700 1 -- 172.21.15.150:0/1998025707 shutdown_connections 2024-05-10T04:03:15.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.556+0000 7f95f7117700 1 -- 172.21.15.150:0/1998025707 wait complete. 2024-05-10T04:03:15.804 INFO:teuthology.orchestra.run.smithi150.stdout:158913789991 2024-05-10T04:03:15.804 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-10T04:03:16.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.806+0000 7f457a73d700 1 -- 172.21.15.150:0/1502723364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4574102e00 msgr2=0x7f4574103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:16.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.806+0000 7f457a73d700 1 --2- 172.21.15.150:0/1502723364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4574102e00 0x7f4574103210 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4564009990 tx=0x7f4564009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:16.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.807+0000 7f457a73d700 1 -- 172.21.15.150:0/1502723364 shutdown_connections 2024-05-10T04:03:16.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.807+0000 7f457a73d700 1 --2- 172.21.15.150:0/1502723364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f457410c230 0x7f457410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:16.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.807+0000 7f457a73d700 1 --2- 172.21.15.150:0/1502723364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4574102e00 0x7f4574103210 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:16.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.807+0000 7f457a73d700 1 --2- 172.21.15.150:0/1502723364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4574103750 0x7f457410bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.807+0000 7f457a73d700 1 -- 172.21.15.150:0/1502723364 >> 172.21.15.150:0/1502723364 conn(0x7f45740fc7a0 msgr2=0x7f45740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.807+0000 7f457a73d700 1 -- 172.21.15.150:0/1502723364 shutdown_connections 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.807+0000 7f457a73d700 1 -- 172.21.15.150:0/1502723364 wait complete. 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.807+0000 7f457a73d700 1 Processor -- start 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f457a73d700 1 -- start start 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f457a73d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4574102e00 0x7f4574198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f457a73d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4574103750 0x7f4574198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f457a73d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f457410c230 0x7f457419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f457a73d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457419d440 con 0x7f4574103750 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f457a73d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457419d5b0 con 0x7f4574102e00 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f457a73d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457419d770 con 0x7f457410c230 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f45737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4574103750 0x7f4574198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f45737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4574103750 0x7f4574198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54598/0 (socket says 172.21.15.150:54598) 2024-05-10T04:03:16.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f45737fe700 1 -- 172.21.15.150:0/2392325890 learned_addr learned my addr 172.21.15.150:0/2392325890 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f4578cda700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f457410c230 0x7f457419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f4573fff700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4574102e00 0x7f4574198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f45737fe700 1 -- 172.21.15.150:0/2392325890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f457410c230 msgr2=0x7f457419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f45737fe700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f457410c230 0x7f457419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f45737fe700 1 -- 172.21.15.150:0/2392325890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4574102e00 msgr2=0x7f4574198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.808+0000 7f45737fe700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4574102e00 0x7f4574198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.809+0000 7f45737fe700 1 -- 172.21.15.150:0/2392325890 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45640095f0 con 0x7f4574103750 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.809+0000 7f45737fe700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4574103750 0x7f4574198a30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f456800d980 tx=0x7f456800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.809+0000 7f45717fa700 1 -- 172.21.15.150:0/2392325890 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568004220 con 0x7f4574103750 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.809+0000 7f457a73d700 1 -- 172.21.15.150:0/2392325890 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457419da50 con 0x7f4574103750 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.809+0000 7f45717fa700 1 -- 172.21.15.150:0/2392325890 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4568007640 con 0x7f4574103750 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.809+0000 7f45717fa700 1 -- 172.21.15.150:0/2392325890 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568005690 con 0x7f4574103750 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.809+0000 7f457a73d700 1 -- 172.21.15.150:0/2392325890 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45741a5350 con 0x7f4574103750 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.810+0000 7f457a73d700 1 -- 172.21.15.150:0/2392325890 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4574111de0 con 0x7f4574103750 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.812+0000 7f45717fa700 1 -- 172.21.15.150:0/2392325890 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4568004750 con 0x7f4574103750 2024-05-10T04:03:16.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.812+0000 7f45717fa700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f455c0a11b0 0x7f455c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.812+0000 7f45717fa700 1 -- 172.21.15.150:0/2392325890 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f45680cd080 con 0x7f4574103750 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.812+0000 7f4573fff700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f455c0a11b0 0x7f455c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.813+0000 7f45717fa700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504] conn(0x7f455c0abd80 0x7f455c0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.813+0000 7f45717fa700 1 -- 172.21.15.150:0/2392325890 --> [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f455c0ae8e0 con 0x7f455c0abd80 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.813+0000 7f45717fa700 1 -- 172.21.15.150:0/2392325890 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45680cd490 con 0x7f4574103750 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.813+0000 7f4578cda700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504] conn(0x7f455c0abd80 0x7f455c0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.813+0000 7f4573fff700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f455c0a11b0 0x7f455c0a36c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4564006010 tx=0x7f456401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.813+0000 7f4578cda700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504] conn(0x7f455c0abd80 0x7f455c0ae210 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.814+0000 7f45717fa700 1 -- 172.21.15.150:0/2392325890 <== osd.6 v2:172.21.15.150:6818/3579837504 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f455c0ae8e0 con 0x7f455c0abd80 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.830+0000 7f457a73d700 1 -- 172.21.15.150:0/2392325890 --> [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4574061890 con 0x7f455c0abd80 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.831+0000 7f45717fa700 1 -- 172.21.15.150:0/2392325890 <== osd.6 v2:172.21.15.150:6818/3579837504 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4574061890 con 0x7f455c0abd80 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.831+0000 7f457a73d700 1 -- 172.21.15.150:0/2392325890 >> [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504] conn(0x7f455c0abd80 msgr2=0x7f455c0ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.831+0000 7f457a73d700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504] conn(0x7f455c0abd80 0x7f455c0ae210 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:16.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.831+0000 7f457a73d700 1 -- 172.21.15.150:0/2392325890 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f455c0a11b0 msgr2=0x7f455c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:16.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.831+0000 7f457a73d700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f455c0a11b0 0x7f455c0a36c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4564006010 tx=0x7f456401a040 comp rx=0 tx=0).stop 2024-05-10T04:03:16.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.831+0000 7f457a73d700 1 -- 172.21.15.150:0/2392325890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4574103750 msgr2=0x7f4574198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:16.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.831+0000 7f457a73d700 1 --2- 172.21.15.150:0/2392325890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4574103750 0x7f4574198a30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f456800d980 tx=0x7f456800dc90 comp rx=0 tx=0).stop 2024-05-10T04:03:16.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.831+0000 7f4578cda700 1 -- 172.21.15.150:0/2392325890 reap_dead start 2024-05-10T04:03:16.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.831+0000 7f457a73d700 1 -- 172.21.15.150:0/2392325890 shutdown_connections 2024-05-10T04:03:16.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.831+0000 7f457a73d700 1 -- 172.21.15.150:0/2392325890 >> 172.21.15.150:0/2392325890 conn(0x7f45740fc7a0 msgr2=0x7f45740fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:16.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.832+0000 7f457a73d700 1 -- 172.21.15.150:0/2392325890 shutdown_connections 2024-05-10T04:03:16.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:15.832+0000 7f457a73d700 1 -- 172.21.15.150:0/2392325890 wait complete. 2024-05-10T04:03:16.396 INFO:teuthology.orchestra.run.smithi150.stdout:274877906970 2024-05-10T04:03:16.397 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-10T04:03:16.424 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:16 smithi150 ceph-mon[42477]: pgmap v248: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-10T04:03:16.491 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:16 smithi181 ceph-mon[46602]: pgmap v248: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-10T04:03:16.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:16 smithi161 ceph-mon[46918]: pgmap v248: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-10T04:03:16.737 INFO:teuthology.orchestra.run.smithi150.stdout:188978561060 2024-05-10T04:03:16.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-10T04:03:17.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.107+0000 7fed4d2db700 1 -- 172.21.15.150:0/1219145577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed481057b0 msgr2=0x7fed48105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.107+0000 7fed4d2db700 1 --2- 172.21.15.150:0/1219145577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed481057b0 0x7fed48105bc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009990 tx=0x7fed3c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.107+0000 7fed4d2db700 1 -- 172.21.15.150:0/1219145577 shutdown_connections 2024-05-10T04:03:17.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.107+0000 7fed4d2db700 1 --2- 172.21.15.150:0/1219145577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed48106100 0x7fed4810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.107+0000 7fed4d2db700 1 --2- 172.21.15.150:0/1219145577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed481015b0 0x7fed481019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.107+0000 7fed4d2db700 1 --2- 172.21.15.150:0/1219145577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed481057b0 0x7fed48105bc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.107+0000 7fed4d2db700 1 -- 172.21.15.150:0/1219145577 >> 172.21.15.150:0/1219145577 conn(0x7fed480fbed0 msgr2=0x7fed480fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:17.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.107+0000 7fed4d2db700 1 -- 172.21.15.150:0/1219145577 shutdown_connections 2024-05-10T04:03:17.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.108+0000 7fed4d2db700 1 -- 172.21.15.150:0/1219145577 wait complete. 2024-05-10T04:03:17.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.108+0000 7fed4d2db700 1 Processor -- start 2024-05-10T04:03:17.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed4d2db700 1 -- start start 2024-05-10T04:03:17.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed4d2db700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed481015b0 0x7fed48197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed4d2db700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed481057b0 0x7fed481979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed4d2db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed48106100 0x7fed4819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed4d2db700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4819c420 con 0x7fed481057b0 2024-05-10T04:03:17.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed4d2db700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4819c590 con 0x7fed481015b0 2024-05-10T04:03:17.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed4d2db700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4819c750 con 0x7fed48106100 2024-05-10T04:03:17.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed467fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed481057b0 0x7fed481979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed467fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed481057b0 0x7fed481979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54616/0 (socket says 172.21.15.150:54616) 2024-05-10T04:03:17.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed467fc700 1 -- 172.21.15.150:0/4199793980 learned_addr learned my addr 172.21.15.150:0/4199793980 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:17.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed46ffd700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed481015b0 0x7fed48197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed477fe700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed48106100 0x7fed4819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed467fc700 1 -- 172.21.15.150:0/4199793980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed48106100 msgr2=0x7fed4819beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed467fc700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed48106100 0x7fed4819beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed467fc700 1 -- 172.21.15.150:0/4199793980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed481015b0 msgr2=0x7fed48197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed467fc700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed481015b0 0x7fed48197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.109+0000 7fed467fc700 1 -- 172.21.15.150:0/4199793980 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c0095f0 con 0x7fed481057b0 2024-05-10T04:03:17.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.110+0000 7fed467fc700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed481057b0 0x7fed481979f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fed3c000c00 tx=0x7fed3c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:17.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.110+0000 7fed2ffff700 1 -- 172.21.15.150:0/4199793980 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c01f070 con 0x7fed481057b0 2024-05-10T04:03:17.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.110+0000 7fed4d2db700 1 -- 172.21.15.150:0/4199793980 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4819c9d0 con 0x7fed481057b0 2024-05-10T04:03:17.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.110+0000 7fed2ffff700 1 -- 172.21.15.150:0/4199793980 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed3c00f460 con 0x7fed481057b0 2024-05-10T04:03:17.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.110+0000 7fed2ffff700 1 -- 172.21.15.150:0/4199793980 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c025680 con 0x7fed481057b0 2024-05-10T04:03:17.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.110+0000 7fed4d2db700 1 -- 172.21.15.150:0/4199793980 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed48100210 con 0x7fed481057b0 2024-05-10T04:03:17.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.111+0000 7fed4d2db700 1 -- 172.21.15.150:0/4199793980 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fed48110dc0 con 0x7fed481057b0 2024-05-10T04:03:17.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.112+0000 7fed2ffff700 1 -- 172.21.15.150:0/4199793980 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fed3c023070 con 0x7fed481057b0 2024-05-10T04:03:17.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.113+0000 7fed2ffff700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fed300a55c0 0x7fed300a7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.113+0000 7fed2ffff700 1 -- 172.21.15.150:0/4199793980 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fed3c0cded0 con 0x7fed481057b0 2024-05-10T04:03:17.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.113+0000 7fed2ffff700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735] conn(0x7fed300b0190 0x7fed300b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.113+0000 7fed2ffff700 1 -- 172.21.15.150:0/4199793980 --> [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed300b2cf0 con 0x7fed300b0190 2024-05-10T04:03:17.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.113+0000 7fed2ffff700 1 -- 172.21.15.150:0/4199793980 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed3c0d04b0 con 0x7fed481057b0 2024-05-10T04:03:17.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.113+0000 7fed46ffd700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fed300a55c0 0x7fed300a7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.114+0000 7fed477fe700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735] conn(0x7fed300b0190 0x7fed300b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.114+0000 7fed46ffd700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fed300a55c0 0x7fed300a7ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fed38005fd0 tx=0x7fed38005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:17.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.114+0000 7fed477fe700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735] conn(0x7fed300b0190 0x7fed300b2620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:17.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.115+0000 7fed2ffff700 1 -- 172.21.15.150:0/4199793980 <== osd.1 v2:172.21.15.161:6800/1606950735 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fed300b2cf0 con 0x7fed300b0190 2024-05-10T04:03:17.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.131+0000 7fed4d2db700 1 -- 172.21.15.150:0/4199793980 --> [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fed48100670 con 0x7fed300b0190 2024-05-10T04:03:17.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.132+0000 7fed2ffff700 1 -- 172.21.15.150:0/4199793980 <== osd.1 v2:172.21.15.161:6800/1606950735 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fed48100670 con 0x7fed300b0190 2024-05-10T04:03:17.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.132+0000 7fed4d2db700 1 -- 172.21.15.150:0/4199793980 >> [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735] conn(0x7fed300b0190 msgr2=0x7fed300b2620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.132+0000 7fed4d2db700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735] conn(0x7fed300b0190 0x7fed300b2620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.132+0000 7fed4d2db700 1 -- 172.21.15.150:0/4199793980 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fed300a55c0 msgr2=0x7fed300a7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.132+0000 7fed4d2db700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fed300a55c0 0x7fed300a7ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fed38005fd0 tx=0x7fed38005f60 comp rx=0 tx=0).stop 2024-05-10T04:03:17.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.132+0000 7fed4d2db700 1 -- 172.21.15.150:0/4199793980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed481057b0 msgr2=0x7fed481979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.132+0000 7fed4d2db700 1 --2- 172.21.15.150:0/4199793980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed481057b0 0x7fed481979f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fed3c000c00 tx=0x7fed3c005ef0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.132+0000 7fed477fe700 1 -- 172.21.15.150:0/4199793980 reap_dead start 2024-05-10T04:03:17.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.132+0000 7fed4d2db700 1 -- 172.21.15.150:0/4199793980 shutdown_connections 2024-05-10T04:03:17.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.132+0000 7fed4d2db700 1 -- 172.21.15.150:0/4199793980 >> 172.21.15.150:0/4199793980 conn(0x7fed480fbed0 msgr2=0x7fed480fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:17.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.133+0000 7fed4d2db700 1 -- 172.21.15.150:0/4199793980 shutdown_connections 2024-05-10T04:03:17.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.133+0000 7fed4d2db700 1 -- 172.21.15.150:0/4199793980 wait complete. 2024-05-10T04:03:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.366+0000 7f26ab99c700 1 -- 172.21.15.150:0/1209480507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26a4100e30 msgr2=0x7f26a4101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.366+0000 7f26ab99c700 1 --2- 172.21.15.150:0/1209480507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26a4100e30 0x7f26a4101240 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009990 tx=0x7f26a0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.366+0000 7f26ab99c700 1 -- 172.21.15.150:0/1209480507 shutdown_connections 2024-05-10T04:03:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.366+0000 7f26ab99c700 1 --2- 172.21.15.150:0/1209480507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a41060b0 0x7f26a410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.366+0000 7f26ab99c700 1 --2- 172.21.15.150:0/1209480507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26a4100e30 0x7f26a4101240 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.366+0000 7f26ab99c700 1 --2- 172.21.15.150:0/1209480507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a4105760 0x7f26a4105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.366+0000 7f26ab99c700 1 -- 172.21.15.150:0/1209480507 >> 172.21.15.150:0/1209480507 conn(0x7f26a40fc760 msgr2=0x7f26a40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.366+0000 7f26ab99c700 1 -- 172.21.15.150:0/1209480507 shutdown_connections 2024-05-10T04:03:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.366+0000 7f26ab99c700 1 -- 172.21.15.150:0/1209480507 wait complete. 2024-05-10T04:03:17.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.367+0000 7f26ab99c700 1 Processor -- start 2024-05-10T04:03:17.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.367+0000 7f26ab99c700 1 -- start start 2024-05-10T04:03:17.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.367+0000 7f26ab99c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a4100e30 0x7f26a4199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.367+0000 7f26ab99c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26a4105760 0x7f26a41998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.367+0000 7f26ab99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a41060b0 0x7f26a41a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.367+0000 7f26ab99c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a419a1a0 con 0x7f26a4105760 2024-05-10T04:03:17.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.367+0000 7f26ab99c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a41a4920 con 0x7f26a41060b0 2024-05-10T04:03:17.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.367+0000 7f26ab99c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a41a4a90 con 0x7f26a4100e30 2024-05-10T04:03:17.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.367+0000 7f26a8f37700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26a4105760 0x7f26a41998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.367+0000 7f26a8f37700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26a4105760 0x7f26a41998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54634/0 (socket says 172.21.15.150:54634) 2024-05-10T04:03:17.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.367+0000 7f26a8f37700 1 -- 172.21.15.150:0/1913643909 learned_addr learned my addr 172.21.15.150:0/1913643909 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:17.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f26a9738700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a4100e30 0x7f26a4199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f26a9f39700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a41060b0 0x7f26a41a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f26a8f37700 1 -- 172.21.15.150:0/1913643909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a4100e30 msgr2=0x7f26a4199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f26a8f37700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a4100e30 0x7f26a4199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f26a8f37700 1 -- 172.21.15.150:0/1913643909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a41060b0 msgr2=0x7f26a41a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f26a8f37700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a41060b0 0x7f26a41a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f26a8f37700 1 -- 172.21.15.150:0/1913643909 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a00095f0 con 0x7f26a4105760 2024-05-10T04:03:17.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f26a9f39700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a41060b0 0x7f26a41a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:17.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f26a8f37700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26a4105760 0x7f26a41998c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f269400d950 tx=0x7f269400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:17.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f269a7fc700 1 -- 172.21.15.150:0/1913643909 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694004220 con 0x7f26a4105760 2024-05-10T04:03:17.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f26ab99c700 1 -- 172.21.15.150:0/1913643909 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a41a4d70 con 0x7f26a4105760 2024-05-10T04:03:17.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f269a7fc700 1 -- 172.21.15.150:0/1913643909 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2694007620 con 0x7f26a4105760 2024-05-10T04:03:17.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f269a7fc700 1 -- 172.21.15.150:0/1913643909 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694004de0 con 0x7f26a4105760 2024-05-10T04:03:17.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.368+0000 7f26ab99c700 1 -- 172.21.15.150:0/1913643909 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a41a5200 con 0x7f26a4105760 2024-05-10T04:03:17.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.369+0000 7f26ab99c700 1 -- 172.21.15.150:0/1913643909 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26a41022e0 con 0x7f26a4105760 2024-05-10T04:03:17.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.371+0000 7f269a7fc700 1 -- 172.21.15.150:0/1913643909 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f269400b900 con 0x7f26a4105760 2024-05-10T04:03:17.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.372+0000 7f269a7fc700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f26900a11b0 0x7f26900a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.372+0000 7f269a7fc700 1 -- 172.21.15.150:0/1913643909 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f26940cd240 con 0x7f26a4105760 2024-05-10T04:03:17.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.372+0000 7f269a7fc700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004] conn(0x7f26900abd80 0x7f26900ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.372+0000 7f269a7fc700 1 -- 172.21.15.150:0/1913643909 --> [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26900ae8e0 con 0x7f26900abd80 2024-05-10T04:03:17.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.372+0000 7f269a7fc700 1 -- 172.21.15.150:0/1913643909 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26940cd5b0 con 0x7f26a4105760 2024-05-10T04:03:17.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.372+0000 7f26a9f39700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004] conn(0x7f26900abd80 0x7f26900ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.372+0000 7f26a9738700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f26900a11b0 0x7f26900a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.373+0000 7f26a9f39700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004] conn(0x7f26900abd80 0x7f26900ae210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:17.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.373+0000 7f26a9738700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f26900a11b0 0x7f26900a36c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f26a0006010 tx=0x7f26a000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:17.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.374+0000 7f269a7fc700 1 -- 172.21.15.150:0/1913643909 <== osd.3 v2:172.21.15.150:6810/1770832004 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f26900ae8e0 con 0x7f26900abd80 2024-05-10T04:03:17.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.390+0000 7f26ab99c700 1 -- 172.21.15.150:0/1913643909 --> [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f26a4061890 con 0x7f26900abd80 2024-05-10T04:03:17.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.390+0000 7f269a7fc700 1 -- 172.21.15.150:0/1913643909 <== osd.3 v2:172.21.15.150:6810/1770832004 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f26a4061890 con 0x7f26900abd80 2024-05-10T04:03:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.391+0000 7f26ab99c700 1 -- 172.21.15.150:0/1913643909 >> [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004] conn(0x7f26900abd80 msgr2=0x7f26900ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.391+0000 7f26ab99c700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004] conn(0x7f26900abd80 0x7f26900ae210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.391+0000 7f26ab99c700 1 -- 172.21.15.150:0/1913643909 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f26900a11b0 msgr2=0x7f26900a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.391+0000 7f26ab99c700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f26900a11b0 0x7f26900a36c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f26a0006010 tx=0x7f26a000b540 comp rx=0 tx=0).stop 2024-05-10T04:03:17.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.391+0000 7f26ab99c700 1 -- 172.21.15.150:0/1913643909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26a4105760 msgr2=0x7f26a41998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.391+0000 7f26ab99c700 1 --2- 172.21.15.150:0/1913643909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26a4105760 0x7f26a41998c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f269400d950 tx=0x7f269400dd10 comp rx=0 tx=0).stop 2024-05-10T04:03:17.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.391+0000 7f26a9f39700 1 -- 172.21.15.150:0/1913643909 reap_dead start 2024-05-10T04:03:17.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.391+0000 7f26ab99c700 1 -- 172.21.15.150:0/1913643909 shutdown_connections 2024-05-10T04:03:17.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.391+0000 7f26ab99c700 1 -- 172.21.15.150:0/1913643909 >> 172.21.15.150:0/1913643909 conn(0x7f26a40fc760 msgr2=0x7f26a40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:17.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.391+0000 7f26ab99c700 1 -- 172.21.15.150:0/1913643909 shutdown_connections 2024-05-10T04:03:17.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.391+0000 7f26ab99c700 1 -- 172.21.15.150:0/1913643909 wait complete. 2024-05-10T04:03:17.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.465+0000 7f6926ec5700 1 -- 172.21.15.150:0/3869951926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6920100e00 msgr2=0x7f6920101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.465+0000 7f6926ec5700 1 --2- 172.21.15.150:0/3869951926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6920100e00 0x7f6920101210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6914009990 tx=0x7f6914009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.466+0000 7f6926ec5700 1 -- 172.21.15.150:0/3869951926 shutdown_connections 2024-05-10T04:03:17.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.466+0000 7f6926ec5700 1 --2- 172.21.15.150:0/3869951926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6920106080 0x7f692010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.466+0000 7f6926ec5700 1 --2- 172.21.15.150:0/3869951926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6920100e00 0x7f6920101210 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.466+0000 7f6926ec5700 1 --2- 172.21.15.150:0/3869951926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6920105730 0x7f6920105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.466+0000 7f6926ec5700 1 -- 172.21.15.150:0/3869951926 >> 172.21.15.150:0/3869951926 conn(0x7f69200fc7a0 msgr2=0x7f69200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:17.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.466+0000 7f6926ec5700 1 -- 172.21.15.150:0/3869951926 shutdown_connections 2024-05-10T04:03:17.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.466+0000 7f6926ec5700 1 -- 172.21.15.150:0/3869951926 wait complete. 2024-05-10T04:03:17.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.466+0000 7f6926ec5700 1 Processor -- start 2024-05-10T04:03:17.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.466+0000 7f6926ec5700 1 -- start start 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f6926ec5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6920100e00 0x7f6920199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f6926ec5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6920105730 0x7f69201999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f6926ec5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6920106080 0x7f69201a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f6926ec5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692019a140 con 0x7f6920105730 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f6926ec5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69201a4870 con 0x7f6920100e00 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f6926ec5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69201a4a30 con 0x7f6920106080 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f691ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6920105730 0x7f69201999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f691ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6920105730 0x7f69201999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54654/0 (socket says 172.21.15.150:54654) 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f691ffff700 1 -- 172.21.15.150:0/1316108636 learned_addr learned my addr 172.21.15.150:0/1316108636 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f6924c61700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6920100e00 0x7f6920199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f6925462700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6920106080 0x7f69201a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f691ffff700 1 -- 172.21.15.150:0/1316108636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6920106080 msgr2=0x7f69201a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f691ffff700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6920106080 0x7f69201a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f691ffff700 1 -- 172.21.15.150:0/1316108636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6920100e00 msgr2=0x7f6920199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f691ffff700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6920100e00 0x7f6920199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f691ffff700 1 -- 172.21.15.150:0/1316108636 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69140095f0 con 0x7f6920105730 2024-05-10T04:03:17.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.467+0000 7f6924c61700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6920100e00 0x7f6920199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:03:17.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.468+0000 7f691ffff700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6920105730 0x7f69201999c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f690c00e970 tx=0x7f690c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:17.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.468+0000 7f691dffb700 1 -- 172.21.15.150:0/1316108636 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c009790 con 0x7f6920105730 2024-05-10T04:03:17.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.468+0000 7f691dffb700 1 -- 172.21.15.150:0/1316108636 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f690c00c7e0 con 0x7f6920105730 2024-05-10T04:03:17.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.468+0000 7f691dffb700 1 -- 172.21.15.150:0/1316108636 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c0054e0 con 0x7f6920105730 2024-05-10T04:03:17.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.468+0000 7f6926ec5700 1 -- 172.21.15.150:0/1316108636 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69201a4d10 con 0x7f6920105730 2024-05-10T04:03:17.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.468+0000 7f6926ec5700 1 -- 172.21.15.150:0/1316108636 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69201a51c0 con 0x7f6920105730 2024-05-10T04:03:17.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.469+0000 7f6926ec5700 1 -- 172.21.15.150:0/1316108636 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6920110cf0 con 0x7f6920105730 2024-05-10T04:03:17.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.470+0000 7f691dffb700 1 -- 172.21.15.150:0/1316108636 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f690c00c950 con 0x7f6920105730 2024-05-10T04:03:17.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.471+0000 7f691dffb700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69100a11b0 0x7f69100a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.471+0000 7f691dffb700 1 -- 172.21.15.150:0/1316108636 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f690c014070 con 0x7f6920105730 2024-05-10T04:03:17.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.471+0000 7f691dffb700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649] conn(0x7f69100abd80 0x7f69100ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:17.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.471+0000 7f691dffb700 1 -- 172.21.15.150:0/1316108636 --> [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69100ae8e0 con 0x7f69100abd80 2024-05-10T04:03:17.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.471+0000 7f691dffb700 1 -- 172.21.15.150:0/1316108636 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f690c0cf370 con 0x7f6920105730 2024-05-10T04:03:17.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.472+0000 7f6924c61700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69100a11b0 0x7f69100a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.473+0000 7f6925462700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649] conn(0x7f69100abd80 0x7f69100ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:17.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.473+0000 7f6924c61700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69100a11b0 0x7f69100a36c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f691400b2e0 tx=0x7f691401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:17.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.473+0000 7f6925462700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649] conn(0x7f69100abd80 0x7f69100ae210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:17.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.474+0000 7f691dffb700 1 -- 172.21.15.150:0/1316108636 <== osd.7 v2:172.21.15.161:6816/1528703649 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f69100ae8e0 con 0x7f69100abd80 2024-05-10T04:03:17.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.491+0000 7f6926ec5700 1 -- 172.21.15.150:0/1316108636 --> [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6920061910 con 0x7f69100abd80 2024-05-10T04:03:17.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.492+0000 7f691dffb700 1 -- 172.21.15.150:0/1316108636 <== osd.7 v2:172.21.15.161:6816/1528703649 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6920061910 con 0x7f69100abd80 2024-05-10T04:03:17.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.492+0000 7f6926ec5700 1 -- 172.21.15.150:0/1316108636 >> [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649] conn(0x7f69100abd80 msgr2=0x7f69100ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.492+0000 7f6926ec5700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649] conn(0x7f69100abd80 0x7f69100ae210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:17.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.492+0000 7f6926ec5700 1 -- 172.21.15.150:0/1316108636 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69100a11b0 msgr2=0x7f69100a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.492+0000 7f6926ec5700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69100a11b0 0x7f69100a36c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f691400b2e0 tx=0x7f691401a040 comp rx=0 tx=0).stop 2024-05-10T04:03:17.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.492+0000 7f6926ec5700 1 -- 172.21.15.150:0/1316108636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6920105730 msgr2=0x7f69201999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:17.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.492+0000 7f6926ec5700 1 --2- 172.21.15.150:0/1316108636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6920105730 0x7f69201999c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f690c00e970 tx=0x7f690c00ec80 comp rx=0 tx=0).stop 2024-05-10T04:03:17.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.492+0000 7f6925462700 1 -- 172.21.15.150:0/1316108636 reap_dead start 2024-05-10T04:03:17.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.492+0000 7f6926ec5700 1 -- 172.21.15.150:0/1316108636 shutdown_connections 2024-05-10T04:03:17.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.492+0000 7f6926ec5700 1 -- 172.21.15.150:0/1316108636 >> 172.21.15.150:0/1316108636 conn(0x7f69200fc7a0 msgr2=0x7f6920104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:17.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.493+0000 7f6926ec5700 1 -- 172.21.15.150:0/1316108636 shutdown_connections 2024-05-10T04:03:17.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:17.493+0000 7f6926ec5700 1 -- 172.21.15.150:0/1316108636 wait complete. 2024-05-10T04:03:18.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.016+0000 7feb8ad46700 1 -- 172.21.15.150:0/3267894644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb84101c50 msgr2=0x7feb8410d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:18.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.016+0000 7feb8ad46700 1 --2- 172.21.15.150:0/3267894644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb84101c50 0x7feb8410d9d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00b0a0 tx=0x7feb7c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.016+0000 7feb8ad46700 1 -- 172.21.15.150:0/3267894644 shutdown_connections 2024-05-10T04:03:18.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.016+0000 7feb8ad46700 1 --2- 172.21.15.150:0/3267894644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb84101c50 0x7feb8410d9d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.016+0000 7feb8ad46700 1 --2- 172.21.15.150:0/3267894644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb84105b60 0x7feb84105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.016+0000 7feb8ad46700 1 --2- 172.21.15.150:0/3267894644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb84101300 0x7feb84101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.016+0000 7feb8ad46700 1 -- 172.21.15.150:0/3267894644 >> 172.21.15.150:0/3267894644 conn(0x7feb840fbed0 msgr2=0x7feb840fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:18.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.017+0000 7feb8ad46700 1 -- 172.21.15.150:0/3267894644 shutdown_connections 2024-05-10T04:03:18.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.017+0000 7feb8ad46700 1 -- 172.21.15.150:0/3267894644 wait complete. 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb8ad46700 1 Processor -- start 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb8ad46700 1 -- start start 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb8ad46700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb84101300 0x7feb84107a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb8ad46700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb84101c50 0x7feb84107fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb8ad46700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb84105b60 0x7feb841a6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb8ad46700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb84108820 con 0x7feb84101c50 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb8ad46700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb841a6c80 con 0x7feb84101300 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb8ad46700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb841a6dc0 con 0x7feb84105b60 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb83fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb84101c50 0x7feb84107fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb83fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb84101c50 0x7feb84107fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54664/0 (socket says 172.21.15.150:54664) 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb83fff700 1 -- 172.21.15.150:0/4011140315 learned_addr learned my addr 172.21.15.150:0/4011140315 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb88ae2700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb84101300 0x7feb84107a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb892e3700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb84105b60 0x7feb841a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:18.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb83fff700 1 -- 172.21.15.150:0/4011140315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb84105b60 msgr2=0x7feb841a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:18.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb83fff700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb84105b60 0x7feb841a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb83fff700 1 -- 172.21.15.150:0/4011140315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb84101300 msgr2=0x7feb84107a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:18.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb83fff700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb84101300 0x7feb84107a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb83fff700 1 -- 172.21.15.150:0/4011140315 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c009d00 con 0x7feb84101c50 2024-05-10T04:03:18.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.018+0000 7feb83fff700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb84101c50 0x7feb84107fb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feb7000c920 tx=0x7feb7000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:18.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.019+0000 7feb81ffb700 1 -- 172.21.15.150:0/4011140315 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70004a30 con 0x7feb84101c50 2024-05-10T04:03:18.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.019+0000 7feb81ffb700 1 -- 172.21.15.150:0/4011140315 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb700054e0 con 0x7feb84101c50 2024-05-10T04:03:18.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.019+0000 7feb8ad46700 1 -- 172.21.15.150:0/4011140315 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb841a6ff0 con 0x7feb84101c50 2024-05-10T04:03:18.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.019+0000 7feb81ffb700 1 -- 172.21.15.150:0/4011140315 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70007c10 con 0x7feb84101c50 2024-05-10T04:03:18.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.019+0000 7feb8ad46700 1 -- 172.21.15.150:0/4011140315 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb841a7460 con 0x7feb84101c50 2024-05-10T04:03:18.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.020+0000 7feb8ad46700 1 -- 172.21.15.150:0/4011140315 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feb84102310 con 0x7feb84101c50 2024-05-10T04:03:18.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.021+0000 7feb81ffb700 1 -- 172.21.15.150:0/4011140315 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7feb70005000 con 0x7feb84101c50 2024-05-10T04:03:18.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.021+0000 7feb81ffb700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb740a11b0 0x7feb740a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:18.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.022+0000 7feb88ae2700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb740a11b0 0x7feb740a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:18.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.022+0000 7feb81ffb700 1 -- 172.21.15.150:0/4011140315 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7feb700ccbd0 con 0x7feb84101c50 2024-05-10T04:03:18.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.022+0000 7feb81ffb700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503] conn(0x7feb740abd80 0x7feb740ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:18.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.022+0000 7feb81ffb700 1 -- 172.21.15.150:0/4011140315 --> [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb740ae8e0 con 0x7feb740abd80 2024-05-10T04:03:18.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.022+0000 7feb81ffb700 1 -- 172.21.15.150:0/4011140315 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb700d12a0 con 0x7feb84101c50 2024-05-10T04:03:18.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.022+0000 7feb88ae2700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb740a11b0 0x7feb740a36c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feb84068cf0 tx=0x7feb78007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:18.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.022+0000 7feb892e3700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503] conn(0x7feb740abd80 0x7feb740ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:18.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.023+0000 7feb892e3700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503] conn(0x7feb740abd80 0x7feb740ae210 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:18.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.023+0000 7feb81ffb700 1 -- 172.21.15.150:0/4011140315 <== osd.4 v2:172.21.15.161:6808/765138503 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7feb740ae8e0 con 0x7feb740abd80 2024-05-10T04:03:18.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.040+0000 7feb8ad46700 1 -- 172.21.15.150:0/4011140315 --> [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7feb840fec40 con 0x7feb740abd80 2024-05-10T04:03:18.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.040+0000 7feb81ffb700 1 -- 172.21.15.150:0/4011140315 <== osd.4 v2:172.21.15.161:6808/765138503 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7feb840fec40 con 0x7feb740abd80 2024-05-10T04:03:18.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.040+0000 7feb8ad46700 1 -- 172.21.15.150:0/4011140315 >> [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503] conn(0x7feb740abd80 msgr2=0x7feb740ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:18.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.041+0000 7feb8ad46700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503] conn(0x7feb740abd80 0x7feb740ae210 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.041+0000 7feb8ad46700 1 -- 172.21.15.150:0/4011140315 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb740a11b0 msgr2=0x7feb740a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:18.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.041+0000 7feb8ad46700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb740a11b0 0x7feb740a36c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feb84068cf0 tx=0x7feb78007400 comp rx=0 tx=0).stop 2024-05-10T04:03:18.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.041+0000 7feb8ad46700 1 -- 172.21.15.150:0/4011140315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb84101c50 msgr2=0x7feb84107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:18.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.041+0000 7feb8ad46700 1 --2- 172.21.15.150:0/4011140315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb84101c50 0x7feb84107fb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feb7000c920 tx=0x7feb7000cc30 comp rx=0 tx=0).stop 2024-05-10T04:03:18.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.041+0000 7feb892e3700 1 -- 172.21.15.150:0/4011140315 reap_dead start 2024-05-10T04:03:18.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.041+0000 7feb8ad46700 1 -- 172.21.15.150:0/4011140315 shutdown_connections 2024-05-10T04:03:18.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.041+0000 7feb8ad46700 1 -- 172.21.15.150:0/4011140315 >> 172.21.15.150:0/4011140315 conn(0x7feb840fbed0 msgr2=0x7feb840fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:18.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.041+0000 7feb8ad46700 1 -- 172.21.15.150:0/4011140315 shutdown_connections 2024-05-10T04:03:18.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.041+0000 7feb8ad46700 1 -- 172.21.15.150:0/4011140315 wait complete. 2024-05-10T04:03:18.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.313+0000 7fa0a8af0700 1 -- 172.21.15.150:0/1376049706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0a410c230 msgr2=0x7fa0a410e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:18.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.313+0000 7fa0a8af0700 1 --2- 172.21.15.150:0/1376049706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0a410c230 0x7fa0a410e710 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00b0a0 tx=0x7fa09c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.313+0000 7fa0a8af0700 1 -- 172.21.15.150:0/1376049706 shutdown_connections 2024-05-10T04:03:18.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.313+0000 7fa0a8af0700 1 --2- 172.21.15.150:0/1376049706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0a410c230 0x7fa0a410e710 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.313+0000 7fa0a8af0700 1 --2- 172.21.15.150:0/1376049706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0a4102e00 0x7fa0a4103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.313+0000 7fa0a8af0700 1 --2- 172.21.15.150:0/1376049706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a4103750 0x7fa0a410bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.313+0000 7fa0a8af0700 1 -- 172.21.15.150:0/1376049706 >> 172.21.15.150:0/1376049706 conn(0x7fa0a40fc7a0 msgr2=0x7fa0a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:18.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.313+0000 7fa0a8af0700 1 -- 172.21.15.150:0/1376049706 shutdown_connections 2024-05-10T04:03:18.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.313+0000 7fa0a8af0700 1 -- 172.21.15.150:0/1376049706 wait complete. 2024-05-10T04:03:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.314+0000 7fa0a8af0700 1 Processor -- start 2024-05-10T04:03:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.314+0000 7fa0a8af0700 1 -- start start 2024-05-10T04:03:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.314+0000 7fa0a8af0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a4102e00 0x7fa0a4100af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.314+0000 7fa0a8af0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0a4103750 0x7fa0a4101030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.314+0000 7fa0a8af0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0a410c230 0x7fa0a41a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.314+0000 7fa0a8af0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a41018a0 con 0x7fa0a4103750 2024-05-10T04:03:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.314+0000 7fa0a8af0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a41a6c70 con 0x7fa0a4102e00 2024-05-10T04:03:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.314+0000 7fa0a8af0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a41a6db0 con 0x7fa0a410c230 2024-05-10T04:03:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.314+0000 7fa0a1d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0a4103750 0x7fa0a4101030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.314+0000 7fa0a1d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0a4103750 0x7fa0a4101030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54670/0 (socket says 172.21.15.150:54670) 2024-05-10T04:03:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0a1d9b700 1 -- 172.21.15.150:0/363512070 learned_addr learned my addr 172.21.15.150:0/363512070 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0a259c700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a4102e00 0x7fa0a4100af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0a2d9d700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0a410c230 0x7fa0a41a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:18.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0a1d9b700 1 -- 172.21.15.150:0/363512070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0a410c230 msgr2=0x7fa0a41a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:18.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0a1d9b700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0a410c230 0x7fa0a41a6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0a1d9b700 1 -- 172.21.15.150:0/363512070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a4102e00 msgr2=0x7fa0a4100af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:18.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0a1d9b700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a4102e00 0x7fa0a4100af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0a1d9b700 1 -- 172.21.15.150:0/363512070 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c009d00 con 0x7fa0a4103750 2024-05-10T04:03:18.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0a1d9b700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0a4103750 0x7fa0a4101030 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa09400d950 tx=0x7fa09400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:18.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0937fe700 1 -- 172.21.15.150:0/363512070 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094009940 con 0x7fa0a4103750 2024-05-10T04:03:18.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0937fe700 1 -- 172.21.15.150:0/363512070 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa094010460 con 0x7fa0a4103750 2024-05-10T04:03:18.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0937fe700 1 -- 172.21.15.150:0/363512070 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09400f620 con 0x7fa0a4103750 2024-05-10T04:03:18.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0a8af0700 1 -- 172.21.15.150:0/363512070 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a41a6fe0 con 0x7fa0a4103750 2024-05-10T04:03:18.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.315+0000 7fa0a8af0700 1 -- 172.21.15.150:0/363512070 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a41a7400 con 0x7fa0a4103750 2024-05-10T04:03:18.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.316+0000 7fa0a8af0700 1 -- 172.21.15.150:0/363512070 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0a40695a0 con 0x7fa0a4103750 2024-05-10T04:03:18.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.318+0000 7fa0937fe700 1 -- 172.21.15.150:0/363512070 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa0940108d0 con 0x7fa0a4103750 2024-05-10T04:03:18.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.318+0000 7fa0937fe700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa08c0a11b0 0x7fa08c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:18.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.318+0000 7fa0a259c700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa08c0a11b0 0x7fa08c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:18.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.318+0000 7fa0937fe700 1 -- 172.21.15.150:0/363512070 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa0940ccef0 con 0x7fa0a4103750 2024-05-10T04:03:18.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.318+0000 7fa0937fe700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145] conn(0x7fa08c0abd80 0x7fa08c0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:18.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.318+0000 7fa0937fe700 1 -- 172.21.15.150:0/363512070 --> [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa08c0ae8e0 con 0x7fa08c0abd80 2024-05-10T04:03:18.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.318+0000 7fa0937fe700 1 -- 172.21.15.150:0/363512070 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0940cd300 con 0x7fa0a4103750 2024-05-10T04:03:18.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.319+0000 7fa0a259c700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa08c0a11b0 0x7fa08c0a36c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0a406ab80 tx=0x7fa09800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:18.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.319+0000 7fa0a2d9d700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145] conn(0x7fa08c0abd80 0x7fa08c0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:18.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.319+0000 7fa0a2d9d700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145] conn(0x7fa08c0abd80 0x7fa08c0ae210 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:18.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.320+0000 7fa0937fe700 1 -- 172.21.15.150:0/363512070 <== osd.2 v2:172.21.15.181:6800/4035594145 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa08c0ae8e0 con 0x7fa08c0abd80 2024-05-10T04:03:18.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.336+0000 7fa0a8af0700 1 -- 172.21.15.150:0/363512070 --> [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa0a40618d0 con 0x7fa08c0abd80 2024-05-10T04:03:18.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.337+0000 7fa0937fe700 1 -- 172.21.15.150:0/363512070 <== osd.2 v2:172.21.15.181:6800/4035594145 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa0a40618d0 con 0x7fa08c0abd80 2024-05-10T04:03:18.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.337+0000 7fa0a8af0700 1 -- 172.21.15.150:0/363512070 >> [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145] conn(0x7fa08c0abd80 msgr2=0x7fa08c0ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:18.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.337+0000 7fa0a8af0700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145] conn(0x7fa08c0abd80 0x7fa08c0ae210 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:18.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.337+0000 7fa0a8af0700 1 -- 172.21.15.150:0/363512070 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa08c0a11b0 msgr2=0x7fa08c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:18.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.337+0000 7fa0a8af0700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa08c0a11b0 0x7fa08c0a36c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0a406ab80 tx=0x7fa09800b410 comp rx=0 tx=0).stop 2024-05-10T04:03:18.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.337+0000 7fa0a8af0700 1 -- 172.21.15.150:0/363512070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0a4103750 msgr2=0x7fa0a4101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:18.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.337+0000 7fa0a8af0700 1 --2- 172.21.15.150:0/363512070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0a4103750 0x7fa0a4101030 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa09400d950 tx=0x7fa09400dc60 comp rx=0 tx=0).stop 2024-05-10T04:03:18.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.337+0000 7fa0a2d9d700 1 -- 172.21.15.150:0/363512070 reap_dead start 2024-05-10T04:03:18.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.338+0000 7fa0a8af0700 1 -- 172.21.15.150:0/363512070 shutdown_connections 2024-05-10T04:03:18.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.338+0000 7fa0a8af0700 1 -- 172.21.15.150:0/363512070 >> 172.21.15.150:0/363512070 conn(0x7fa0a40fc7a0 msgr2=0x7fa0a40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:18.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.338+0000 7fa0a8af0700 1 -- 172.21.15.150:0/363512070 shutdown_connections 2024-05-10T04:03:18.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:18.338+0000 7fa0a8af0700 1 -- 172.21.15.150:0/363512070 wait complete. 2024-05-10T04:03:18.373 INFO:teuthology.orchestra.run.smithi150.stdout:107374182448 2024-05-10T04:03:18.374 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-10T04:03:18.403 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:18 smithi150 ceph-mon[42477]: pgmap v249: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-10T04:03:18.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:18 smithi181 ceph-mon[46602]: pgmap v249: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-10T04:03:18.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:18 smithi161 ceph-mon[46918]: pgmap v249: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-10T04:03:18.688 INFO:teuthology.orchestra.run.smithi150.stdout:210453397538 2024-05-10T04:03:18.688 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-10T04:03:18.817 INFO:teuthology.orchestra.run.smithi150.stdout:137438953516 2024-05-10T04:03:18.818 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-10T04:03:18.980 INFO:teuthology.orchestra.run.smithi150.stdout:77309411379 2024-05-10T04:03:18.980 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-10T04:03:19.132 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:19.940 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:19.941 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:20.211 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:20.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:20 smithi150 ceph-mon[42477]: pgmap v250: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-10T04:03:20.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.295+0000 7f07d3526700 1 -- 172.21.15.150:0/2603310417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07cc105730 msgr2=0x7f07cc105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:20.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.295+0000 7f07d3526700 1 --2- 172.21.15.150:0/2603310417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07cc105730 0x7f07cc105b40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009990 tx=0x7f07c8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:20.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.295+0000 7f07d3526700 1 -- 172.21.15.150:0/2603310417 shutdown_connections 2024-05-10T04:03:20.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.295+0000 7f07d3526700 1 --2- 172.21.15.150:0/2603310417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07cc106080 0x7f07cc10d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:20.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.295+0000 7f07d3526700 1 --2- 172.21.15.150:0/2603310417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07cc100e00 0x7f07cc101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:20.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.295+0000 7f07d3526700 1 --2- 172.21.15.150:0/2603310417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07cc105730 0x7f07cc105b40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:20.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.295+0000 7f07d3526700 1 -- 172.21.15.150:0/2603310417 >> 172.21.15.150:0/2603310417 conn(0x7f07cc0fc7a0 msgr2=0x7f07cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:20.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.295+0000 7f07d3526700 1 -- 172.21.15.150:0/2603310417 shutdown_connections 2024-05-10T04:03:20.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.295+0000 7f07d3526700 1 -- 172.21.15.150:0/2603310417 wait complete. 2024-05-10T04:03:20.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.296+0000 7f07d3526700 1 Processor -- start 2024-05-10T04:03:20.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.296+0000 7f07d3526700 1 -- start start 2024-05-10T04:03:20.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.296+0000 7f07d3526700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07cc100e00 0x7f07cc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:20.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.296+0000 7f07d3526700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07cc105730 0x7f07cc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:20.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.296+0000 7f07d3526700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07cc106080 0x7f07cc1a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:20.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.296+0000 7f07d3526700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0fda30 con 0x7f07cc106080 2024-05-10T04:03:20.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.296+0000 7f07d3526700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc1a6ba0 con 0x7f07cc105730 2024-05-10T04:03:20.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.296+0000 7f07d3526700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc1a6ce0 con 0x7f07cc100e00 2024-05-10T04:03:20.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.296+0000 7f07d1ac3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07cc106080 0x7f07cc1a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:20.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.296+0000 7f07d1ac3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07cc106080 0x7f07cc1a6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52528/0 (socket says 172.21.15.150:52528) 2024-05-10T04:03:20.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.296+0000 7f07d1ac3700 1 -- 172.21.15.150:0/738106120 learned_addr learned my addr 172.21.15.150:0/738106120 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:20.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.296+0000 7f07d0ac1700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07cc105730 0x7f07cc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:20.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.296+0000 7f07d12c2700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07cc100e00 0x7f07cc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:20.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.297+0000 7f07d1ac3700 1 -- 172.21.15.150:0/738106120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07cc100e00 msgr2=0x7f07cc0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:20.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.297+0000 7f07d1ac3700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07cc100e00 0x7f07cc0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:20.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.297+0000 7f07d1ac3700 1 -- 172.21.15.150:0/738106120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07cc105730 msgr2=0x7f07cc0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:20.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.297+0000 7f07d1ac3700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07cc105730 0x7f07cc0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:20.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.297+0000 7f07d1ac3700 1 -- 172.21.15.150:0/738106120 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c80095f0 con 0x7f07cc106080 2024-05-10T04:03:20.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.297+0000 7f07d0ac1700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07cc105730 0x7f07cc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:20.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.297+0000 7f07d12c2700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07cc100e00 0x7f07cc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:20.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.297+0000 7f07d1ac3700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07cc106080 0x7f07cc1a6660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f07c4007ee0 tx=0x7f07c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:20.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.297+0000 7f07c27fc700 1 -- 172.21.15.150:0/738106120 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c4017070 con 0x7f07cc106080 2024-05-10T04:03:20.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.297+0000 7f07c27fc700 1 -- 172.21.15.150:0/738106120 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07c400f040 con 0x7f07cc106080 2024-05-10T04:03:20.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.297+0000 7f07d3526700 1 -- 172.21.15.150:0/738106120 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07cc1a6f70 con 0x7f07cc106080 2024-05-10T04:03:20.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.297+0000 7f07c27fc700 1 -- 172.21.15.150:0/738106120 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c4005740 con 0x7f07cc106080 2024-05-10T04:03:20.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.297+0000 7f07d3526700 1 -- 172.21.15.150:0/738106120 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07cc1a7360 con 0x7f07cc106080 2024-05-10T04:03:20.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.298+0000 7f07d3526700 1 -- 172.21.15.150:0/738106120 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07cc1022b0 con 0x7f07cc106080 2024-05-10T04:03:20.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.299+0000 7f07c27fc700 1 -- 172.21.15.150:0/738106120 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f07c40058a0 con 0x7f07cc106080 2024-05-10T04:03:20.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.300+0000 7f07c27fc700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f07b80a11b0 0x7f07b80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:20.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.300+0000 7f07d12c2700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f07b80a11b0 0x7f07b80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:20.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.300+0000 7f07c27fc700 1 -- 172.21.15.150:0/738106120 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f07c40ccb30 con 0x7f07cc106080 2024-05-10T04:03:20.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.301+0000 7f07d12c2700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f07b80a11b0 0x7f07b80a36c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f07bc005950 tx=0x7f07bc005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:20.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.304+0000 7f07c27fc700 1 -- 172.21.15.150:0/738106120 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f07c4095f70 con 0x7f07cc106080 2024-05-10T04:03:20.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:20 smithi181 ceph-mon[46602]: pgmap v250: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-10T04:03:20.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:20 smithi161 ceph-mon[46918]: pgmap v250: 129 pgs: 129 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-10T04:03:20.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.535+0000 7f07d3526700 1 -- 172.21.15.150:0/738106120 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f07cc0fe3c0 con 0x7f07cc106080 2024-05-10T04:03:20.547 INFO:teuthology.orchestra.run.smithi150.stdout:326417514516 2024-05-10T04:03:20.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.535+0000 7f07c27fc700 1 -- 172.21.15.150:0/738106120 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f07c4099590 con 0x7f07cc106080 2024-05-10T04:03:20.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.537+0000 7f07d3526700 1 -- 172.21.15.150:0/738106120 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f07b80a11b0 msgr2=0x7f07b80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:20.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.537+0000 7f07d3526700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f07b80a11b0 0x7f07b80a36c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f07bc005950 tx=0x7f07bc005f50 comp rx=0 tx=0).stop 2024-05-10T04:03:20.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.537+0000 7f07d3526700 1 -- 172.21.15.150:0/738106120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07cc106080 msgr2=0x7f07cc1a6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:20.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.537+0000 7f07d3526700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07cc106080 0x7f07cc1a6660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f07c4007ee0 tx=0x7f07c400d3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:20.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.537+0000 7f07d3526700 1 -- 172.21.15.150:0/738106120 shutdown_connections 2024-05-10T04:03:20.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.537+0000 7f07d3526700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f07b80a11b0 0x7f07b80a36c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:20.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.537+0000 7f07d3526700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07cc106080 0x7f07cc1a6660 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:20.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.537+0000 7f07d3526700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07cc100e00 0x7f07cc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:20.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.537+0000 7f07d3526700 1 --2- 172.21.15.150:0/738106120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07cc105730 0x7f07cc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:20.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.537+0000 7f07d3526700 1 -- 172.21.15.150:0/738106120 >> 172.21.15.150:0/738106120 conn(0x7f07cc0fc7a0 msgr2=0x7f07cc106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:20.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.538+0000 7f07d3526700 1 -- 172.21.15.150:0/738106120 shutdown_connections 2024-05-10T04:03:20.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:20.538+0000 7f07d3526700 1 -- 172.21.15.150:0/738106120 wait complete. 2024-05-10T04:03:20.817 INFO:teuthology.orchestra.run.smithi150.stdout:55834574903 2024-05-10T04:03:20.818 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-10T04:03:20.962 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514515 got 326417514516 for osd.11 2024-05-10T04:03:20.962 DEBUG:teuthology.parallel:result is None 2024-05-10T04:03:21.381 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:21 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/738106120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T04:03:21.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:21 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/738106120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T04:03:21.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:21 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/738106120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T04:03:21.625 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:21.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.797+0000 7f555b08a700 1 -- 172.21.15.150:0/3415797565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5554105750 msgr2=0x7f5554105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:21.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.797+0000 7f555b08a700 1 --2- 172.21.15.150:0/3415797565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5554105750 0x7f5554105b60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5540009990 tx=0x7f5540009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:21.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.797+0000 7f555b08a700 1 -- 172.21.15.150:0/3415797565 shutdown_connections 2024-05-10T04:03:21.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.797+0000 7f555b08a700 1 --2- 172.21.15.150:0/3415797565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55541060a0 0x7f555410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:21.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.797+0000 7f555b08a700 1 --2- 172.21.15.150:0/3415797565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5554100e20 0x7f5554101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.797+0000 7f555b08a700 1 --2- 172.21.15.150:0/3415797565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5554105750 0x7f5554105b60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.797+0000 7f555b08a700 1 -- 172.21.15.150:0/3415797565 >> 172.21.15.150:0/3415797565 conn(0x7f55540fc7a0 msgr2=0x7f55540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.797+0000 7f555b08a700 1 -- 172.21.15.150:0/3415797565 shutdown_connections 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.797+0000 7f555b08a700 1 -- 172.21.15.150:0/3415797565 wait complete. 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f555b08a700 1 Processor -- start 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f555b08a700 1 -- start start 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f555b08a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5554100e20 0x7f55541973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f555b08a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5554105750 0x7f5554197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f555b08a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55541060a0 0x7f555419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f555b08a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555419c3f0 con 0x7f55541060a0 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f555b08a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555419c560 con 0x7f5554105750 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f555b08a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555419c720 con 0x7f5554100e20 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f5559627700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55541060a0 0x7f555419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:21.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f5559627700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55541060a0 0x7f555419be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52542/0 (socket says 172.21.15.150:52542) 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f5559627700 1 -- 172.21.15.150:0/308430867 learned_addr learned my addr 172.21.15.150:0/308430867 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f5553fff700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5554105750 0x7f5554197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f5558e26700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5554100e20 0x7f55541973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f5559627700 1 -- 172.21.15.150:0/308430867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5554100e20 msgr2=0x7f55541973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f5559627700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5554100e20 0x7f55541973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f5559627700 1 -- 172.21.15.150:0/308430867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5554105750 msgr2=0x7f5554197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f5559627700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5554105750 0x7f5554197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f5559627700 1 -- 172.21.15.150:0/308430867 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55400095f0 con 0x7f55541060a0 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f5553fff700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5554105750 0x7f5554197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.798+0000 7f5559627700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55541060a0 0x7f555419be80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f554c007ee0 tx=0x7f554c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.799+0000 7f5551ffb700 1 -- 172.21.15.150:0/308430867 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c017070 con 0x7f55541060a0 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.799+0000 7f5551ffb700 1 -- 172.21.15.150:0/308430867 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f554c00f040 con 0x7f55541060a0 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.799+0000 7f555b08a700 1 -- 172.21.15.150:0/308430867 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555419ca00 con 0x7f55541060a0 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.799+0000 7f5551ffb700 1 -- 172.21.15.150:0/308430867 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c005740 con 0x7f55541060a0 2024-05-10T04:03:21.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.799+0000 7f555b08a700 1 -- 172.21.15.150:0/308430867 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555419cce0 con 0x7f55541060a0 2024-05-10T04:03:21.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.800+0000 7f555b08a700 1 -- 172.21.15.150:0/308430867 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5554109150 con 0x7f55541060a0 2024-05-10T04:03:21.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.801+0000 7f5551ffb700 1 -- 172.21.15.150:0/308430867 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f554c0058a0 con 0x7f55541060a0 2024-05-10T04:03:21.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.802+0000 7f5551ffb700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f55440a11b0 0x7f55440a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:21.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.802+0000 7f5558e26700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f55440a11b0 0x7f55440a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:21.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.802+0000 7f5551ffb700 1 -- 172.21.15.150:0/308430867 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f554c0cc500 con 0x7f55541060a0 2024-05-10T04:03:21.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.802+0000 7f5558e26700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f55440a11b0 0x7f55440a36c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5548005950 tx=0x7f55480058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:21.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:21.807+0000 7f5551ffb700 1 -- 172.21.15.150:0/308430867 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f554c0958e0 con 0x7f55541060a0 2024-05-10T04:03:22.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.015+0000 7f555b08a700 1 -- 172.21.15.150:0/308430867 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f555404edc0 con 0x7f55541060a0 2024-05-10T04:03:22.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.015+0000 7f5551ffb700 1 -- 172.21.15.150:0/308430867 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f554c098f00 con 0x7f55541060a0 2024-05-10T04:03:22.018 INFO:teuthology.orchestra.run.smithi150.stdout:158913789992 2024-05-10T04:03:22.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.018+0000 7f555b08a700 1 -- 172.21.15.150:0/308430867 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f55440a11b0 msgr2=0x7f55440a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:22.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.018+0000 7f555b08a700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f55440a11b0 0x7f55440a36c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5548005950 tx=0x7f55480058e0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.018+0000 7f555b08a700 1 -- 172.21.15.150:0/308430867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55541060a0 msgr2=0x7f555419be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:22.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.018+0000 7f555b08a700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55541060a0 0x7f555419be80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f554c007ee0 tx=0x7f554c00d3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.018+0000 7f555b08a700 1 -- 172.21.15.150:0/308430867 shutdown_connections 2024-05-10T04:03:22.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.018+0000 7f555b08a700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f55440a11b0 0x7f55440a36c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.018+0000 7f555b08a700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55541060a0 0x7f555419be80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.018+0000 7f555b08a700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5554100e20 0x7f55541973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.018+0000 7f555b08a700 1 --2- 172.21.15.150:0/308430867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5554105750 0x7f5554197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.018+0000 7f555b08a700 1 -- 172.21.15.150:0/308430867 >> 172.21.15.150:0/308430867 conn(0x7f55540fc7a0 msgr2=0x7f5554107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:22.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.019+0000 7f555b08a700 1 -- 172.21.15.150:0/308430867 shutdown_connections 2024-05-10T04:03:22.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.019+0000 7f555b08a700 1 -- 172.21.15.150:0/308430867 wait complete. 2024-05-10T04:03:22.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.159+0000 7ff561842700 1 -- 172.21.15.150:0/2712203131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55c102df0 msgr2=0x7ff55c103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:22.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.159+0000 7ff561842700 1 --2- 172.21.15.150:0/2712203131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55c102df0 0x7ff55c103200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff544009990 tx=0x7ff544009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.159+0000 7ff561842700 1 -- 172.21.15.150:0/2712203131 shutdown_connections 2024-05-10T04:03:22.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.159+0000 7ff561842700 1 --2- 172.21.15.150:0/2712203131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff55c10c260 0x7ff55c10e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.159+0000 7ff561842700 1 --2- 172.21.15.150:0/2712203131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55c102df0 0x7ff55c103200 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.159+0000 7ff561842700 1 --2- 172.21.15.150:0/2712203131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff55c103740 0x7ff55c10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.159+0000 7ff561842700 1 -- 172.21.15.150:0/2712203131 >> 172.21.15.150:0/2712203131 conn(0x7ff55c0fc7c0 msgr2=0x7ff55c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:22.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.159+0000 7ff561842700 1 -- 172.21.15.150:0/2712203131 shutdown_connections 2024-05-10T04:03:22.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.159+0000 7ff561842700 1 -- 172.21.15.150:0/2712203131 wait complete. 2024-05-10T04:03:22.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.160+0000 7ff561842700 1 Processor -- start 2024-05-10T04:03:22.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.160+0000 7ff561842700 1 -- start start 2024-05-10T04:03:22.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.160+0000 7ff561842700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55c102df0 0x7ff55c100ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:22.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.160+0000 7ff561842700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff55c103740 0x7ff55c101020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:22.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.160+0000 7ff561842700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff55c10c260 0x7ff55c1a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:22.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.160+0000 7ff561842700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c101890 con 0x7ff55c102df0 2024-05-10T04:03:22.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.160+0000 7ff561842700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c1a6bd0 con 0x7ff55c10c260 2024-05-10T04:03:22.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.160+0000 7ff561842700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c1a6d10 con 0x7ff55c103740 2024-05-10T04:03:22.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.161+0000 7ff55affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55c102df0 0x7ff55c100ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:22.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.161+0000 7ff55affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55c102df0 0x7ff55c100ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52554/0 (socket says 172.21.15.150:52554) 2024-05-10T04:03:22.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.161+0000 7ff55affd700 1 -- 172.21.15.150:0/2890457949 learned_addr learned my addr 172.21.15.150:0/2890457949 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:22.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.161+0000 7ff55a7fc700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff55c103740 0x7ff55c101020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:22.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.161+0000 7ff55b7fe700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff55c10c260 0x7ff55c1a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:22.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.161+0000 7ff55affd700 1 -- 172.21.15.150:0/2890457949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff55c103740 msgr2=0x7ff55c101020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:22.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.161+0000 7ff55affd700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff55c103740 0x7ff55c101020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.161+0000 7ff55affd700 1 -- 172.21.15.150:0/2890457949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff55c10c260 msgr2=0x7ff55c1a6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:22.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.161+0000 7ff55affd700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff55c10c260 0x7ff55c1a6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.161+0000 7ff55affd700 1 -- 172.21.15.150:0/2890457949 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5440095f0 con 0x7ff55c102df0 2024-05-10T04:03:22.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.161+0000 7ff55affd700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55c102df0 0x7ff55c100ae0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5440098e0 tx=0x7ff54400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:22.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.165+0000 7ff560840700 1 -- 172.21.15.150:0/2890457949 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54401d070 con 0x7ff55c102df0 2024-05-10T04:03:22.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.165+0000 7ff560840700 1 -- 172.21.15.150:0/2890457949 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff544022470 con 0x7ff55c102df0 2024-05-10T04:03:22.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.165+0000 7ff560840700 1 -- 172.21.15.150:0/2890457949 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544017650 con 0x7ff55c102df0 2024-05-10T04:03:22.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.165+0000 7ff561842700 1 -- 172.21.15.150:0/2890457949 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55c1a6f40 con 0x7ff55c102df0 2024-05-10T04:03:22.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.165+0000 7ff561842700 1 -- 172.21.15.150:0/2890457949 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55c1a72a0 con 0x7ff55c102df0 2024-05-10T04:03:22.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.166+0000 7ff561842700 1 -- 172.21.15.150:0/2890457949 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55c1073e0 con 0x7ff55c102df0 2024-05-10T04:03:22.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.167+0000 7ff560840700 1 -- 172.21.15.150:0/2890457949 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff5440225e0 con 0x7ff55c102df0 2024-05-10T04:03:22.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.168+0000 7ff560840700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff5480a11b0 0x7ff5480a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:22.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.168+0000 7ff560840700 1 -- 172.21.15.150:0/2890457949 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff5440ce030 con 0x7ff55c102df0 2024-05-10T04:03:22.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.168+0000 7ff55a7fc700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff5480a11b0 0x7ff5480a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:22.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.168+0000 7ff55a7fc700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff5480a11b0 0x7ff5480a36c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0ffc70 tx=0x7ff54c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:22.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.172+0000 7ff560840700 1 -- 172.21.15.150:0/2890457949 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff54409af90 con 0x7ff55c102df0 2024-05-10T04:03:22.220 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:22 smithi150 ceph-mon[42477]: pgmap v251: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-10T04:03:22.220 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:22 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/308430867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T04:03:22.305 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:22.306 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:22.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.416+0000 7ff561842700 1 -- 172.21.15.150:0/2890457949 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff55c073d20 con 0x7ff55c102df0 2024-05-10T04:03:22.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.416+0000 7ff560840700 1 -- 172.21.15.150:0/2890457949 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff54409ab20 con 0x7ff55c102df0 2024-05-10T04:03:22.419 INFO:teuthology.orchestra.run.smithi150.stdout:34359738426 2024-05-10T04:03:22.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.418+0000 7ff561842700 1 -- 172.21.15.150:0/2890457949 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff5480a11b0 msgr2=0x7ff5480a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:22.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.419+0000 7ff561842700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff5480a11b0 0x7ff5480a36c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0ffc70 tx=0x7ff54c00b410 comp rx=0 tx=0).stop 2024-05-10T04:03:22.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.419+0000 7ff561842700 1 -- 172.21.15.150:0/2890457949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55c102df0 msgr2=0x7ff55c100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:22.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.419+0000 7ff561842700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55c102df0 0x7ff55c100ae0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5440098e0 tx=0x7ff54400f670 comp rx=0 tx=0).stop 2024-05-10T04:03:22.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.419+0000 7ff561842700 1 -- 172.21.15.150:0/2890457949 shutdown_connections 2024-05-10T04:03:22.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.419+0000 7ff561842700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff5480a11b0 0x7ff5480a36c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.419+0000 7ff561842700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff55c10c260 0x7ff55c1a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.419+0000 7ff561842700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff55c102df0 0x7ff55c100ae0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.419+0000 7ff561842700 1 --2- 172.21.15.150:0/2890457949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff55c103740 0x7ff55c101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:22.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.419+0000 7ff561842700 1 -- 172.21.15.150:0/2890457949 >> 172.21.15.150:0/2890457949 conn(0x7ff55c0fc7c0 msgr2=0x7ff55c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:22.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.419+0000 7ff561842700 1 -- 172.21.15.150:0/2890457949 shutdown_connections 2024-05-10T04:03:22.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:22.419+0000 7ff561842700 1 -- 172.21.15.150:0/2890457949 wait complete. 2024-05-10T04:03:22.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:22 smithi181 ceph-mon[46602]: pgmap v251: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-10T04:03:22.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:22 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/308430867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T04:03:22.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:22 smithi161 ceph-mon[46918]: pgmap v251: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-10T04:03:22.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:22 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/308430867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T04:03:22.538 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789991 got 158913789992 for osd.5 2024-05-10T04:03:22.538 DEBUG:teuthology.parallel:result is None 2024-05-10T04:03:22.671 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:23.162 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738424 got 34359738426 for osd.0 2024-05-10T04:03:23.162 DEBUG:teuthology.parallel:result is None 2024-05-10T04:03:23.364 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:23 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2890457949' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T04:03:23.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:23 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2890457949' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T04:03:23.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:23 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2890457949' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T04:03:23.510 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:24.164 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:24.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:24 smithi150 ceph-mon[42477]: pgmap v252: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T04:03:24.265 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:24.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.475+0000 7f68e7e10700 1 -- 172.21.15.150:0/4212501827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e0105730 msgr2=0x7f68e0105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.475+0000 7f68e7e10700 1 --2- 172.21.15.150:0/4212501827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e0105730 0x7f68e0105b40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009990 tx=0x7f68d0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.475+0000 7f68e7e10700 1 -- 172.21.15.150:0/4212501827 shutdown_connections 2024-05-10T04:03:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.475+0000 7f68e7e10700 1 --2- 172.21.15.150:0/4212501827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68e0106080 0x7f68e010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.475+0000 7f68e7e10700 1 --2- 172.21.15.150:0/4212501827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68e0100e00 0x7f68e0101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.475+0000 7f68e7e10700 1 --2- 172.21.15.150:0/4212501827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e0105730 0x7f68e0105b40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.475+0000 7f68e7e10700 1 -- 172.21.15.150:0/4212501827 >> 172.21.15.150:0/4212501827 conn(0x7f68e00fc7a0 msgr2=0x7f68e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.475+0000 7f68e7e10700 1 -- 172.21.15.150:0/4212501827 shutdown_connections 2024-05-10T04:03:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.475+0000 7f68e7e10700 1 -- 172.21.15.150:0/4212501827 wait complete. 2024-05-10T04:03:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.476+0000 7f68e7e10700 1 Processor -- start 2024-05-10T04:03:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.476+0000 7f68e7e10700 1 -- start start 2024-05-10T04:03:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.476+0000 7f68e7e10700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e0100e00 0x7f68e01973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.476+0000 7f68e7e10700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68e0105730 0x7f68e0197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.476+0000 7f68e7e10700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68e0106080 0x7f68e019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.476+0000 7f68e7e10700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e019c3d0 con 0x7f68e0100e00 2024-05-10T04:03:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.476+0000 7f68e7e10700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e019c540 con 0x7f68e0106080 2024-05-10T04:03:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.476+0000 7f68e7e10700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e019c700 con 0x7f68e0105730 2024-05-10T04:03:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68e5bac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e0100e00 0x7f68e01973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68e5bac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e0100e00 0x7f68e01973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52580/0 (socket says 172.21.15.150:52580) 2024-05-10T04:03:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68e5bac700 1 -- 172.21.15.150:0/1940082447 learned_addr learned my addr 172.21.15.150:0/1940082447 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:24.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68e63ad700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68e0106080 0x7f68e019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:24.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68e53ab700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68e0105730 0x7f68e0197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:24.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68e5bac700 1 -- 172.21.15.150:0/1940082447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68e0105730 msgr2=0x7f68e0197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68e5bac700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68e0105730 0x7f68e0197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68e5bac700 1 -- 172.21.15.150:0/1940082447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68e0106080 msgr2=0x7f68e019be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68e5bac700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68e0106080 0x7f68e019be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68e5bac700 1 -- 172.21.15.150:0/1940082447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d00095f0 con 0x7f68e0100e00 2024-05-10T04:03:24.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68e5bac700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e0100e00 0x7f68e01973b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00d950 tx=0x7f68dc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:24.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68d6ffd700 1 -- 172.21.15.150:0/1940082447 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc009940 con 0x7f68e0100e00 2024-05-10T04:03:24.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68d6ffd700 1 -- 172.21.15.150:0/1940082447 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68dc010460 con 0x7f68e0100e00 2024-05-10T04:03:24.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68e7e10700 1 -- 172.21.15.150:0/1940082447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e019c9e0 con 0x7f68e0100e00 2024-05-10T04:03:24.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68d6ffd700 1 -- 172.21.15.150:0/1940082447 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc00f620 con 0x7f68e0100e00 2024-05-10T04:03:24.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.477+0000 7f68e7e10700 1 -- 172.21.15.150:0/1940082447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e01a42e0 con 0x7f68e0100e00 2024-05-10T04:03:24.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.479+0000 7f68e7e10700 1 -- 172.21.15.150:0/1940082447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e0109130 con 0x7f68e0100e00 2024-05-10T04:03:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.481+0000 7f68d6ffd700 1 -- 172.21.15.150:0/1940082447 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f68dc0108d0 con 0x7f68e0100e00 2024-05-10T04:03:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.481+0000 7f68d6ffd700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f68cc0a1230 0x7f68cc0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.481+0000 7f68e53ab700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f68cc0a1230 0x7f68cc0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.481+0000 7f68d6ffd700 1 -- 172.21.15.150:0/1940082447 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f68dc0cca90 con 0x7f68e0100e00 2024-05-10T04:03:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.482+0000 7f68e53ab700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f68cc0a1230 0x7f68cc0a3740 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f68d0006010 tx=0x7f68d000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:24.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.485+0000 7f68d6ffd700 1 -- 172.21.15.150:0/1940082447 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f68dc095ed0 con 0x7f68e0100e00 2024-05-10T04:03:24.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:24 smithi181 ceph-mon[46602]: pgmap v252: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T04:03:24.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:24 smithi161 ceph-mon[46918]: pgmap v252: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T04:03:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.670+0000 7f560dd46700 1 -- 172.21.15.150:0/1864050671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608105770 msgr2=0x7f5608105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.670+0000 7f560dd46700 1 --2- 172.21.15.150:0/1864050671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608105770 0x7f5608105b80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009990 tx=0x7f55fc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.670+0000 7f560dd46700 1 -- 172.21.15.150:0/1864050671 shutdown_connections 2024-05-10T04:03:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.670+0000 7f560dd46700 1 --2- 172.21.15.150:0/1864050671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56081060c0 0x7f560810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.670+0000 7f560dd46700 1 --2- 172.21.15.150:0/1864050671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5608100e40 0x7f5608101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.670+0000 7f560dd46700 1 --2- 172.21.15.150:0/1864050671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608105770 0x7f5608105b80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.670+0000 7f560dd46700 1 -- 172.21.15.150:0/1864050671 >> 172.21.15.150:0/1864050671 conn(0x7f56080fc730 msgr2=0x7f56080feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.670+0000 7f560dd46700 1 -- 172.21.15.150:0/1864050671 shutdown_connections 2024-05-10T04:03:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.670+0000 7f560dd46700 1 -- 172.21.15.150:0/1864050671 wait complete. 2024-05-10T04:03:24.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.671+0000 7f560dd46700 1 Processor -- start 2024-05-10T04:03:24.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.671+0000 7f560dd46700 1 -- start start 2024-05-10T04:03:24.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.671+0000 7f560dd46700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608100e40 0x7f5608198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:24.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.671+0000 7f560dd46700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5608105770 0x7f5608198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:24.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.671+0000 7f560dd46700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56081060c0 0x7f560819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:24.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.671+0000 7f560dd46700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560819d480 con 0x7f5608100e40 2024-05-10T04:03:24.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.671+0000 7f560dd46700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560819d5f0 con 0x7f5608105770 2024-05-10T04:03:24.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.671+0000 7f560dd46700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560819d7b0 con 0x7f56081060c0 2024-05-10T04:03:24.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f56077fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608100e40 0x7f5608198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:24.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f56077fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608100e40 0x7f5608198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52608/0 (socket says 172.21.15.150:52608) 2024-05-10T04:03:24.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f56077fe700 1 -- 172.21.15.150:0/1105402281 learned_addr learned my addr 172.21.15.150:0/1105402281 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:24.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f5606ffd700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5608105770 0x7f5608198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:24.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f5607fff700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56081060c0 0x7f560819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:24.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f56077fe700 1 -- 172.21.15.150:0/1105402281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56081060c0 msgr2=0x7f560819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f56077fe700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56081060c0 0x7f560819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f56077fe700 1 -- 172.21.15.150:0/1105402281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5608105770 msgr2=0x7f5608198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f56077fe700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5608105770 0x7f5608198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f56077fe700 1 -- 172.21.15.150:0/1105402281 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc0095f0 con 0x7f5608100e40 2024-05-10T04:03:24.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f5606ffd700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5608105770 0x7f5608198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:03:24.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f56077fe700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608100e40 0x7f5608198460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f55f800e9a0 tx=0x7f55f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:24.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f5604ff9700 1 -- 172.21.15.150:0/1105402281 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8009790 con 0x7f5608100e40 2024-05-10T04:03:24.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f560dd46700 1 -- 172.21.15.150:0/1105402281 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560819da90 con 0x7f5608100e40 2024-05-10T04:03:24.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f560dd46700 1 -- 172.21.15.150:0/1105402281 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56081a5390 con 0x7f5608100e40 2024-05-10T04:03:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.672+0000 7f5604ff9700 1 -- 172.21.15.150:0/1105402281 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55f8010430 con 0x7f5608100e40 2024-05-10T04:03:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.673+0000 7f5604ff9700 1 -- 172.21.15.150:0/1105402281 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8018680 con 0x7f5608100e40 2024-05-10T04:03:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.674+0000 7f560dd46700 1 -- 172.21.15.150:0/1105402281 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560810a180 con 0x7f5608100e40 2024-05-10T04:03:24.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.676+0000 7f5604ff9700 1 -- 172.21.15.150:0/1105402281 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f55f8010810 con 0x7f5608100e40 2024-05-10T04:03:24.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.676+0000 7f5604ff9700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f55f00a55c0 0x7f55f00a7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:24.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.676+0000 7f5604ff9700 1 -- 172.21.15.150:0/1105402281 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f55f8014070 con 0x7f5608100e40 2024-05-10T04:03:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.676+0000 7f5606ffd700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f55f00a55c0 0x7f55f00a7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.677+0000 7f5606ffd700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f55f00a55c0 0x7f55f00a7ad0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f55fc006010 tx=0x7f55fc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:24.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.682+0000 7f5604ff9700 1 -- 172.21.15.150:0/1105402281 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f55f8052d60 con 0x7f5608100e40 2024-05-10T04:03:24.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.699+0000 7f68e7e10700 1 -- 172.21.15.150:0/1940082447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f68e01022b0 con 0x7f68e0100e00 2024-05-10T04:03:24.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.700+0000 7f68d6ffd700 1 -- 172.21.15.150:0/1940082447 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f68dc020090 con 0x7f68e0100e00 2024-05-10T04:03:24.702 INFO:teuthology.orchestra.run.smithi150.stdout:107374182449 2024-05-10T04:03:24.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.702+0000 7f68e7e10700 1 -- 172.21.15.150:0/1940082447 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f68cc0a1230 msgr2=0x7f68cc0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.702+0000 7f68e7e10700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f68cc0a1230 0x7f68cc0a3740 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f68d0006010 tx=0x7f68d000b540 comp rx=0 tx=0).stop 2024-05-10T04:03:24.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.702+0000 7f68e7e10700 1 -- 172.21.15.150:0/1940082447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e0100e00 msgr2=0x7f68e01973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.702+0000 7f68e7e10700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e0100e00 0x7f68e01973b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00d950 tx=0x7f68dc00dc60 comp rx=0 tx=0).stop 2024-05-10T04:03:24.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.702+0000 7f68e7e10700 1 -- 172.21.15.150:0/1940082447 shutdown_connections 2024-05-10T04:03:24.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.702+0000 7f68e7e10700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f68cc0a1230 0x7f68cc0a3740 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.702+0000 7f68e7e10700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68e0106080 0x7f68e019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.702+0000 7f68e7e10700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e0100e00 0x7f68e01973b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.702+0000 7f68e7e10700 1 --2- 172.21.15.150:0/1940082447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68e0105730 0x7f68e0197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.702+0000 7f68e7e10700 1 -- 172.21.15.150:0/1940082447 >> 172.21.15.150:0/1940082447 conn(0x7f68e00fc7a0 msgr2=0x7f68e0106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:24.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.702+0000 7f68e7e10700 1 -- 172.21.15.150:0/1940082447 shutdown_connections 2024-05-10T04:03:24.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.702+0000 7f68e7e10700 1 -- 172.21.15.150:0/1940082447 wait complete. 2024-05-10T04:03:24.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.894+0000 7f560dd46700 1 -- 172.21.15.150:0/1105402281 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f56081022f0 con 0x7f5608100e40 2024-05-10T04:03:24.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.894+0000 7f5604ff9700 1 -- 172.21.15.150:0/1105402281 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f55f8099de0 con 0x7f5608100e40 2024-05-10T04:03:24.897 INFO:teuthology.orchestra.run.smithi150.stdout:274877906972 2024-05-10T04:03:24.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.897+0000 7f560dd46700 1 -- 172.21.15.150:0/1105402281 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f55f00a55c0 msgr2=0x7f55f00a7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.897+0000 7f560dd46700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f55f00a55c0 0x7f55f00a7ad0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f55fc006010 tx=0x7f55fc01a040 comp rx=0 tx=0).stop 2024-05-10T04:03:24.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.897+0000 7f560dd46700 1 -- 172.21.15.150:0/1105402281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608100e40 msgr2=0x7f5608198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.897+0000 7f560dd46700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608100e40 0x7f5608198460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f55f800e9a0 tx=0x7f55f800ecb0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.897+0000 7f560dd46700 1 -- 172.21.15.150:0/1105402281 shutdown_connections 2024-05-10T04:03:24.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.897+0000 7f560dd46700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f55f00a55c0 0x7f55f00a7ad0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.897+0000 7f560dd46700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56081060c0 0x7f560819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.897+0000 7f560dd46700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5608100e40 0x7f5608198460 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.897+0000 7f560dd46700 1 --2- 172.21.15.150:0/1105402281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5608105770 0x7f5608198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.897+0000 7f560dd46700 1 -- 172.21.15.150:0/1105402281 >> 172.21.15.150:0/1105402281 conn(0x7f56080fc730 msgr2=0x7f5608069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:24.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.897+0000 7f560dd46700 1 -- 172.21.15.150:0/1105402281 shutdown_connections 2024-05-10T04:03:24.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.897+0000 7f560dd46700 1 -- 172.21.15.150:0/1105402281 wait complete. 2024-05-10T04:03:24.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.440+0000 7f41207c3700 1 -- 172.21.15.150:0/1194177747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4118105730 msgr2=0x7f4118105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.440+0000 7f41207c3700 1 --2- 172.21.15.150:0/1194177747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4118105730 0x7f4118105b40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4108009990 tx=0x7f4108009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.440+0000 7f41207c3700 1 -- 172.21.15.150:0/1194177747 shutdown_connections 2024-05-10T04:03:24.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.440+0000 7f41207c3700 1 --2- 172.21.15.150:0/1194177747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4118106080 0x7f411810d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.440+0000 7f41207c3700 1 --2- 172.21.15.150:0/1194177747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4118100e00 0x7f4118101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.440+0000 7f41207c3700 1 --2- 172.21.15.150:0/1194177747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4118105730 0x7f4118105b40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.440+0000 7f41207c3700 1 -- 172.21.15.150:0/1194177747 >> 172.21.15.150:0/1194177747 conn(0x7f41180fc7a0 msgr2=0x7f41180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:24.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.440+0000 7f41207c3700 1 -- 172.21.15.150:0/1194177747 shutdown_connections 2024-05-10T04:03:24.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.440+0000 7f41207c3700 1 -- 172.21.15.150:0/1194177747 wait complete. 2024-05-10T04:03:24.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.440+0000 7f41207c3700 1 Processor -- start 2024-05-10T04:03:24.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f41207c3700 1 -- start start 2024-05-10T04:03:24.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f41207c3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4118100e00 0x7f41181984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:24.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f41207c3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4118105730 0x7f4118198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:24.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f41207c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4118106080 0x7f411819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:24.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f41207c3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411819d500 con 0x7f4118105730 2024-05-10T04:03:24.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f41207c3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411819d670 con 0x7f4118100e00 2024-05-10T04:03:24.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f41207c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411819d830 con 0x7f4118106080 2024-05-10T04:03:24.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f411dd5e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4118105730 0x7f4118198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:24.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f411dd5e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4118105730 0x7f4118198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52558/0 (socket says 172.21.15.150:52558) 2024-05-10T04:03:24.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f411dd5e700 1 -- 172.21.15.150:0/4178990392 learned_addr learned my addr 172.21.15.150:0/4178990392 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:24.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f411ed60700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4118106080 0x7f411819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:24.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f411e55f700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4118100e00 0x7f41181984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f411dd5e700 1 -- 172.21.15.150:0/4178990392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4118106080 msgr2=0x7f411819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f411dd5e700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4118106080 0x7f411819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f411dd5e700 1 -- 172.21.15.150:0/4178990392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4118100e00 msgr2=0x7f41181984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f411dd5e700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4118100e00 0x7f41181984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f411dd5e700 1 -- 172.21.15.150:0/4178990392 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080095f0 con 0x7f4118105730 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f411e55f700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4118100e00 0x7f41181984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.441+0000 7f411dd5e700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4118105730 0x7f4118198a80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f41080098e0 tx=0x7f410800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.442+0000 7f410f7fe700 1 -- 172.21.15.150:0/4178990392 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410801d070 con 0x7f4118105730 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.442+0000 7f410f7fe700 1 -- 172.21.15.150:0/4178990392 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4108022470 con 0x7f4118105730 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.442+0000 7f41207c3700 1 -- 172.21.15.150:0/4178990392 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411819dab0 con 0x7f4118105730 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.442+0000 7f410f7fe700 1 -- 172.21.15.150:0/4178990392 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108017650 con 0x7f4118105730 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.442+0000 7f41207c3700 1 -- 172.21.15.150:0/4178990392 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41181a5240 con 0x7f4118105730 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.443+0000 7f41207c3700 1 -- 172.21.15.150:0/4178990392 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411810a140 con 0x7f4118105730 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.445+0000 7f410f7fe700 1 -- 172.21.15.150:0/4178990392 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f41080225e0 con 0x7f4118105730 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.445+0000 7f410f7fe700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f41040a11b0 0x7f41040a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:24.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.445+0000 7f410f7fe700 1 -- 172.21.15.150:0/4178990392 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f41080ce030 con 0x7f4118105730 2024-05-10T04:03:24.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.446+0000 7f411e55f700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f41040a11b0 0x7f41040a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:24.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.446+0000 7f411e55f700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f41040a11b0 0x7f41040a36c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4114005950 tx=0x7f41140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:24.912 INFO:teuthology.orchestra.run.smithi150.stdout:137438953517 2024-05-10T04:03:24.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.449+0000 7f410f7fe700 1 -- 172.21.15.150:0/4178990392 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f410809af90 con 0x7f4118105730 2024-05-10T04:03:24.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.661+0000 7f41207c3700 1 -- 172.21.15.150:0/4178990392 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f41181022b0 con 0x7f4118105730 2024-05-10T04:03:24.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.662+0000 7f410f7fe700 1 -- 172.21.15.150:0/4178990392 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4108027020 con 0x7f4118105730 2024-05-10T04:03:24.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.664+0000 7f41207c3700 1 -- 172.21.15.150:0/4178990392 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f41040a11b0 msgr2=0x7f41040a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.664+0000 7f41207c3700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f41040a11b0 0x7f41040a36c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4114005950 tx=0x7f41140058e0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.664+0000 7f41207c3700 1 -- 172.21.15.150:0/4178990392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4118105730 msgr2=0x7f4118198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:24.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.664+0000 7f41207c3700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4118105730 0x7f4118198a80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f41080098e0 tx=0x7f410800f670 comp rx=0 tx=0).stop 2024-05-10T04:03:24.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.664+0000 7f41207c3700 1 -- 172.21.15.150:0/4178990392 shutdown_connections 2024-05-10T04:03:24.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.664+0000 7f41207c3700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f41040a11b0 0x7f41040a36c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.664+0000 7f41207c3700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4118106080 0x7f411819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.664+0000 7f41207c3700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4118100e00 0x7f41181984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.664+0000 7f41207c3700 1 --2- 172.21.15.150:0/4178990392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4118105730 0x7f4118198a80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:24.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.664+0000 7f41207c3700 1 -- 172.21.15.150:0/4178990392 >> 172.21.15.150:0/4178990392 conn(0x7f41180fc7a0 msgr2=0x7f4118069140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:24.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.665+0000 7f41207c3700 1 -- 172.21.15.150:0/4178990392 shutdown_connections 2024-05-10T04:03:24.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:24.665+0000 7f41207c3700 1 -- 172.21.15.150:0/4178990392 wait complete. 2024-05-10T04:03:25.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:25 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4178990392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T04:03:25.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:25 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1940082447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T04:03:25.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:25 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1105402281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T04:03:25.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:25 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4178990392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T04:03:25.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:25 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1940082447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T04:03:25.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:25 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1105402281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T04:03:25.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:25 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4178990392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T04:03:25.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:25 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1940082447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T04:03:25.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:25 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1105402281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T04:03:25.520 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953516 got 137438953517 for osd.4 2024-05-10T04:03:25.520 DEBUG:teuthology.parallel:result is None 2024-05-10T04:03:25.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.421+0000 7f7297289700 1 -- 172.21.15.150:0/2022832756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7290101f30 msgr2=0x7f7290102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:25.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.421+0000 7f7297289700 1 --2- 172.21.15.150:0/2022832756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7290101f30 0x7f7290102340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7280009990 tx=0x7f7280009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:25.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.421+0000 7f7297289700 1 -- 172.21.15.150:0/2022832756 shutdown_connections 2024-05-10T04:03:25.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.421+0000 7f7297289700 1 --2- 172.21.15.150:0/2022832756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72901071b0 0x7f729010eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:25.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.421+0000 7f7297289700 1 --2- 172.21.15.150:0/2022832756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7290101f30 0x7f7290102340 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:25.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.421+0000 7f7297289700 1 --2- 172.21.15.150:0/2022832756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7290106860 0x7f7290106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:25.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.421+0000 7f7297289700 1 -- 172.21.15.150:0/2022832756 >> 172.21.15.150:0/2022832756 conn(0x7f72900fd9c0 msgr2=0x7f72900ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:25.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.421+0000 7f7297289700 1 -- 172.21.15.150:0/2022832756 shutdown_connections 2024-05-10T04:03:25.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.421+0000 7f7297289700 1 -- 172.21.15.150:0/2022832756 wait complete. 2024-05-10T04:03:25.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.422+0000 7f7297289700 1 Processor -- start 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.422+0000 7f7297289700 1 -- start start 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7297289700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7290101f30 0x7f72901984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7297289700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7290106860 0x7f7290198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7297289700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72901071b0 0x7f729019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7297289700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729019d500 con 0x7f7290106860 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7297289700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729019d670 con 0x7f72901071b0 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7297289700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729019d830 con 0x7f7290101f30 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7294824700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7290106860 0x7f7290198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7294824700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7290106860 0x7f7290198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52616/0 (socket says 172.21.15.150:52616) 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7294824700 1 -- 172.21.15.150:0/69553758 learned_addr learned my addr 172.21.15.150:0/69553758 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7295826700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72901071b0 0x7f729019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7295025700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7290101f30 0x7f72901984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7294824700 1 -- 172.21.15.150:0/69553758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7290101f30 msgr2=0x7f72901984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:25.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7294824700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7290101f30 0x7f72901984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7294824700 1 -- 172.21.15.150:0/69553758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72901071b0 msgr2=0x7f729019cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7294824700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72901071b0 0x7f729019cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7294824700 1 -- 172.21.15.150:0/69553758 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72800095f0 con 0x7f7290106860 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7295025700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7290101f30 0x7f72901984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7294824700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7290106860 0x7f7290198a80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f728c00d980 tx=0x7f728c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f72867fc700 1 -- 172.21.15.150:0/69553758 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c009930 con 0x7f7290106860 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f72867fc700 1 -- 172.21.15.150:0/69553758 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f728c010460 con 0x7f7290106860 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7297289700 1 -- 172.21.15.150:0/69553758 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729019db10 con 0x7f7290106860 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f72867fc700 1 -- 172.21.15.150:0/69553758 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c00f620 con 0x7f7290106860 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.423+0000 7f7297289700 1 -- 172.21.15.150:0/69553758 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72901a5410 con 0x7f7290106860 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.424+0000 7f7297289700 1 -- 172.21.15.150:0/69553758 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729010a170 con 0x7f7290106860 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.425+0000 7f72867fc700 1 -- 172.21.15.150:0/69553758 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f728c0108d0 con 0x7f7290106860 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.426+0000 7f72867fc700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f727c0a11b0 0x7f727c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:25.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.426+0000 7f7295025700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f727c0a11b0 0x7f727c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:25.725 INFO:teuthology.orchestra.run.smithi150.stdout:240518168608 2024-05-10T04:03:25.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.426+0000 7f72867fc700 1 -- 172.21.15.150:0/69553758 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f728c0cc6f0 con 0x7f7290106860 2024-05-10T04:03:25.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.427+0000 7f7295025700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f727c0a11b0 0x7f727c0a36c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f728000b2e0 tx=0x7f7280005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:25.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.430+0000 7f72867fc700 1 -- 172.21.15.150:0/69553758 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f728c095b30 con 0x7f7290106860 2024-05-10T04:03:25.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.640+0000 7f7297289700 1 -- 172.21.15.150:0/69553758 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f729004edc0 con 0x7f7290106860 2024-05-10T04:03:25.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.640+0000 7f72867fc700 1 -- 172.21.15.150:0/69553758 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f728c020090 con 0x7f7290106860 2024-05-10T04:03:25.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.642+0000 7f7297289700 1 -- 172.21.15.150:0/69553758 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f727c0a11b0 msgr2=0x7f727c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:25.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.642+0000 7f7297289700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f727c0a11b0 0x7f727c0a36c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f728000b2e0 tx=0x7f7280005fb0 comp rx=0 tx=0).stop 2024-05-10T04:03:25.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.642+0000 7f7297289700 1 -- 172.21.15.150:0/69553758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7290106860 msgr2=0x7f7290198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:25.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.642+0000 7f7297289700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7290106860 0x7f7290198a80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f728c00d980 tx=0x7f728c00dc90 comp rx=0 tx=0).stop 2024-05-10T04:03:25.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.643+0000 7f7297289700 1 -- 172.21.15.150:0/69553758 shutdown_connections 2024-05-10T04:03:25.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.643+0000 7f7297289700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f727c0a11b0 0x7f727c0a36c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:25.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.643+0000 7f7297289700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72901071b0 0x7f729019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:25.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.643+0000 7f7297289700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7290101f30 0x7f72901984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:25.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.643+0000 7f7297289700 1 --2- 172.21.15.150:0/69553758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7290106860 0x7f7290198a80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:25.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.643+0000 7f7297289700 1 -- 172.21.15.150:0/69553758 >> 172.21.15.150:0/69553758 conn(0x7f72900fd9c0 msgr2=0x7f7290100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:25.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.643+0000 7f7297289700 1 -- 172.21.15.150:0/69553758 shutdown_connections 2024-05-10T04:03:25.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:25.643+0000 7f7297289700 1 -- 172.21.15.150:0/69553758 wait complete. 2024-05-10T04:03:26.435 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:26 smithi150 ceph-mon[42477]: pgmap v253: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-10T04:03:26.435 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:26 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/69553758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T04:03:26.436 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906970 got 274877906972 for osd.9 2024-05-10T04:03:26.437 DEBUG:teuthology.parallel:result is None 2024-05-10T04:03:26.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:26 smithi181 ceph-mon[46602]: pgmap v253: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-10T04:03:26.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:26 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/69553758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T04:03:26.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:26 smithi161 ceph-mon[46918]: pgmap v253: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-10T04:03:26.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:26 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/69553758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T04:03:26.581 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182448 got 107374182449 for osd.3 2024-05-10T04:03:26.582 DEBUG:teuthology.parallel:result is None 2024-05-10T04:03:26.863 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168605 got 240518168608 for osd.8 2024-05-10T04:03:26.863 DEBUG:teuthology.parallel:result is None 2024-05-10T04:03:26.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.988+0000 7f516e378700 1 -- 172.21.15.150:0/1200957593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5168100a60 msgr2=0x7f516810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:26.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.988+0000 7f516e378700 1 --2- 172.21.15.150:0/1200957593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5168100a60 0x7f516810eaf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f516000b0a0 tx=0x7f516000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:26.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.988+0000 7f516e378700 1 -- 172.21.15.150:0/1200957593 shutdown_connections 2024-05-10T04:03:26.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.988+0000 7f516e378700 1 --2- 172.21.15.150:0/1200957593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5168100a60 0x7f516810eaf0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:26.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.988+0000 7f516e378700 1 --2- 172.21.15.150:0/1200957593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5168106c80 0x7f5168107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:26.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.988+0000 7f516e378700 1 --2- 172.21.15.150:0/1200957593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5168100110 0x7f5168100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:26.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.988+0000 7f516e378700 1 -- 172.21.15.150:0/1200957593 >> 172.21.15.150:0/1200957593 conn(0x7f51680fbea0 msgr2=0x7f51680fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:26.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.988+0000 7f516e378700 1 -- 172.21.15.150:0/1200957593 shutdown_connections 2024-05-10T04:03:26.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.989+0000 7f516e378700 1 -- 172.21.15.150:0/1200957593 wait complete. 2024-05-10T04:03:26.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.989+0000 7f516e378700 1 Processor -- start 2024-05-10T04:03:26.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.989+0000 7f516e378700 1 -- start start 2024-05-10T04:03:26.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f516e378700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5168100110 0x7f516819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:26.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f516e378700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5168100a60 0x7f516819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:26.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f516e378700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5168106c80 0x7f516819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:26.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f516e378700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516819f7e0 con 0x7f5168100110 2024-05-10T04:03:26.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f516e378700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516819f920 con 0x7f5168100a60 2024-05-10T04:03:26.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f516e378700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516819fae0 con 0x7f5168106c80 2024-05-10T04:03:26.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f5167fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5168100110 0x7f516819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:26.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f5167fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5168100110 0x7f516819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52634/0 (socket says 172.21.15.150:52634) 2024-05-10T04:03:26.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f5167fff700 1 -- 172.21.15.150:0/1529221672 learned_addr learned my addr 172.21.15.150:0/1529221672 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:26.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f516c915700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5168106c80 0x7f516819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:26.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f51677fe700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5168100a60 0x7f516819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:26.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f5167fff700 1 -- 172.21.15.150:0/1529221672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5168106c80 msgr2=0x7f516819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:26.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f5167fff700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5168106c80 0x7f516819f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:26.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f5167fff700 1 -- 172.21.15.150:0/1529221672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5168100a60 msgr2=0x7f516819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:26.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.990+0000 7f5167fff700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5168100a60 0x7f516819ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:26.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.991+0000 7f5167fff700 1 -- 172.21.15.150:0/1529221672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160009d00 con 0x7f5168100110 2024-05-10T04:03:26.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.991+0000 7f5167fff700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5168100110 0x7f516819a7e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f515800b4d0 tx=0x7f515800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:26.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.991+0000 7f51657fa700 1 -- 172.21.15.150:0/1529221672 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158011830 con 0x7f5168100110 2024-05-10T04:03:26.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.991+0000 7f516e378700 1 -- 172.21.15.150:0/1529221672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516819fdc0 con 0x7f5168100110 2024-05-10T04:03:26.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.991+0000 7f516e378700 1 -- 172.21.15.150:0/1529221672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51681a7720 con 0x7f5168100110 2024-05-10T04:03:26.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.991+0000 7f51657fa700 1 -- 172.21.15.150:0/1529221672 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5158011e50 con 0x7f5168100110 2024-05-10T04:03:26.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.991+0000 7f51657fa700 1 -- 172.21.15.150:0/1529221672 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158010dc0 con 0x7f5168100110 2024-05-10T04:03:26.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.993+0000 7f516e378700 1 -- 172.21.15.150:0/1529221672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51540052f0 con 0x7f5168100110 2024-05-10T04:03:26.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.994+0000 7f51657fa700 1 -- 172.21.15.150:0/1529221672 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5158011990 con 0x7f5168100110 2024-05-10T04:03:26.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.995+0000 7f51657fa700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f51500a1230 0x7f51500a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:26.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.995+0000 7f51677fe700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f51500a1230 0x7f51500a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:26.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.995+0000 7f51657fa700 1 -- 172.21.15.150:0/1529221672 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f51580cc400 con 0x7f5168100110 2024-05-10T04:03:26.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:26.995+0000 7f51677fe700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f51500a1230 0x7f51500a3740 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f51681061d0 tx=0x7f515c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:27.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.004+0000 7f51657fa700 1 -- 172.21.15.150:0/1529221672 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f51580958f0 con 0x7f5168100110 2024-05-10T04:03:27.055 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:27.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.265+0000 7f516e378700 1 -- 172.21.15.150:0/1529221672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5154005160 con 0x7f5168100110 2024-05-10T04:03:27.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.266+0000 7f51657fa700 1 -- 172.21.15.150:0/1529221672 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5158098f10 con 0x7f5168100110 2024-05-10T04:03:27.268 INFO:teuthology.orchestra.run.smithi150.stdout:55834574905 2024-05-10T04:03:27.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.268+0000 7f516e378700 1 -- 172.21.15.150:0/1529221672 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f51500a1230 msgr2=0x7f51500a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:27.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.268+0000 7f516e378700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f51500a1230 0x7f51500a3740 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f51681061d0 tx=0x7f515c00a300 comp rx=0 tx=0).stop 2024-05-10T04:03:27.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.268+0000 7f516e378700 1 -- 172.21.15.150:0/1529221672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5168100110 msgr2=0x7f516819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:27.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.268+0000 7f516e378700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5168100110 0x7f516819a7e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f515800b4d0 tx=0x7f515800b7e0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.268+0000 7f516e378700 1 -- 172.21.15.150:0/1529221672 shutdown_connections 2024-05-10T04:03:27.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.268+0000 7f516e378700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f51500a1230 0x7f51500a3740 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.268+0000 7f516e378700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5168106c80 0x7f516819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.268+0000 7f516e378700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5168100110 0x7f516819a7e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.268+0000 7f516e378700 1 --2- 172.21.15.150:0/1529221672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5168100a60 0x7f516819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.268+0000 7f516e378700 1 -- 172.21.15.150:0/1529221672 >> 172.21.15.150:0/1529221672 conn(0x7f51680fbea0 msgr2=0x7f5168104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:27.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.268+0000 7f516e378700 1 -- 172.21.15.150:0/1529221672 shutdown_connections 2024-05-10T04:03:27.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.268+0000 7f516e378700 1 -- 172.21.15.150:0/1529221672 wait complete. 2024-05-10T04:03:27.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.226+0000 7fd6396dd700 1 -- 172.21.15.150:0/3898720400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634112cb0 msgr2=0x7fd6341150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:27.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.226+0000 7fd6396dd700 1 --2- 172.21.15.150:0/3898720400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634112cb0 0x7fd6341150d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd62c01c0d0 tx=0x7fd62c01c3e0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.226+0000 7fd6396dd700 1 -- 172.21.15.150:0/3898720400 shutdown_connections 2024-05-10T04:03:27.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.226+0000 7fd6396dd700 1 --2- 172.21.15.150:0/3898720400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634112cb0 0x7fd6341150d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.226+0000 7fd6396dd700 1 --2- 172.21.15.150:0/3898720400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6340735b0 0x7fd6340739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.226+0000 7fd6396dd700 1 --2- 172.21.15.150:0/3898720400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd634073f00 0x7fd634112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.226+0000 7fd6396dd700 1 -- 172.21.15.150:0/3898720400 >> 172.21.15.150:0/3898720400 conn(0x7fd6340fd9c0 msgr2=0x7fd6340ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:27.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.226+0000 7fd6396dd700 1 -- 172.21.15.150:0/3898720400 shutdown_connections 2024-05-10T04:03:27.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.226+0000 7fd6396dd700 1 -- 172.21.15.150:0/3898720400 wait complete. 2024-05-10T04:03:27.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.227+0000 7fd6396dd700 1 Processor -- start 2024-05-10T04:03:27.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.227+0000 7fd6396dd700 1 -- start start 2024-05-10T04:03:27.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.227+0000 7fd6396dd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6340735b0 0x7fd63410e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.227+0000 7fd6396dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd634073f00 0x7fd63410ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.227+0000 7fd6396dd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634112cb0 0x7fd63410f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.227+0000 7fd6396dd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd634112ac0 con 0x7fd6340735b0 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.227+0000 7fd6396dd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63406ac30 con 0x7fd634112cb0 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.227+0000 7fd6396dd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63406adf0 con 0x7fd634073f00 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.228+0000 7fd6337fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634112cb0 0x7fd63410f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.228+0000 7fd6337fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634112cb0 0x7fd63410f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:49894/0 (socket says 172.21.15.150:49894) 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.228+0000 7fd6337fe700 1 -- 172.21.15.150:0/915239386 learned_addr learned my addr 172.21.15.150:0/915239386 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.228+0000 7fd6337fe700 1 -- 172.21.15.150:0/915239386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd634073f00 msgr2=0x7fd63410ebf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.228+0000 7fd6337fe700 1 --2- 172.21.15.150:0/915239386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd634073f00 0x7fd63410ebf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.228+0000 7fd6337fe700 1 -- 172.21.15.150:0/915239386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6340735b0 msgr2=0x7fd63410e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.228+0000 7fd6337fe700 1 --2- 172.21.15.150:0/915239386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6340735b0 0x7fd63410e6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.228+0000 7fd6337fe700 1 -- 172.21.15.150:0/915239386 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c01ad10 con 0x7fd634112cb0 2024-05-10T04:03:27.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.228+0000 7fd6337fe700 1 --2- 172.21.15.150:0/915239386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634112cb0 0x7fd63410f130 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd62c018800 tx=0x7fd62c0188e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:27.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.228+0000 7fd630ff9700 1 -- 172.21.15.150:0/915239386 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c01f040 con 0x7fd634112cb0 2024-05-10T04:03:27.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.229+0000 7fd630ff9700 1 -- 172.21.15.150:0/915239386 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62c003e10 con 0x7fd634112cb0 2024-05-10T04:03:27.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.229+0000 7fd630ff9700 1 -- 172.21.15.150:0/915239386 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c02c880 con 0x7fd634112cb0 2024-05-10T04:03:27.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.229+0000 7fd6396dd700 1 -- 172.21.15.150:0/915239386 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63406b070 con 0x7fd634112cb0 2024-05-10T04:03:27.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.229+0000 7fd6396dd700 1 -- 172.21.15.150:0/915239386 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63406b4c0 con 0x7fd634112cb0 2024-05-10T04:03:27.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.230+0000 7fd6396dd700 1 -- 172.21.15.150:0/915239386 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd634105260 con 0x7fd634112cb0 2024-05-10T04:03:27.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.231+0000 7fd630ff9700 1 -- 172.21.15.150:0/915239386 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd62c0056c0 con 0x7fd634112cb0 2024-05-10T04:03:27.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.231+0000 7fd630ff9700 1 --2- 172.21.15.150:0/915239386 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd6200a11b0 0x7fd6200a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:27.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.232+0000 7fd630ff9700 1 -- 172.21.15.150:0/915239386 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd62c0def10 con 0x7fd634112cb0 2024-05-10T04:03:27.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.232+0000 7fd632ffd700 1 --2- 172.21.15.150:0/915239386 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd6200a11b0 0x7fd6200a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:27.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.232+0000 7fd632ffd700 1 --2- 172.21.15.150:0/915239386 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd6200a11b0 0x7fd6200a36c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd62400ba30 tx=0x7fd62400b3f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:27.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.237+0000 7fd630ff9700 1 -- 172.21.15.150:0/915239386 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd62c0a83e0 con 0x7fd634112cb0 2024-05-10T04:03:27.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.459+0000 7fd6396dd700 1 -- 172.21.15.150:0/915239386 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd634074cf0 con 0x7fd634112cb0 2024-05-10T04:03:27.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.460+0000 7fd630ff9700 1 -- 172.21.15.150:0/915239386 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd62c0aba00 con 0x7fd634112cb0 2024-05-10T04:03:27.462 INFO:teuthology.orchestra.run.smithi150.stdout:188978561062 2024-05-10T04:03:27.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.462+0000 7fd6396dd700 1 -- 172.21.15.150:0/915239386 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd6200a11b0 msgr2=0x7fd6200a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:27.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.462+0000 7fd6396dd700 1 --2- 172.21.15.150:0/915239386 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd6200a11b0 0x7fd6200a36c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd62400ba30 tx=0x7fd62400b3f0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.462+0000 7fd6396dd700 1 -- 172.21.15.150:0/915239386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634112cb0 msgr2=0x7fd63410f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:27.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.462+0000 7fd6396dd700 1 --2- 172.21.15.150:0/915239386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634112cb0 0x7fd63410f130 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd62c018800 tx=0x7fd62c0188e0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.463+0000 7fd6396dd700 1 -- 172.21.15.150:0/915239386 shutdown_connections 2024-05-10T04:03:27.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.463+0000 7fd6396dd700 1 --2- 172.21.15.150:0/915239386 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd6200a11b0 0x7fd6200a36c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.463+0000 7fd6396dd700 1 --2- 172.21.15.150:0/915239386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd634112cb0 0x7fd63410f130 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.463+0000 7fd6396dd700 1 --2- 172.21.15.150:0/915239386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6340735b0 0x7fd63410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.463+0000 7fd6396dd700 1 --2- 172.21.15.150:0/915239386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd634073f00 0x7fd63410ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.463+0000 7fd6396dd700 1 -- 172.21.15.150:0/915239386 >> 172.21.15.150:0/915239386 conn(0x7fd6340fd9c0 msgr2=0x7fd6340fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:27.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.463+0000 7fd6396dd700 1 -- 172.21.15.150:0/915239386 shutdown_connections 2024-05-10T04:03:27.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.463+0000 7fd6396dd700 1 -- 172.21.15.150:0/915239386 wait complete. 2024-05-10T04:03:27.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.272+0000 7f7c5b20d700 1 -- 172.21.15.150:0/914789275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c5406af70 msgr2=0x7f7c5406b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:27.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.272+0000 7f7c5b20d700 1 --2- 172.21.15.150:0/914789275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c5406af70 0x7f7c5406b380 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c4801a9a0 tx=0x7f7c4801acb0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.272+0000 7f7c5b20d700 1 -- 172.21.15.150:0/914789275 shutdown_connections 2024-05-10T04:03:27.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.272+0000 7f7c5b20d700 1 --2- 172.21.15.150:0/914789275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c5411b4a0 0x7f7c5411d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.272+0000 7f7c5b20d700 1 --2- 172.21.15.150:0/914789275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c5406af70 0x7f7c5406b380 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.272+0000 7f7c5b20d700 1 --2- 172.21.15.150:0/914789275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c5406b8c0 0x7f7c5411af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.272+0000 7f7c5b20d700 1 -- 172.21.15.150:0/914789275 >> 172.21.15.150:0/914789275 conn(0x7f7c540712c0 msgr2=0x7f7c540716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:27.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.272+0000 7f7c5b20d700 1 -- 172.21.15.150:0/914789275 shutdown_connections 2024-05-10T04:03:27.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.272+0000 7f7c5b20d700 1 -- 172.21.15.150:0/914789275 wait complete. 2024-05-10T04:03:27.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.273+0000 7f7c5b20d700 1 Processor -- start 2024-05-10T04:03:27.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.273+0000 7f7c5b20d700 1 -- start start 2024-05-10T04:03:27.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.273+0000 7f7c5b20d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c5406af70 0x7f7c5411ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:27.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.273+0000 7f7c5b20d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c5406b8c0 0x7f7c54116d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:27.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.273+0000 7f7c5b20d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c5411b4a0 0x7f7c54117290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:27.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.273+0000 7f7c5b20d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c541179a0 con 0x7f7c5406b8c0 2024-05-10T04:03:27.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.273+0000 7f7c5b20d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c54117b10 con 0x7f7c5406af70 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.273+0000 7f7c5b20d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c541ab100 con 0x7f7c5411b4a0 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.273+0000 7f7c53fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c5406b8c0 0x7f7c54116d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.273+0000 7f7c53fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c5406b8c0 0x7f7c54116d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52664/0 (socket says 172.21.15.150:52664) 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.273+0000 7f7c53fff700 1 -- 172.21.15.150:0/3563593927 learned_addr learned my addr 172.21.15.150:0/3563593927 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.274+0000 7f7c58fa9700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c5406af70 0x7f7c5411ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.274+0000 7f7c597aa700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c5411b4a0 0x7f7c54117290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.274+0000 7f7c53fff700 1 -- 172.21.15.150:0/3563593927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c5411b4a0 msgr2=0x7f7c54117290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.274+0000 7f7c53fff700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c5411b4a0 0x7f7c54117290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.274+0000 7f7c53fff700 1 -- 172.21.15.150:0/3563593927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c5406af70 msgr2=0x7f7c5411ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.274+0000 7f7c53fff700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c5406af70 0x7f7c5411ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.274+0000 7f7c53fff700 1 -- 172.21.15.150:0/3563593927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4801a600 con 0x7f7c5406b8c0 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.274+0000 7f7c53fff700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c5406b8c0 0x7f7c54116d50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000e9d0 tx=0x7f7c4000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.274+0000 7f7c51ffb700 1 -- 172.21.15.150:0/3563593927 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c40009790 con 0x7f7c5406b8c0 2024-05-10T04:03:27.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.274+0000 7f7c51ffb700 1 -- 172.21.15.150:0/3563593927 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c4000cbd0 con 0x7f7c5406b8c0 2024-05-10T04:03:27.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.274+0000 7f7c5b20d700 1 -- 172.21.15.150:0/3563593927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c541ab360 con 0x7f7c5406b8c0 2024-05-10T04:03:27.534 INFO:teuthology.orchestra.run.smithi150.stdout:210453397540 2024-05-10T04:03:27.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.274+0000 7f7c51ffb700 1 -- 172.21.15.150:0/3563593927 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c40018680 con 0x7f7c5406b8c0 2024-05-10T04:03:27.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.275+0000 7f7c5b20d700 1 -- 172.21.15.150:0/3563593927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c541ab860 con 0x7f7c5406b8c0 2024-05-10T04:03:27.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.276+0000 7f7c5b20d700 1 -- 172.21.15.150:0/3563593927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5410bbf0 con 0x7f7c5406b8c0 2024-05-10T04:03:27.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.277+0000 7f7c51ffb700 1 -- 172.21.15.150:0/3563593927 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7c40010430 con 0x7f7c5406b8c0 2024-05-10T04:03:27.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.278+0000 7f7c51ffb700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7c440a11b0 0x7f7c440a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.278+0000 7f7c58fa9700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7c440a11b0 0x7f7c440a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.278+0000 7f7c51ffb700 1 -- 172.21.15.150:0/3563593927 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7c40014070 con 0x7f7c5406b8c0 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.279+0000 7f7c58fa9700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7c440a11b0 0x7f7c440a36c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7c48003800 tx=0x7f7c48005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.283+0000 7f7c51ffb700 1 -- 172.21.15.150:0/3563593927 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7c40052d60 con 0x7f7c5406b8c0 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.501+0000 7f7c5b20d700 1 -- 172.21.15.150:0/3563593927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7c5404edc0 con 0x7f7c5406b8c0 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.501+0000 7f7c51ffb700 1 -- 172.21.15.150:0/3563593927 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c40099e90 con 0x7f7c5406b8c0 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.504+0000 7f7c5b20d700 1 -- 172.21.15.150:0/3563593927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7c440a11b0 msgr2=0x7f7c440a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.504+0000 7f7c5b20d700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7c440a11b0 0x7f7c440a36c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7c48003800 tx=0x7f7c48005fb0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.504+0000 7f7c5b20d700 1 -- 172.21.15.150:0/3563593927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c5406b8c0 msgr2=0x7f7c54116d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.504+0000 7f7c5b20d700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c5406b8c0 0x7f7c54116d50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000e9d0 tx=0x7f7c4000ed90 comp rx=0 tx=0).stop 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.504+0000 7f7c5b20d700 1 -- 172.21.15.150:0/3563593927 shutdown_connections 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.504+0000 7f7c5b20d700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7c440a11b0 0x7f7c440a36c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.504+0000 7f7c5b20d700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c5411b4a0 0x7f7c54117290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.504+0000 7f7c5b20d700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c5406af70 0x7f7c5411ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.504+0000 7f7c5b20d700 1 --2- 172.21.15.150:0/3563593927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c5406b8c0 0x7f7c54116d50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:27.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.504+0000 7f7c5b20d700 1 -- 172.21.15.150:0/3563593927 >> 172.21.15.150:0/3563593927 conn(0x7f7c540712c0 msgr2=0x7f7c5410c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:27.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.504+0000 7f7c5b20d700 1 -- 172.21.15.150:0/3563593927 shutdown_connections 2024-05-10T04:03:27.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:27.504+0000 7f7c5b20d700 1 -- 172.21.15.150:0/3563593927 wait complete. 2024-05-10T04:03:28.180 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574903 got 55834574905 for osd.1 2024-05-10T04:03:28.181 DEBUG:teuthology.parallel:result is None 2024-05-10T04:03:28.278 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:28 smithi150 ceph-mon[42477]: pgmap v254: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T04:03:28.278 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:28 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1529221672' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T04:03:28.278 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:28 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/915239386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T04:03:28.279 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:28 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3563593927' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T04:03:28.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.427+0000 7ffb6533a700 1 -- 172.21.15.150:0/3182673949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb6010c210 msgr2=0x7ffb6010e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:28.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.427+0000 7ffb6533a700 1 --2- 172.21.15.150:0/3182673949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb6010c210 0x7ffb6010e720 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800b0a0 tx=0x7ffb5800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:28.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.427+0000 7ffb6533a700 1 -- 172.21.15.150:0/3182673949 shutdown_connections 2024-05-10T04:03:28.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.427+0000 7ffb6533a700 1 --2- 172.21.15.150:0/3182673949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb6010c210 0x7ffb6010e720 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:28.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.427+0000 7ffb6533a700 1 --2- 172.21.15.150:0/3182673949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb60102da0 0x7ffb601031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:28.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.427+0000 7ffb6533a700 1 --2- 172.21.15.150:0/3182673949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb601036f0 0x7ffb6010bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:28.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.427+0000 7ffb6533a700 1 -- 172.21.15.150:0/3182673949 >> 172.21.15.150:0/3182673949 conn(0x7ffb600fc7a0 msgr2=0x7ffb600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:28.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.428+0000 7ffb6533a700 1 -- 172.21.15.150:0/3182673949 shutdown_connections 2024-05-10T04:03:28.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.428+0000 7ffb6533a700 1 -- 172.21.15.150:0/3182673949 wait complete. 2024-05-10T04:03:28.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.428+0000 7ffb6533a700 1 Processor -- start 2024-05-10T04:03:28.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.428+0000 7ffb6533a700 1 -- start start 2024-05-10T04:03:28.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb6533a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb60102da0 0x7ffb6019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:28.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb6533a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb601036f0 0x7ffb6019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:28.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb6533a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb6010c210 0x7ffb60198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:28.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb6533a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb60198b70 con 0x7ffb601036f0 2024-05-10T04:03:28.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb6533a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb60198ce0 con 0x7ffb60102da0 2024-05-10T04:03:28.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb6533a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb60198ea0 con 0x7ffb6010c210 2024-05-10T04:03:28.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb5e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb601036f0 0x7ffb6019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb5e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb601036f0 0x7ffb6019eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52670/0 (socket says 172.21.15.150:52670) 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb5e7fc700 1 -- 172.21.15.150:0/878466381 learned_addr learned my addr 172.21.15.150:0/878466381 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb5effd700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb60102da0 0x7ffb6019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb5f7fe700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb6010c210 0x7ffb60198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb5e7fc700 1 -- 172.21.15.150:0/878466381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb6010c210 msgr2=0x7ffb60198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb5e7fc700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb6010c210 0x7ffb60198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb5e7fc700 1 -- 172.21.15.150:0/878466381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb60102da0 msgr2=0x7ffb6019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb5e7fc700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb60102da0 0x7ffb6019e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb5e7fc700 1 -- 172.21.15.150:0/878466381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58009d00 con 0x7ffb601036f0 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb5f7fe700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb6010c210 0x7ffb60198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb5e7fc700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb601036f0 0x7ffb6019eb10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000d980 tx=0x7ffb5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.429+0000 7ffb47fff700 1 -- 172.21.15.150:0/878466381 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50009930 con 0x7ffb601036f0 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.430+0000 7ffb47fff700 1 -- 172.21.15.150:0/878466381 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb50010460 con 0x7ffb601036f0 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.430+0000 7ffb6533a700 1 -- 172.21.15.150:0/878466381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb60199180 con 0x7ffb601036f0 2024-05-10T04:03:28.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.430+0000 7ffb47fff700 1 -- 172.21.15.150:0/878466381 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5000f620 con 0x7ffb601036f0 2024-05-10T04:03:28.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.430+0000 7ffb6533a700 1 -- 172.21.15.150:0/878466381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb601a55e0 con 0x7ffb601036f0 2024-05-10T04:03:28.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.431+0000 7ffb6533a700 1 -- 172.21.15.150:0/878466381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb601072f0 con 0x7ffb601036f0 2024-05-10T04:03:28.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.432+0000 7ffb47fff700 1 -- 172.21.15.150:0/878466381 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ffb500108d0 con 0x7ffb601036f0 2024-05-10T04:03:28.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.433+0000 7ffb47fff700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffb480a11b0 0x7ffb480a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:28.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.433+0000 7ffb5effd700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffb480a11b0 0x7ffb480a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:28.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.433+0000 7ffb47fff700 1 -- 172.21.15.150:0/878466381 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ffb500cc6e0 con 0x7ffb601036f0 2024-05-10T04:03:28.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.433+0000 7ffb5effd700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffb480a11b0 0x7ffb480a36c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffb601998a0 tx=0x7ffb54009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:28.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.436+0000 7ffb47fff700 1 -- 172.21.15.150:0/878466381 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffb50095ac0 con 0x7ffb601036f0 2024-05-10T04:03:28.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:28 smithi181 ceph-mon[46602]: pgmap v254: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T04:03:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:28 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1529221672' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T04:03:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:28 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/915239386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T04:03:28.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:28 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3563593927' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T04:03:28.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:28 smithi161 ceph-mon[46918]: pgmap v254: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T04:03:28.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:28 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1529221672' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T04:03:28.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:28 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/915239386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T04:03:28.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:28 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3563593927' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T04:03:28.506 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561060 got 188978561062 for osd.6 2024-05-10T04:03:28.506 DEBUG:teuthology.parallel:result is None 2024-05-10T04:03:28.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.642+0000 7ffb6533a700 1 -- 172.21.15.150:0/878466381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7ffb60199c90 con 0x7ffb601036f0 2024-05-10T04:03:28.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.642+0000 7ffb47fff700 1 -- 172.21.15.150:0/878466381 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7ffb50017020 con 0x7ffb601036f0 2024-05-10T04:03:28.645 INFO:teuthology.orchestra.run.smithi150.stdout:292057776153 2024-05-10T04:03:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.644+0000 7ffb6533a700 1 -- 172.21.15.150:0/878466381 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffb480a11b0 msgr2=0x7ffb480a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.644+0000 7ffb6533a700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffb480a11b0 0x7ffb480a36c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffb601998a0 tx=0x7ffb54009190 comp rx=0 tx=0).stop 2024-05-10T04:03:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.644+0000 7ffb6533a700 1 -- 172.21.15.150:0/878466381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb601036f0 msgr2=0x7ffb6019eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.644+0000 7ffb6533a700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb601036f0 0x7ffb6019eb10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000d980 tx=0x7ffb5000dc90 comp rx=0 tx=0).stop 2024-05-10T04:03:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.645+0000 7ffb6533a700 1 -- 172.21.15.150:0/878466381 shutdown_connections 2024-05-10T04:03:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.645+0000 7ffb6533a700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffb480a11b0 0x7ffb480a36c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.645+0000 7ffb6533a700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb6010c210 0x7ffb60198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.645+0000 7ffb6533a700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb60102da0 0x7ffb6019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.645+0000 7ffb6533a700 1 --2- 172.21.15.150:0/878466381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb601036f0 0x7ffb6019eb10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:28.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.645+0000 7ffb6533a700 1 -- 172.21.15.150:0/878466381 >> 172.21.15.150:0/878466381 conn(0x7ffb600fc7a0 msgr2=0x7ffb600fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:28.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.645+0000 7ffb6533a700 1 -- 172.21.15.150:0/878466381 shutdown_connections 2024-05-10T04:03:28.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:28.645+0000 7ffb6533a700 1 -- 172.21.15.150:0/878466381 wait complete. 2024-05-10T04:03:28.928 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397538 got 210453397540 for osd.7 2024-05-10T04:03:28.928 DEBUG:teuthology.parallel:result is None 2024-05-10T04:03:29.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.058+0000 7fdd3f145700 1 -- 172.21.15.150:0/633086382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd38106860 msgr2=0x7fdd38106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:29.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.058+0000 7fdd3f145700 1 --2- 172.21.15.150:0/633086382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd38106860 0x7fdd38106c70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009990 tx=0x7fdd24009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:29.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.058+0000 7fdd3f145700 1 -- 172.21.15.150:0/633086382 shutdown_connections 2024-05-10T04:03:29.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.058+0000 7fdd3f145700 1 --2- 172.21.15.150:0/633086382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd381071b0 0x7fdd3810eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:29.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.058+0000 7fdd3f145700 1 --2- 172.21.15.150:0/633086382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd38101f30 0x7fdd38102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:29.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.058+0000 7fdd3f145700 1 --2- 172.21.15.150:0/633086382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd38106860 0x7fdd38106c70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:29.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.058+0000 7fdd3f145700 1 -- 172.21.15.150:0/633086382 >> 172.21.15.150:0/633086382 conn(0x7fdd380fd9c0 msgr2=0x7fdd380ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:29.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.058+0000 7fdd3f145700 1 -- 172.21.15.150:0/633086382 shutdown_connections 2024-05-10T04:03:29.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.058+0000 7fdd3f145700 1 -- 172.21.15.150:0/633086382 wait complete. 2024-05-10T04:03:29.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.059+0000 7fdd3f145700 1 Processor -- start 2024-05-10T04:03:29.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.059+0000 7fdd3f145700 1 -- start start 2024-05-10T04:03:29.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.059+0000 7fdd3f145700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd38101f30 0x7fdd3819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:29.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.059+0000 7fdd3f145700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd38106860 0x7fdd3819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:29.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.059+0000 7fdd3f145700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd381071b0 0x7fdd381983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:29.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.059+0000 7fdd3f145700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd381989a0 con 0x7fdd38106860 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.059+0000 7fdd3f145700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd38198b10 con 0x7fdd38101f30 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.059+0000 7fdd3f145700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd38198cd0 con 0x7fdd381071b0 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.059+0000 7fdd37fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd38106860 0x7fdd3819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.059+0000 7fdd37fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd38106860 0x7fdd3819e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52688/0 (socket says 172.21.15.150:52688) 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.059+0000 7fdd37fff700 1 -- 172.21.15.150:0/2345792313 learned_addr learned my addr 172.21.15.150:0/2345792313 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.059+0000 7fdd3d6e2700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd381071b0 0x7fdd381983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.060+0000 7fdd3cee1700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd38101f30 0x7fdd3819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.060+0000 7fdd37fff700 1 -- 172.21.15.150:0/2345792313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd381071b0 msgr2=0x7fdd381983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.060+0000 7fdd37fff700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd381071b0 0x7fdd381983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.060+0000 7fdd37fff700 1 -- 172.21.15.150:0/2345792313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd38101f30 msgr2=0x7fdd3819e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.060+0000 7fdd37fff700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd38101f30 0x7fdd3819e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.060+0000 7fdd37fff700 1 -- 172.21.15.150:0/2345792313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd240095f0 con 0x7fdd38106860 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.060+0000 7fdd3cee1700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd38101f30 0x7fdd3819e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.060+0000 7fdd37fff700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd38106860 0x7fdd3819e9b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdd240098e0 tx=0x7fdd2400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:29.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.060+0000 7fdd35ffb700 1 -- 172.21.15.150:0/2345792313 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2401d070 con 0x7fdd38106860 2024-05-10T04:03:29.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.060+0000 7fdd35ffb700 1 -- 172.21.15.150:0/2345792313 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd24022470 con 0x7fdd38106860 2024-05-10T04:03:29.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.060+0000 7fdd3f145700 1 -- 172.21.15.150:0/2345792313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd38198f50 con 0x7fdd38106860 2024-05-10T04:03:29.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.060+0000 7fdd35ffb700 1 -- 172.21.15.150:0/2345792313 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24017650 con 0x7fdd38106860 2024-05-10T04:03:29.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.060+0000 7fdd3f145700 1 -- 172.21.15.150:0/2345792313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd38199220 con 0x7fdd38106860 2024-05-10T04:03:29.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.061+0000 7fdd3f145700 1 -- 172.21.15.150:0/2345792313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3810a260 con 0x7fdd38106860 2024-05-10T04:03:29.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.062+0000 7fdd35ffb700 1 -- 172.21.15.150:0/2345792313 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fdd240225e0 con 0x7fdd38106860 2024-05-10T04:03:29.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.063+0000 7fdd35ffb700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdd280a11b0 0x7fdd280a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:29.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.063+0000 7fdd3cee1700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdd280a11b0 0x7fdd280a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:29.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.063+0000 7fdd35ffb700 1 -- 172.21.15.150:0/2345792313 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdd240cdca0 con 0x7fdd38106860 2024-05-10T04:03:29.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.064+0000 7fdd3cee1700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdd280a11b0 0x7fdd280a36c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c005d90 tx=0x7fdd2c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:29.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.069+0000 7fdd35ffb700 1 -- 172.21.15.150:0/2345792313 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdd24097170 con 0x7fdd38106860 2024-05-10T04:03:29.154 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776150 got 292057776153 for osd.10 2024-05-10T04:03:29.155 DEBUG:teuthology.parallel:result is None 2024-05-10T04:03:29.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.282+0000 7fdd3f145700 1 -- 172.21.15.150:0/2345792313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdd381033e0 con 0x7fdd38106860 2024-05-10T04:03:29.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.282+0000 7fdd35ffb700 1 -- 172.21.15.150:0/2345792313 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdd2409a790 con 0x7fdd38106860 2024-05-10T04:03:29.284 INFO:teuthology.orchestra.run.smithi150.stdout:77309411382 2024-05-10T04:03:29.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.285+0000 7fdd3f145700 1 -- 172.21.15.150:0/2345792313 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdd280a11b0 msgr2=0x7fdd280a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:29.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.285+0000 7fdd3f145700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdd280a11b0 0x7fdd280a36c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c005d90 tx=0x7fdd2c005d00 comp rx=0 tx=0).stop 2024-05-10T04:03:29.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.285+0000 7fdd3f145700 1 -- 172.21.15.150:0/2345792313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd38106860 msgr2=0x7fdd3819e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:29.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.285+0000 7fdd3f145700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd38106860 0x7fdd3819e9b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdd240098e0 tx=0x7fdd2400f670 comp rx=0 tx=0).stop 2024-05-10T04:03:29.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.285+0000 7fdd3f145700 1 -- 172.21.15.150:0/2345792313 shutdown_connections 2024-05-10T04:03:29.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.285+0000 7fdd3f145700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdd280a11b0 0x7fdd280a36c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:29.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.285+0000 7fdd3f145700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd381071b0 0x7fdd381983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:29.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.285+0000 7fdd3f145700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd38101f30 0x7fdd3819e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:29.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.285+0000 7fdd3f145700 1 --2- 172.21.15.150:0/2345792313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd38106860 0x7fdd3819e9b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:29.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.285+0000 7fdd3f145700 1 -- 172.21.15.150:0/2345792313 >> 172.21.15.150:0/2345792313 conn(0x7fdd380fd9c0 msgr2=0x7fdd38108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:29.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.285+0000 7fdd3f145700 1 -- 172.21.15.150:0/2345792313 shutdown_connections 2024-05-10T04:03:29.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:29.285+0000 7fdd3f145700 1 -- 172.21.15.150:0/2345792313 wait complete. 2024-05-10T04:03:29.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:29 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/878466381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T04:03:29.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-10T04:03:29.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-10T04:03:29.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-10T04:03:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-10T04:03:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-10T04:03:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-10T04:03:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-10T04:03:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-05-10T04:03:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-10T04:03:29.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:29 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-10T04:03:29.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:29 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/878466381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T04:03:29.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-10T04:03:29.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-10T04:03:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-10T04:03:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-10T04:03:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-10T04:03:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-10T04:03:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-10T04:03:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-05-10T04:03:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-10T04:03:29.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:29 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-10T04:03:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:29 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/878466381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T04:03:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-10T04:03:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-10T04:03:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-10T04:03:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-10T04:03:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-10T04:03:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-10T04:03:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-10T04:03:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-05-10T04:03:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-10T04:03:29.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:29 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-10T04:03:29.702 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411379 got 77309411382 for osd.2 2024-05-10T04:03:29.702 DEBUG:teuthology.parallel:result is None 2024-05-10T04:03:29.703 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-10T04:03:29.703 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T04:03:30.097 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: pgmap v255: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T04:03:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-10T04:03:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-10T04:03:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-10T04:03:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-10T04:03:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-10T04:03:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-10T04:03:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-10T04:03:30.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-05-10T04:03:30.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-10T04:03:30.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-10T04:03:30.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T04:03:30.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2345792313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T04:03:30.465 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:30 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:03:30.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: pgmap v255: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T04:03:30.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-10T04:03:30.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-10T04:03:30.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-10T04:03:30.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-10T04:03:30.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-10T04:03:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-10T04:03:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-10T04:03:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-05-10T04:03:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-10T04:03:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-10T04:03:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T04:03:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2345792313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T04:03:30.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:30 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:03:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: pgmap v255: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T04:03:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-10T04:03:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-10T04:03:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-10T04:03:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-10T04:03:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-10T04:03:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-10T04:03:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-10T04:03:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-05-10T04:03:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-10T04:03:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-10T04:03:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T04:03:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2345792313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T04:03:30.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:30 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:03:31.198 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:31 smithi150 ceph-mon[42477]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T04:03:31.198 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:31 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:31.198 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:31 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:31.199 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:31 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:31.199 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:31 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:31.199 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:31 smithi150 ceph-mon[42477]: pgmap v258: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T04:03:31.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.393+0000 7fda4b562700 1 -- 172.21.15.150:0/2363331006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda44106c70 msgr2=0x7fda44107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:31.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.393+0000 7fda4b562700 1 --2- 172.21.15.150:0/2363331006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda44106c70 0x7fda44107080 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fda40009990 tx=0x7fda40009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:31.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.393+0000 7fda4b562700 1 -- 172.21.15.150:0/2363331006 shutdown_connections 2024-05-10T04:03:31.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.393+0000 7fda4b562700 1 --2- 172.21.15.150:0/2363331006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda44100a50 0x7fda4410eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:31.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.393+0000 7fda4b562700 1 --2- 172.21.15.150:0/2363331006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda44106c70 0x7fda44107080 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:31.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.393+0000 7fda4b562700 1 --2- 172.21.15.150:0/2363331006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda44100100 0x7fda44100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:31.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.393+0000 7fda4b562700 1 -- 172.21.15.150:0/2363331006 >> 172.21.15.150:0/2363331006 conn(0x7fda440fbe70 msgr2=0x7fda440fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:31.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.393+0000 7fda4b562700 1 -- 172.21.15.150:0/2363331006 shutdown_connections 2024-05-10T04:03:31.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.393+0000 7fda4b562700 1 -- 172.21.15.150:0/2363331006 wait complete. 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.393+0000 7fda4b562700 1 Processor -- start 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda4b562700 1 -- start start 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda4b562700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda44100100 0x7fda441012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda4b562700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda44100a50 0x7fda44101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda4b562700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda44106c70 0x7fda441a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda4b562700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda44102090 con 0x7fda44106c70 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda4b562700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda441a6d30 con 0x7fda44100a50 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda4b562700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda441a6e70 con 0x7fda44100100 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda49aff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda44106c70 0x7fda441a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda49aff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda44106c70 0x7fda441a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41860/0 (socket says 172.21.15.150:41860) 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda49aff700 1 -- 172.21.15.150:0/4145681670 learned_addr learned my addr 172.21.15.150:0/4145681670 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda492fe700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda44100100 0x7fda441012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda48afd700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda44100a50 0x7fda44101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda49aff700 1 -- 172.21.15.150:0/4145681670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda44100100 msgr2=0x7fda441012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:31.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda49aff700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda44100100 0x7fda441012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:31.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda49aff700 1 -- 172.21.15.150:0/4145681670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda44100a50 msgr2=0x7fda44101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:31.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda49aff700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda44100a50 0x7fda44101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:31.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda49aff700 1 -- 172.21.15.150:0/4145681670 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda400095f0 con 0x7fda44106c70 2024-05-10T04:03:31.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.394+0000 7fda49aff700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda44106c70 0x7fda441a67f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00ada0 tx=0x7fda3c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:31.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.395+0000 7fda3a7fc700 1 -- 172.21.15.150:0/4145681670 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c00f070 con 0x7fda44106c70 2024-05-10T04:03:31.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.395+0000 7fda3a7fc700 1 -- 172.21.15.150:0/4145681670 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda3c007b30 con 0x7fda44106c70 2024-05-10T04:03:31.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.395+0000 7fda4b562700 1 -- 172.21.15.150:0/4145681670 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda441a7100 con 0x7fda44106c70 2024-05-10T04:03:31.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.395+0000 7fda3a7fc700 1 -- 172.21.15.150:0/4145681670 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c005280 con 0x7fda44106c70 2024-05-10T04:03:31.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.395+0000 7fda4b562700 1 -- 172.21.15.150:0/4145681670 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda441a75b0 con 0x7fda44106c70 2024-05-10T04:03:31.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.396+0000 7fda4b562700 1 -- 172.21.15.150:0/4145681670 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4410a230 con 0x7fda44106c70 2024-05-10T04:03:31.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.398+0000 7fda3a7fc700 1 -- 172.21.15.150:0/4145681670 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fda3c00a090 con 0x7fda44106c70 2024-05-10T04:03:31.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.398+0000 7fda3a7fc700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fda300a11b0 0x7fda300a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:31.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.399+0000 7fda492fe700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fda300a11b0 0x7fda300a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:31.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.399+0000 7fda3a7fc700 1 -- 172.21.15.150:0/4145681670 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fda3c0cc8e0 con 0x7fda44106c70 2024-05-10T04:03:31.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.399+0000 7fda492fe700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fda300a11b0 0x7fda300a36c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fda4000b2e0 tx=0x7fda4001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:31.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.403+0000 7fda3a7fc700 1 -- 172.21.15.150:0/4145681670 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fda3c099720 con 0x7fda44106c70 2024-05-10T04:03:31.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:31 smithi181 ceph-mon[46602]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T04:03:31.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:31 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:31.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:31 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:31.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:31 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:31.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:31 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:31.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:31 smithi181 ceph-mon[46602]: pgmap v258: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T04:03:31.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:31 smithi161 ceph-mon[46918]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T04:03:31.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:31 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:31.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:31 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:31.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:31 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:31.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:31 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:31.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:31 smithi161 ceph-mon[46918]: pgmap v258: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T04:03:31.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.604+0000 7fda4b562700 1 -- 172.21.15.150:0/4145681670 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fda44102a90 con 0x7fda300a11b0 2024-05-10T04:03:31.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.616+0000 7fda3a7fc700 1 -- 172.21.15.150:0/4145681670 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423640 (secure 0 0 0) 0x7fda44102a90 con 0x7fda300a11b0 2024-05-10T04:03:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:31.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.620+0000 7fda4b562700 1 -- 172.21.15.150:0/4145681670 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fda300a11b0 msgr2=0x7fda300a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:31.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.620+0000 7fda4b562700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fda300a11b0 0x7fda300a36c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fda4000b2e0 tx=0x7fda4001a040 comp rx=0 tx=0).stop 2024-05-10T04:03:31.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.620+0000 7fda4b562700 1 -- 172.21.15.150:0/4145681670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda44106c70 msgr2=0x7fda441a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:31.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.620+0000 7fda4b562700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda44106c70 0x7fda441a67f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00ada0 tx=0x7fda3c00c330 comp rx=0 tx=0).stop 2024-05-10T04:03:31.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.620+0000 7fda4b562700 1 -- 172.21.15.150:0/4145681670 shutdown_connections 2024-05-10T04:03:31.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.620+0000 7fda4b562700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fda300a11b0 0x7fda300a36c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:31.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.620+0000 7fda4b562700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda44106c70 0x7fda441a67f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:31.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.620+0000 7fda4b562700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda44100100 0x7fda441012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:31.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.620+0000 7fda4b562700 1 --2- 172.21.15.150:0/4145681670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda44100a50 0x7fda44101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:31.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.620+0000 7fda4b562700 1 -- 172.21.15.150:0/4145681670 >> 172.21.15.150:0/4145681670 conn(0x7fda440fbe70 msgr2=0x7fda44104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:31.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.620+0000 7fda4b562700 1 -- 172.21.15.150:0/4145681670 shutdown_connections 2024-05-10T04:03:31.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:31.620+0000 7fda4b562700 1 -- 172.21.15.150:0/4145681670 wait complete. 2024-05-10T04:03:31.629 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:03:31.994 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-05-10T04:03:31.095620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656633,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":345,"num_read_kb":291,"num_write":337,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372416,"kb_used_data":11352,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569408,"statfs":{"total":1151940427776,"available":1151559073792,"internally_reserved":0,"allocated":11624448,"data_stored":7515861,"data_compressed":260431,"data_compressed_allocated":356352,"data_compressed_original":2211840,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002368"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640492+0000","last_change":"2024-05-10T04:02:31.642089+0000","last_active":"2024-05-10T04:02:35.640492+0000","last_peered":"2024-05-10T04:02:35.640492+0000","last_clean":"2024-05-10T04:02:35.640492+0000","last_became_active":"2024-05-10T04:02:31.641080+0000","last_became_peered":"2024-05-10T04:02:31.641080+0000","last_unstale":"2024-05-10T04:02:35.640492+0000","last_undegraded":"2024-05-10T04:02:35.640492+0000","last_fullsized":"2024-05-10T04:02:35.640492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:29.693139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636966+0000","last_change":"2024-05-10T04:02:31.633056+0000","last_active":"2024-05-10T04:02:35.636966+0000","last_peered":"2024-05-10T04:02:35.636966+0000","last_clean":"2024-05-10T04:02:35.636966+0000","last_became_active":"2024-05-10T04:02:31.632725+0000","last_became_peered":"2024-05-10T04:02:31.632725+0000","last_unstale":"2024-05-10T04:02:35.636966+0000","last_undegraded":"2024-05-10T04:02:35.636966+0000","last_fullsized":"2024-05-10T04:02:35.636966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:08:40.365433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636999+0000","last_change":"2024-05-10T04:02:31.625219+0000","last_active":"2024-05-10T04:02:35.636999+0000","last_peered":"2024-05-10T04:02:35.636999+0000","last_clean":"2024-05-10T04:02:35.636999+0000","last_became_active":"2024-05-10T04:02:31.624889+0000","last_became_peered":"2024-05-10T04:02:31.624889+0000","last_unstale":"2024-05-10T04:02:35.636999+0000","last_undegraded":"2024-05-10T04:02:35.636999+0000","last_fullsized":"2024-05-10T04:02:35.636999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:10:59.081031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635079+0000","last_change":"2024-05-10T04:02:31.640651+0000","last_active":"2024-05-10T04:02:35.635079+0000","last_peered":"2024-05-10T04:02:35.635079+0000","last_clean":"2024-05-10T04:02:35.635079+0000","last_became_active":"2024-05-10T04:02:31.640242+0000","last_became_peered":"2024-05-10T04:02:31.640242+0000","last_unstale":"2024-05-10T04:02:35.635079+0000","last_undegraded":"2024-05-10T04:02:35.635079+0000","last_fullsized":"2024-05-10T04:02:35.635079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:07.562099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635834+0000","last_change":"2024-05-10T04:02:31.630688+0000","last_active":"2024-05-10T04:02:35.635834+0000","last_peered":"2024-05-10T04:02:35.635834+0000","last_clean":"2024-05-10T04:02:35.635834+0000","last_became_active":"2024-05-10T04:02:31.629252+0000","last_became_peered":"2024-05-10T04:02:31.629252+0000","last_unstale":"2024-05-10T04:02:35.635834+0000","last_undegraded":"2024-05-10T04:02:35.635834+0000","last_fullsized":"2024-05-10T04:02:35.635834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:37.647516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640499+0000","last_change":"2024-05-10T04:02:31.642586+0000","last_active":"2024-05-10T04:02:36.640499+0000","last_peered":"2024-05-10T04:02:36.640499+0000","last_clean":"2024-05-10T04:02:36.640499+0000","last_became_active":"2024-05-10T04:02:31.642244+0000","last_became_peered":"2024-05-10T04:02:31.642244+0000","last_unstale":"2024-05-10T04:02:36.640499+0000","last_undegraded":"2024-05-10T04:02:36.640499+0000","last_fullsized":"2024-05-10T04:02:36.640499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:28.020151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.638641+0000","last_change":"2024-05-10T04:02:31.636706+0000","last_active":"2024-05-10T04:02:35.638641+0000","last_peered":"2024-05-10T04:02:35.638641+0000","last_clean":"2024-05-10T04:02:35.638641+0000","last_became_active":"2024-05-10T04:02:31.636303+0000","last_became_peered":"2024-05-10T04:02:31.636303+0000","last_unstale":"2024-05-10T04:02:35.638641+0000","last_undegraded":"2024-05-10T04:02:35.638641+0000","last_fullsized":"2024-05-10T04:02:35.638641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:16.941400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198297+0000","last_change":"2024-05-10T04:02:31.640032+0000","last_active":"2024-05-10T04:03:29.198297+0000","last_peered":"2024-05-10T04:03:29.198297+0000","last_clean":"2024-05-10T04:03:29.198297+0000","last_became_active":"2024-05-10T04:02:31.639733+0000","last_became_peered":"2024-05-10T04:02:31.639733+0000","last_unstale":"2024-05-10T04:03:29.198297+0000","last_undegraded":"2024-05-10T04:03:29.198297+0000","last_fullsized":"2024-05-10T04:03:29.198297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:02:36.963414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640667+0000","last_change":"2024-05-10T04:02:31.638940+0000","last_active":"2024-05-10T04:02:36.640667+0000","last_peered":"2024-05-10T04:02:36.640667+0000","last_clean":"2024-05-10T04:02:36.640667+0000","last_became_active":"2024-05-10T04:02:31.638617+0000","last_became_peered":"2024-05-10T04:02:31.638617+0000","last_unstale":"2024-05-10T04:02:36.640667+0000","last_undegraded":"2024-05-10T04:02:36.640667+0000","last_fullsized":"2024-05-10T04:02:36.640667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:10:36.348117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.639933+0000","last_change":"2024-05-10T04:02:31.635702+0000","last_active":"2024-05-10T04:02:36.639933+0000","last_peered":"2024-05-10T04:02:36.639933+0000","last_clean":"2024-05-10T04:02:36.639933+0000","last_became_active":"2024-05-10T04:02:31.635330+0000","last_became_peered":"2024-05-10T04:02:31.635330+0000","last_unstale":"2024-05-10T04:02:36.639933+0000","last_undegraded":"2024-05-10T04:02:36.639933+0000","last_fullsized":"2024-05-10T04:02:36.639933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:21:15.160340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641553+0000","last_change":"2024-05-10T04:02:31.628252+0000","last_active":"2024-05-10T04:02:35.641553+0000","last_peered":"2024-05-10T04:02:35.641553+0000","last_clean":"2024-05-10T04:02:35.641553+0000","last_became_active":"2024-05-10T04:02:31.627394+0000","last_became_peered":"2024-05-10T04:02:31.627394+0000","last_unstale":"2024-05-10T04:02:35.641553+0000","last_undegraded":"2024-05-10T04:02:35.641553+0000","last_fullsized":"2024-05-10T04:02:35.641553+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:54.569786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635041+0000","last_change":"2024-05-10T04:02:31.641788+0000","last_active":"2024-05-10T04:02:35.635041+0000","last_peered":"2024-05-10T04:02:35.635041+0000","last_clean":"2024-05-10T04:02:35.635041+0000","last_became_active":"2024-05-10T04:02:31.641328+0000","last_became_peered":"2024-05-10T04:02:31.641328+0000","last_unstale":"2024-05-10T04:02:35.635041+0000","last_undegraded":"2024-05-10T04:02:35.635041+0000","last_fullsized":"2024-05-10T04:02:35.635041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:10:55.920245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.631051+0000","last_change":"2024-05-10T04:02:31.631919+0000","last_active":"2024-05-10T04:02:35.631051+0000","last_peered":"2024-05-10T04:02:35.631051+0000","last_clean":"2024-05-10T04:02:35.631051+0000","last_became_active":"2024-05-10T04:02:31.631592+0000","last_became_peered":"2024-05-10T04:02:31.631592+0000","last_unstale":"2024-05-10T04:02:35.631051+0000","last_undegraded":"2024-05-10T04:02:35.631051+0000","last_fullsized":"2024-05-10T04:02:35.631051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:59:03.792090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636548+0000","last_change":"2024-05-10T04:02:31.633288+0000","last_active":"2024-05-10T04:02:35.636548+0000","last_peered":"2024-05-10T04:02:35.636548+0000","last_clean":"2024-05-10T04:02:35.636548+0000","last_became_active":"2024-05-10T04:02:31.632582+0000","last_became_peered":"2024-05-10T04:02:31.632582+0000","last_unstale":"2024-05-10T04:02:35.636548+0000","last_undegraded":"2024-05-10T04:02:35.636548+0000","last_fullsized":"2024-05-10T04:02:35.636548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:27.991439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636432+0000","last_change":"2024-05-10T04:02:31.635833+0000","last_active":"2024-05-10T04:02:35.636432+0000","last_peered":"2024-05-10T04:02:35.636432+0000","last_clean":"2024-05-10T04:02:35.636432+0000","last_became_active":"2024-05-10T04:02:31.635520+0000","last_became_peered":"2024-05-10T04:02:31.635520+0000","last_unstale":"2024-05-10T04:02:35.636432+0000","last_undegraded":"2024-05-10T04:02:35.636432+0000","last_fullsized":"2024-05-10T04:02:35.636432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:40:06.312819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638804+0000","last_change":"2024-05-10T04:02:31.631734+0000","last_active":"2024-05-10T04:02:36.638804+0000","last_peered":"2024-05-10T04:02:36.638804+0000","last_clean":"2024-05-10T04:02:36.638804+0000","last_became_active":"2024-05-10T04:02:31.631375+0000","last_became_peered":"2024-05-10T04:02:31.631375+0000","last_unstale":"2024-05-10T04:02:36.638804+0000","last_undegraded":"2024-05-10T04:02:36.638804+0000","last_fullsized":"2024-05-10T04:02:36.638804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:25:13.815406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640254+0000","last_change":"2024-05-10T04:02:31.642292+0000","last_active":"2024-05-10T04:02:35.640254+0000","last_peered":"2024-05-10T04:02:35.640254+0000","last_clean":"2024-05-10T04:02:35.640254+0000","last_became_active":"2024-05-10T04:02:31.641570+0000","last_became_peered":"2024-05-10T04:02:31.641570+0000","last_unstale":"2024-05-10T04:02:35.640254+0000","last_undegraded":"2024-05-10T04:02:35.640254+0000","last_fullsized":"2024-05-10T04:02:35.640254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:16:33.704809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.646273+0000","last_change":"2024-05-10T04:02:31.625695+0000","last_active":"2024-05-10T04:02:35.646273+0000","last_peered":"2024-05-10T04:02:35.646273+0000","last_clean":"2024-05-10T04:02:35.646273+0000","last_became_active":"2024-05-10T04:02:31.625393+0000","last_became_peered":"2024-05-10T04:02:31.625393+0000","last_unstale":"2024-05-10T04:02:35.646273+0000","last_undegraded":"2024-05-10T04:02:35.646273+0000","last_fullsized":"2024-05-10T04:02:35.646273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:53.955921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.633141+0000","last_change":"2024-05-10T04:02:31.629152+0000","last_active":"2024-05-10T04:02:35.633141+0000","last_peered":"2024-05-10T04:02:35.633141+0000","last_clean":"2024-05-10T04:02:35.633141+0000","last_became_active":"2024-05-10T04:02:31.628778+0000","last_became_peered":"2024-05-10T04:02:31.628778+0000","last_unstale":"2024-05-10T04:02:35.633141+0000","last_undegraded":"2024-05-10T04:02:35.633141+0000","last_fullsized":"2024-05-10T04:02:35.633141+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:39.772500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.409997+0000","last_change":"2024-05-10T04:02:35.650536+0000","last_active":"2024-05-10T04:02:36.409997+0000","last_peered":"2024-05-10T04:02:36.409997+0000","last_clean":"2024-05-10T04:02:36.409997+0000","last_became_active":"2024-05-10T04:02:35.650102+0000","last_became_peered":"2024-05-10T04:02:35.650102+0000","last_unstale":"2024-05-10T04:02:36.409997+0000","last_undegraded":"2024-05-10T04:02:36.409997+0000","last_fullsized":"2024-05-10T04:02:36.409997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:19:35.605844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182814+0000","last_change":"2024-05-10T04:02:31.186846+0000","last_active":"2024-05-10T04:02:56.182814+0000","last_peered":"2024-05-10T04:02:56.182814+0000","last_clean":"2024-05-10T04:02:56.182814+0000","last_became_active":"2024-05-10T04:02:31.186497+0000","last_became_peered":"2024-05-10T04:02:31.186497+0000","last_unstale":"2024-05-10T04:02:56.182814+0000","last_undegraded":"2024-05-10T04:02:56.182814+0000","last_fullsized":"2024-05-10T04:02:56.182814+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:15:40.832455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.956915+0000","last_change":"2024-05-10T04:02:31.631278+0000","last_active":"2024-05-10T04:02:35.956915+0000","last_peered":"2024-05-10T04:02:35.956915+0000","last_clean":"2024-05-10T04:02:35.956915+0000","last_became_active":"2024-05-10T04:02:31.630979+0000","last_became_peered":"2024-05-10T04:02:31.630979+0000","last_unstale":"2024-05-10T04:02:35.956915+0000","last_undegraded":"2024-05-10T04:02:35.956915+0000","last_fullsized":"2024-05-10T04:02:35.956915+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:50.227115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641619+0000","last_change":"2024-05-10T04:02:35.651622+0000","last_active":"2024-05-10T04:02:36.641619+0000","last_peered":"2024-05-10T04:02:36.641619+0000","last_clean":"2024-05-10T04:02:36.641619+0000","last_became_active":"2024-05-10T04:02:35.651148+0000","last_became_peered":"2024-05-10T04:02:35.651148+0000","last_unstale":"2024-05-10T04:02:36.641619+0000","last_undegraded":"2024-05-10T04:02:36.641619+0000","last_fullsized":"2024-05-10T04:02:36.641619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:40:09.019286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199659+0000","last_change":"2024-05-10T04:02:31.187785+0000","last_active":"2024-05-10T04:03:29.199659+0000","last_peered":"2024-05-10T04:03:29.199659+0000","last_clean":"2024-05-10T04:03:29.199659+0000","last_became_active":"2024-05-10T04:02:31.187499+0000","last_became_peered":"2024-05-10T04:02:31.187499+0000","last_unstale":"2024-05-10T04:03:29.199659+0000","last_undegraded":"2024-05-10T04:03:29.199659+0000","last_fullsized":"2024-05-10T04:03:29.199659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:05:29.015767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636966+0000","last_change":"2024-05-10T04:02:31.624433+0000","last_active":"2024-05-10T04:02:35.636966+0000","last_peered":"2024-05-10T04:02:35.636966+0000","last_clean":"2024-05-10T04:02:35.636966+0000","last_became_active":"2024-05-10T04:02:31.624115+0000","last_became_peered":"2024-05-10T04:02:31.624115+0000","last_unstale":"2024-05-10T04:02:35.636966+0000","last_undegraded":"2024-05-10T04:02:35.636966+0000","last_fullsized":"2024-05-10T04:02:35.636966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:54.584681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641726+0000","last_change":"2024-05-10T04:02:35.651535+0000","last_active":"2024-05-10T04:02:36.641726+0000","last_peered":"2024-05-10T04:02:36.641726+0000","last_clean":"2024-05-10T04:02:36.641726+0000","last_became_active":"2024-05-10T04:02:35.651077+0000","last_became_peered":"2024-05-10T04:02:35.651077+0000","last_unstale":"2024-05-10T04:02:36.641726+0000","last_undegraded":"2024-05-10T04:02:36.641726+0000","last_fullsized":"2024-05-10T04:02:36.641726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:21:04.647730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.180078+0000","last_change":"2024-05-10T04:02:31.195666+0000","last_active":"2024-05-10T04:02:56.180078+0000","last_peered":"2024-05-10T04:02:56.180078+0000","last_clean":"2024-05-10T04:02:56.180078+0000","last_became_active":"2024-05-10T04:02:31.195285+0000","last_became_peered":"2024-05-10T04:02:31.195285+0000","last_unstale":"2024-05-10T04:02:56.180078+0000","last_undegraded":"2024-05-10T04:02:56.180078+0000","last_fullsized":"2024-05-10T04:02:56.180078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:02.906701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640147+0000","last_change":"2024-05-10T04:02:31.628124+0000","last_active":"2024-05-10T04:02:36.640147+0000","last_peered":"2024-05-10T04:02:36.640147+0000","last_clean":"2024-05-10T04:02:36.640147+0000","last_became_active":"2024-05-10T04:02:31.627789+0000","last_became_peered":"2024-05-10T04:02:31.627789+0000","last_unstale":"2024-05-10T04:02:36.640147+0000","last_undegraded":"2024-05-10T04:02:36.640147+0000","last_fullsized":"2024-05-10T04:02:36.640147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:40:11.240726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641791+0000","last_change":"2024-05-10T04:02:35.649536+0000","last_active":"2024-05-10T04:02:36.641791+0000","last_peered":"2024-05-10T04:02:36.641791+0000","last_clean":"2024-05-10T04:02:36.641791+0000","last_became_active":"2024-05-10T04:02:35.649130+0000","last_became_peered":"2024-05-10T04:02:35.649130+0000","last_unstale":"2024-05-10T04:02:36.641791+0000","last_undegraded":"2024-05-10T04:02:36.641791+0000","last_fullsized":"2024-05-10T04:02:36.641791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:03.870071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637193+0000","last_change":"2024-05-10T04:02:31.192627+0000","last_active":"2024-05-10T04:02:35.637193+0000","last_peered":"2024-05-10T04:02:35.637193+0000","last_clean":"2024-05-10T04:02:35.637193+0000","last_became_active":"2024-05-10T04:02:31.191433+0000","last_became_peered":"2024-05-10T04:02:31.191433+0000","last_unstale":"2024-05-10T04:02:35.637193+0000","last_undegraded":"2024-05-10T04:02:35.637193+0000","last_fullsized":"2024-05-10T04:02:35.637193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:29.235483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641519+0000","last_change":"2024-05-10T04:02:31.641192+0000","last_active":"2024-05-10T04:02:35.641519+0000","last_peered":"2024-05-10T04:02:35.641519+0000","last_clean":"2024-05-10T04:02:35.641519+0000","last_became_active":"2024-05-10T04:02:31.640540+0000","last_became_peered":"2024-05-10T04:02:31.640540+0000","last_unstale":"2024-05-10T04:02:35.641519+0000","last_undegraded":"2024-05-10T04:02:35.641519+0000","last_fullsized":"2024-05-10T04:02:35.641519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:36.734231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.410181+0000","last_change":"2024-05-10T04:02:35.644661+0000","last_active":"2024-05-10T04:02:36.410181+0000","last_peered":"2024-05-10T04:02:36.410181+0000","last_clean":"2024-05-10T04:02:36.410181+0000","last_became_active":"2024-05-10T04:02:35.644216+0000","last_became_peered":"2024-05-10T04:02:35.644216+0000","last_unstale":"2024-05-10T04:02:36.410181+0000","last_undegraded":"2024-05-10T04:02:36.410181+0000","last_fullsized":"2024-05-10T04:02:36.410181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:02.618922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:03:24.237537+0000","last_change":"2024-05-10T04:02:31.198037+0000","last_active":"2024-05-10T04:03:24.237537+0000","last_peered":"2024-05-10T04:03:24.237537+0000","last_clean":"2024-05-10T04:03:24.237537+0000","last_became_active":"2024-05-10T04:02:31.197679+0000","last_became_peered":"2024-05-10T04:02:31.197679+0000","last_unstale":"2024-05-10T04:03:24.237537+0000","last_undegraded":"2024-05-10T04:03:24.237537+0000","last_fullsized":"2024-05-10T04:03:24.237537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:19.059645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636057+0000","last_change":"2024-05-10T04:02:31.630020+0000","last_active":"2024-05-10T04:02:35.636057+0000","last_peered":"2024-05-10T04:02:35.636057+0000","last_clean":"2024-05-10T04:02:35.636057+0000","last_became_active":"2024-05-10T04:02:31.629738+0000","last_became_peered":"2024-05-10T04:02:31.629738+0000","last_unstale":"2024-05-10T04:02:35.636057+0000","last_undegraded":"2024-05-10T04:02:35.636057+0000","last_fullsized":"2024-05-10T04:02:35.636057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:58:42.317143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.410114+0000","last_change":"2024-05-10T04:02:35.647242+0000","last_active":"2024-05-10T04:02:36.410114+0000","last_peered":"2024-05-10T04:02:36.410114+0000","last_clean":"2024-05-10T04:02:36.410114+0000","last_became_active":"2024-05-10T04:02:35.646774+0000","last_became_peered":"2024-05-10T04:02:35.646774+0000","last_unstale":"2024-05-10T04:02:36.410114+0000","last_undegraded":"2024-05-10T04:02:36.410114+0000","last_fullsized":"2024-05-10T04:02:36.410114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:20:05.602820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.180856+0000","last_change":"2024-05-10T04:02:31.191981+0000","last_active":"2024-05-10T04:02:56.180856+0000","last_peered":"2024-05-10T04:02:56.180856+0000","last_clean":"2024-05-10T04:02:56.180856+0000","last_became_active":"2024-05-10T04:02:31.191632+0000","last_became_peered":"2024-05-10T04:02:31.191632+0000","last_unstale":"2024-05-10T04:02:56.180856+0000","last_undegraded":"2024-05-10T04:02:56.180856+0000","last_fullsized":"2024-05-10T04:02:56.180856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:44:52.744661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640083+0000","last_change":"2024-05-10T04:02:31.643240+0000","last_active":"2024-05-10T04:02:35.640083+0000","last_peered":"2024-05-10T04:02:35.640083+0000","last_clean":"2024-05-10T04:02:35.640083+0000","last_became_active":"2024-05-10T04:02:31.642891+0000","last_became_peered":"2024-05-10T04:02:31.642891+0000","last_unstale":"2024-05-10T04:02:35.640083+0000","last_undegraded":"2024-05-10T04:02:35.640083+0000","last_fullsized":"2024-05-10T04:02:35.640083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:41:42.901192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.639026+0000","last_change":"2024-05-10T04:02:35.652370+0000","last_active":"2024-05-10T04:02:36.639026+0000","last_peered":"2024-05-10T04:02:36.639026+0000","last_clean":"2024-05-10T04:02:36.639026+0000","last_became_active":"2024-05-10T04:02:35.651803+0000","last_became_peered":"2024-05-10T04:02:35.651803+0000","last_unstale":"2024-05-10T04:02:36.639026+0000","last_undegraded":"2024-05-10T04:02:36.639026+0000","last_fullsized":"2024-05-10T04:02:36.639026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:33:36.049311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182414+0000","last_change":"2024-05-10T04:02:31.189053+0000","last_active":"2024-05-10T04:02:56.182414+0000","last_peered":"2024-05-10T04:02:56.182414+0000","last_clean":"2024-05-10T04:02:56.182414+0000","last_became_active":"2024-05-10T04:02:31.188614+0000","last_became_peered":"2024-05-10T04:02:31.188614+0000","last_unstale":"2024-05-10T04:02:56.182414+0000","last_undegraded":"2024-05-10T04:02:56.182414+0000","last_fullsized":"2024-05-10T04:02:56.182414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:59:37.750507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198409+0000","last_change":"2024-05-10T04:02:31.630379+0000","last_active":"2024-05-10T04:03:29.198409+0000","last_peered":"2024-05-10T04:03:29.198409+0000","last_clean":"2024-05-10T04:03:29.198409+0000","last_became_active":"2024-05-10T04:02:31.630094+0000","last_became_peered":"2024-05-10T04:02:31.630094+0000","last_unstale":"2024-05-10T04:03:29.198409+0000","last_undegraded":"2024-05-10T04:03:29.198409+0000","last_fullsized":"2024-05-10T04:03:29.198409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:05.699923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638588+0000","last_change":"2024-05-10T04:02:35.650770+0000","last_active":"2024-05-10T04:02:36.638588+0000","last_peered":"2024-05-10T04:02:36.638588+0000","last_clean":"2024-05-10T04:02:36.638588+0000","last_became_active":"2024-05-10T04:02:35.650351+0000","last_became_peered":"2024-05-10T04:02:35.650351+0000","last_unstale":"2024-05-10T04:02:36.638588+0000","last_undegraded":"2024-05-10T04:02:36.638588+0000","last_fullsized":"2024-05-10T04:02:36.638588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:41:06.728126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.068369+0000","last_change":"2024-05-10T04:02:31.189557+0000","last_active":"2024-05-10T04:02:36.068369+0000","last_peered":"2024-05-10T04:02:36.068369+0000","last_clean":"2024-05-10T04:02:36.068369+0000","last_became_active":"2024-05-10T04:02:31.189206+0000","last_became_peered":"2024-05-10T04:02:31.189206+0000","last_unstale":"2024-05-10T04:02:36.068369+0000","last_undegraded":"2024-05-10T04:02:36.068369+0000","last_fullsized":"2024-05-10T04:02:36.068369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:11:08.225965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.645298+0000","last_change":"2024-05-10T04:02:31.633561+0000","last_active":"2024-05-10T04:02:35.645298+0000","last_peered":"2024-05-10T04:02:35.645298+0000","last_clean":"2024-05-10T04:02:35.645298+0000","last_became_active":"2024-05-10T04:02:31.633224+0000","last_became_peered":"2024-05-10T04:02:31.633224+0000","last_unstale":"2024-05-10T04:02:35.645298+0000","last_undegraded":"2024-05-10T04:02:35.645298+0000","last_fullsized":"2024-05-10T04:02:35.645298+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:32.901146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198675+0000","last_change":"2024-05-10T04:02:35.649327+0000","last_active":"2024-05-10T04:03:29.198675+0000","last_peered":"2024-05-10T04:03:29.198675+0000","last_clean":"2024-05-10T04:03:29.198675+0000","last_became_active":"2024-05-10T04:02:35.648982+0000","last_became_peered":"2024-05-10T04:02:35.648982+0000","last_unstale":"2024-05-10T04:03:29.198675+0000","last_undegraded":"2024-05-10T04:03:29.198675+0000","last_fullsized":"2024-05-10T04:03:29.198675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:25.405748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198995+0000","last_change":"2024-05-10T04:02:31.186447+0000","last_active":"2024-05-10T04:03:29.198995+0000","last_peered":"2024-05-10T04:03:29.198995+0000","last_clean":"2024-05-10T04:03:29.198995+0000","last_became_active":"2024-05-10T04:02:31.186073+0000","last_became_peered":"2024-05-10T04:02:31.186073+0000","last_unstale":"2024-05-10T04:03:29.198995+0000","last_undegraded":"2024-05-10T04:03:29.198995+0000","last_fullsized":"2024-05-10T04:03:29.198995+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:53.381147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640890+0000","last_change":"2024-05-10T04:02:31.633772+0000","last_active":"2024-05-10T04:02:36.640890+0000","last_peered":"2024-05-10T04:02:36.640890+0000","last_clean":"2024-05-10T04:02:36.640890+0000","last_became_active":"2024-05-10T04:02:31.629592+0000","last_became_peered":"2024-05-10T04:02:31.629592+0000","last_unstale":"2024-05-10T04:02:36.640890+0000","last_undegraded":"2024-05-10T04:02:36.640890+0000","last_fullsized":"2024-05-10T04:02:36.640890+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:38:37.342351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.088114+0000","last_change":"2024-05-10T04:02:35.646960+0000","last_active":"2024-05-10T04:02:36.088114+0000","last_peered":"2024-05-10T04:02:36.088114+0000","last_clean":"2024-05-10T04:02:36.088114+0000","last_became_active":"2024-05-10T04:02:35.646598+0000","last_became_peered":"2024-05-10T04:02:35.646598+0000","last_unstale":"2024-05-10T04:02:36.088114+0000","last_undegraded":"2024-05-10T04:02:36.088114+0000","last_fullsized":"2024-05-10T04:02:36.088114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:40:10.478745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.175169+0000","last_change":"2024-05-10T04:02:31.192846+0000","last_active":"2024-05-10T04:02:55.175169+0000","last_peered":"2024-05-10T04:02:55.175169+0000","last_clean":"2024-05-10T04:02:55.175169+0000","last_became_active":"2024-05-10T04:02:31.192514+0000","last_became_peered":"2024-05-10T04:02:31.192514+0000","last_unstale":"2024-05-10T04:02:55.175169+0000","last_undegraded":"2024-05-10T04:02:55.175169+0000","last_fullsized":"2024-05-10T04:02:55.175169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:50:32.649440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637130+0000","last_change":"2024-05-10T04:02:31.641578+0000","last_active":"2024-05-10T04:02:35.637130+0000","last_peered":"2024-05-10T04:02:35.637130+0000","last_clean":"2024-05-10T04:02:35.637130+0000","last_became_active":"2024-05-10T04:02:31.641268+0000","last_became_peered":"2024-05-10T04:02:31.641268+0000","last_unstale":"2024-05-10T04:02:35.637130+0000","last_undegraded":"2024-05-10T04:02:35.637130+0000","last_fullsized":"2024-05-10T04:02:35.637130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:31:00.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641920+0000","last_change":"2024-05-10T04:02:35.646754+0000","last_active":"2024-05-10T04:02:36.641920+0000","last_peered":"2024-05-10T04:02:36.641920+0000","last_clean":"2024-05-10T04:02:36.641920+0000","last_became_active":"2024-05-10T04:02:35.646342+0000","last_became_peered":"2024-05-10T04:02:35.646342+0000","last_unstale":"2024-05-10T04:02:36.641920+0000","last_undegraded":"2024-05-10T04:02:36.641920+0000","last_fullsized":"2024-05-10T04:02:36.641920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:28:33.373312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.174611+0000","last_change":"2024-05-10T04:02:31.186427+0000","last_active":"2024-05-10T04:02:55.174611+0000","last_peered":"2024-05-10T04:02:55.174611+0000","last_clean":"2024-05-10T04:02:55.174611+0000","last_became_active":"2024-05-10T04:02:31.185937+0000","last_became_peered":"2024-05-10T04:02:31.185937+0000","last_unstale":"2024-05-10T04:02:55.174611+0000","last_undegraded":"2024-05-10T04:02:55.174611+0000","last_fullsized":"2024-05-10T04:02:55.174611+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.818126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640331+0000","last_change":"2024-05-10T04:02:31.645582+0000","last_active":"2024-05-10T04:02:36.640331+0000","last_peered":"2024-05-10T04:02:36.640331+0000","last_clean":"2024-05-10T04:02:36.640331+0000","last_became_active":"2024-05-10T04:02:31.645259+0000","last_became_peered":"2024-05-10T04:02:31.645259+0000","last_unstale":"2024-05-10T04:02:36.640331+0000","last_undegraded":"2024-05-10T04:02:36.640331+0000","last_fullsized":"2024-05-10T04:02:36.640331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:17:31.818116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.088108+0000","last_change":"2024-05-10T04:02:35.647341+0000","last_active":"2024-05-10T04:02:36.088108+0000","last_peered":"2024-05-10T04:02:36.088108+0000","last_clean":"2024-05-10T04:02:36.088108+0000","last_became_active":"2024-05-10T04:02:35.646806+0000","last_became_peered":"2024-05-10T04:02:35.646806+0000","last_unstale":"2024-05-10T04:02:36.088108+0000","last_undegraded":"2024-05-10T04:02:36.088108+0000","last_fullsized":"2024-05-10T04:02:36.088108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:01:51.252683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'31","reported_seq":119,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.250684+0000","last_change":"2024-05-10T04:02:31.195377+0000","last_active":"2024-05-10T04:03:29.250684+0000","last_peered":"2024-05-10T04:03:29.250684+0000","last_clean":"2024-05-10T04:03:29.250684+0000","last_became_active":"2024-05-10T04:02:31.194965+0000","last_became_peered":"2024-05-10T04:02:31.194965+0000","last_unstale":"2024-05-10T04:03:29.250684+0000","last_undegraded":"2024-05-10T04:03:29.250684+0000","last_fullsized":"2024-05-10T04:03:29.250684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:11:09.237525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61731,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":73,"num_read_kb":97,"num_write":32,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636559+0000","last_change":"2024-05-10T04:02:31.629600+0000","last_active":"2024-05-10T04:02:35.636559+0000","last_peered":"2024-05-10T04:02:35.636559+0000","last_clean":"2024-05-10T04:02:35.636559+0000","last_became_active":"2024-05-10T04:02:31.629264+0000","last_became_peered":"2024-05-10T04:02:31.629264+0000","last_unstale":"2024-05-10T04:02:35.636559+0000","last_undegraded":"2024-05-10T04:02:35.636559+0000","last_fullsized":"2024-05-10T04:02:35.636559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:45:08.867783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.069212+0000","last_change":"2024-05-10T04:02:35.649468+0000","last_active":"2024-05-10T04:02:36.069212+0000","last_peered":"2024-05-10T04:02:36.069212+0000","last_clean":"2024-05-10T04:02:36.069212+0000","last_became_active":"2024-05-10T04:02:35.649053+0000","last_became_peered":"2024-05-10T04:02:35.649053+0000","last_unstale":"2024-05-10T04:02:36.069212+0000","last_undegraded":"2024-05-10T04:02:36.069212+0000","last_fullsized":"2024-05-10T04:02:36.069212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:01:05.761287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182116+0000","last_change":"2024-05-10T04:02:31.195169+0000","last_active":"2024-05-10T04:02:56.182116+0000","last_peered":"2024-05-10T04:02:56.182116+0000","last_clean":"2024-05-10T04:02:56.182116+0000","last_became_active":"2024-05-10T04:02:31.194720+0000","last_became_peered":"2024-05-10T04:02:31.194720+0000","last_unstale":"2024-05-10T04:02:56.182116+0000","last_undegraded":"2024-05-10T04:02:56.182116+0000","last_fullsized":"2024-05-10T04:02:56.182116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:42.727841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630936+0000","last_change":"2024-05-10T04:02:31.631739+0000","last_active":"2024-05-10T04:02:35.630936+0000","last_peered":"2024-05-10T04:02:35.630936+0000","last_clean":"2024-05-10T04:02:35.630936+0000","last_became_active":"2024-05-10T04:02:31.631401+0000","last_became_peered":"2024-05-10T04:02:31.631401+0000","last_unstale":"2024-05-10T04:02:35.630936+0000","last_undegraded":"2024-05-10T04:02:35.630936+0000","last_fullsized":"2024-05-10T04:02:35.630936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:06.958328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638793+0000","last_change":"2024-05-10T04:02:35.651484+0000","last_active":"2024-05-10T04:02:36.638793+0000","last_peered":"2024-05-10T04:02:36.638793+0000","last_clean":"2024-05-10T04:02:36.638793+0000","last_became_active":"2024-05-10T04:02:35.651069+0000","last_became_peered":"2024-05-10T04:02:35.651069+0000","last_unstale":"2024-05-10T04:02:36.638793+0000","last_undegraded":"2024-05-10T04:02:36.638793+0000","last_fullsized":"2024-05-10T04:02:36.638793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:26:45.229101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.172679+0000","last_change":"2024-05-10T04:02:31.186314+0000","last_active":"2024-05-10T04:02:55.172679+0000","last_peered":"2024-05-10T04:02:55.172679+0000","last_clean":"2024-05-10T04:02:55.172679+0000","last_became_active":"2024-05-10T04:02:31.185861+0000","last_became_peered":"2024-05-10T04:02:31.185861+0000","last_unstale":"2024-05-10T04:02:55.172679+0000","last_undegraded":"2024-05-10T04:02:55.172679+0000","last_fullsized":"2024-05-10T04:02:55.172679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:07.386978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640394+0000","last_change":"2024-05-10T04:02:31.637151+0000","last_active":"2024-05-10T04:02:35.640394+0000","last_peered":"2024-05-10T04:02:35.640394+0000","last_clean":"2024-05-10T04:02:35.640394+0000","last_became_active":"2024-05-10T04:02:31.636745+0000","last_became_peered":"2024-05-10T04:02:31.636745+0000","last_unstale":"2024-05-10T04:02:35.640394+0000","last_undegraded":"2024-05-10T04:02:35.640394+0000","last_fullsized":"2024-05-10T04:02:35.640394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:59.804391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.088363+0000","last_change":"2024-05-10T04:02:35.656108+0000","last_active":"2024-05-10T04:02:36.088363+0000","last_peered":"2024-05-10T04:02:36.088363+0000","last_clean":"2024-05-10T04:02:36.088363+0000","last_became_active":"2024-05-10T04:02:35.655664+0000","last_became_peered":"2024-05-10T04:02:35.655664+0000","last_unstale":"2024-05-10T04:02:36.088363+0000","last_undegraded":"2024-05-10T04:02:36.088363+0000","last_fullsized":"2024-05-10T04:02:36.088363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:10.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:03:27.241862+0000","last_change":"2024-05-10T04:02:31.189432+0000","last_active":"2024-05-10T04:03:27.241862+0000","last_peered":"2024-05-10T04:03:27.241862+0000","last_clean":"2024-05-10T04:03:27.241862+0000","last_became_active":"2024-05-10T04:02:31.189062+0000","last_became_peered":"2024-05-10T04:02:31.189062+0000","last_unstale":"2024-05-10T04:03:27.241862+0000","last_undegraded":"2024-05-10T04:03:27.241862+0000","last_fullsized":"2024-05-10T04:03:27.241862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:28.452100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.639024+0000","last_change":"2024-05-10T04:02:31.637517+0000","last_active":"2024-05-10T04:02:35.639024+0000","last_peered":"2024-05-10T04:02:35.639024+0000","last_clean":"2024-05-10T04:02:35.639024+0000","last_became_active":"2024-05-10T04:02:31.636518+0000","last_became_peered":"2024-05-10T04:02:31.636518+0000","last_unstale":"2024-05-10T04:02:35.639024+0000","last_undegraded":"2024-05-10T04:02:35.639024+0000","last_fullsized":"2024-05-10T04:02:35.639024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:07:09.732117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.067746+0000","last_change":"2024-05-10T04:02:35.649693+0000","last_active":"2024-05-10T04:02:36.067746+0000","last_peered":"2024-05-10T04:02:36.067746+0000","last_clean":"2024-05-10T04:02:36.067746+0000","last_became_active":"2024-05-10T04:02:35.649175+0000","last_became_peered":"2024-05-10T04:02:35.649175+0000","last_unstale":"2024-05-10T04:02:36.067746+0000","last_undegraded":"2024-05-10T04:02:36.067746+0000","last_fullsized":"2024-05-10T04:02:36.067746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:03:52.340444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.181556+0000","last_change":"2024-05-10T04:02:31.196909+0000","last_active":"2024-05-10T04:02:56.181556+0000","last_peered":"2024-05-10T04:02:56.181556+0000","last_clean":"2024-05-10T04:02:56.181556+0000","last_became_active":"2024-05-10T04:02:31.196599+0000","last_became_peered":"2024-05-10T04:02:31.196599+0000","last_unstale":"2024-05-10T04:02:56.181556+0000","last_undegraded":"2024-05-10T04:02:56.181556+0000","last_fullsized":"2024-05-10T04:02:56.181556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:16:51.483498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641842+0000","last_change":"2024-05-10T04:02:31.641426+0000","last_active":"2024-05-10T04:02:35.641842+0000","last_peered":"2024-05-10T04:02:35.641842+0000","last_clean":"2024-05-10T04:02:35.641842+0000","last_became_active":"2024-05-10T04:02:31.641054+0000","last_became_peered":"2024-05-10T04:02:31.641054+0000","last_unstale":"2024-05-10T04:02:35.641842+0000","last_undegraded":"2024-05-10T04:02:35.641842+0000","last_fullsized":"2024-05-10T04:02:35.641842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:12:48.472674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.068553+0000","last_change":"2024-05-10T04:02:35.653258+0000","last_active":"2024-05-10T04:02:36.068553+0000","last_peered":"2024-05-10T04:02:36.068553+0000","last_clean":"2024-05-10T04:02:36.068553+0000","last_became_active":"2024-05-10T04:02:35.652859+0000","last_became_peered":"2024-05-10T04:02:35.652859+0000","last_unstale":"2024-05-10T04:02:36.068553+0000","last_undegraded":"2024-05-10T04:02:36.068553+0000","last_fullsized":"2024-05-10T04:02:36.068553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:15:57.814963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:57.193415+0000","last_change":"2024-05-10T04:02:31.192231+0000","last_active":"2024-05-10T04:02:57.193415+0000","last_peered":"2024-05-10T04:02:57.193415+0000","last_clean":"2024-05-10T04:02:57.193415+0000","last_became_active":"2024-05-10T04:02:31.191902+0000","last_became_peered":"2024-05-10T04:02:31.191902+0000","last_unstale":"2024-05-10T04:02:57.193415+0000","last_undegraded":"2024-05-10T04:02:57.193415+0000","last_fullsized":"2024-05-10T04:02:57.193415+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:18.637112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.638702+0000","last_change":"2024-05-10T04:02:31.635514+0000","last_active":"2024-05-10T04:02:35.638702+0000","last_peered":"2024-05-10T04:02:35.638702+0000","last_clean":"2024-05-10T04:02:35.638702+0000","last_became_active":"2024-05-10T04:02:31.635171+0000","last_became_peered":"2024-05-10T04:02:31.635171+0000","last_unstale":"2024-05-10T04:02:35.638702+0000","last_undegraded":"2024-05-10T04:02:35.638702+0000","last_fullsized":"2024-05-10T04:02:35.638702+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:48.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.090937+0000","last_change":"2024-05-10T04:02:35.900174+0000","last_active":"2024-05-10T04:02:36.090937+0000","last_peered":"2024-05-10T04:02:36.090937+0000","last_clean":"2024-05-10T04:02:36.090937+0000","last_became_active":"2024-05-10T04:02:35.899653+0000","last_became_peered":"2024-05-10T04:02:35.899653+0000","last_unstale":"2024-05-10T04:02:36.090937+0000","last_undegraded":"2024-05-10T04:02:36.090937+0000","last_fullsized":"2024-05-10T04:02:36.090937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:32:56.962361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.645483+0000","last_change":"2024-05-10T04:02:31.190173+0000","last_active":"2024-05-10T04:02:35.645483+0000","last_peered":"2024-05-10T04:02:35.645483+0000","last_clean":"2024-05-10T04:02:35.645483+0000","last_became_active":"2024-05-10T04:02:31.189840+0000","last_became_peered":"2024-05-10T04:02:31.189840+0000","last_unstale":"2024-05-10T04:02:35.645483+0000","last_undegraded":"2024-05-10T04:02:35.645483+0000","last_fullsized":"2024-05-10T04:02:35.645483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:31:49.382482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.639305+0000","last_change":"2024-05-10T04:02:31.629497+0000","last_active":"2024-05-10T04:02:36.639305+0000","last_peered":"2024-05-10T04:02:36.639305+0000","last_clean":"2024-05-10T04:02:36.639305+0000","last_became_active":"2024-05-10T04:02:31.629182+0000","last_became_peered":"2024-05-10T04:02:31.629182+0000","last_unstale":"2024-05-10T04:02:36.639305+0000","last_undegraded":"2024-05-10T04:02:36.639305+0000","last_fullsized":"2024-05-10T04:02:36.639305+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:11:33.478737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.067644+0000","last_change":"2024-05-10T04:02:35.647249+0000","last_active":"2024-05-10T04:02:36.067644+0000","last_peered":"2024-05-10T04:02:36.067644+0000","last_clean":"2024-05-10T04:02:36.067644+0000","last_became_active":"2024-05-10T04:02:35.646716+0000","last_became_peered":"2024-05-10T04:02:35.646716+0000","last_unstale":"2024-05-10T04:02:36.067644+0000","last_undegraded":"2024-05-10T04:02:36.067644+0000","last_fullsized":"2024-05-10T04:02:36.067644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:44.651836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.639382+0000","last_change":"2024-05-10T04:02:31.191241+0000","last_active":"2024-05-10T04:02:36.639382+0000","last_peered":"2024-05-10T04:02:36.639382+0000","last_clean":"2024-05-10T04:02:36.639382+0000","last_became_active":"2024-05-10T04:02:31.190888+0000","last_became_peered":"2024-05-10T04:02:31.190888+0000","last_unstale":"2024-05-10T04:02:36.639382+0000","last_undegraded":"2024-05-10T04:02:36.639382+0000","last_fullsized":"2024-05-10T04:02:36.639382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:55.256119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.639659+0000","last_change":"2024-05-10T04:02:31.638394+0000","last_active":"2024-05-10T04:02:36.639659+0000","last_peered":"2024-05-10T04:02:36.639659+0000","last_clean":"2024-05-10T04:02:36.639659+0000","last_became_active":"2024-05-10T04:02:31.638025+0000","last_became_peered":"2024-05-10T04:02:31.638025+0000","last_unstale":"2024-05-10T04:02:36.639659+0000","last_undegraded":"2024-05-10T04:02:36.639659+0000","last_fullsized":"2024-05-10T04:02:36.639659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:13.088393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.067640+0000","last_change":"2024-05-10T04:02:35.899898+0000","last_active":"2024-05-10T04:02:36.067640+0000","last_peered":"2024-05-10T04:02:36.067640+0000","last_clean":"2024-05-10T04:02:36.067640+0000","last_became_active":"2024-05-10T04:02:35.899485+0000","last_became_peered":"2024-05-10T04:02:35.899485+0000","last_unstale":"2024-05-10T04:02:36.067640+0000","last_undegraded":"2024-05-10T04:02:36.067640+0000","last_fullsized":"2024-05-10T04:02:36.067640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:26:50.909122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640359+0000","last_change":"2024-05-10T04:02:31.640923+0000","last_active":"2024-05-10T04:02:35.640359+0000","last_peered":"2024-05-10T04:02:35.640359+0000","last_clean":"2024-05-10T04:02:35.640359+0000","last_became_active":"2024-05-10T04:02:31.640490+0000","last_became_peered":"2024-05-10T04:02:31.640490+0000","last_unstale":"2024-05-10T04:02:35.640359+0000","last_undegraded":"2024-05-10T04:02:35.640359+0000","last_fullsized":"2024-05-10T04:02:35.640359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:31:58.585789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:03:27.241092+0000","last_change":"2024-05-10T04:02:31.191326+0000","last_active":"2024-05-10T04:03:27.241092+0000","last_peered":"2024-05-10T04:03:27.241092+0000","last_clean":"2024-05-10T04:03:27.241092+0000","last_became_active":"2024-05-10T04:02:31.190974+0000","last_became_peered":"2024-05-10T04:02:31.190974+0000","last_unstale":"2024-05-10T04:03:27.241092+0000","last_undegraded":"2024-05-10T04:03:27.241092+0000","last_fullsized":"2024-05-10T04:03:27.241092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:43.670619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.639965+0000","last_change":"2024-05-10T04:02:35.647526+0000","last_active":"2024-05-10T04:02:36.639965+0000","last_peered":"2024-05-10T04:02:36.639965+0000","last_clean":"2024-05-10T04:02:36.639965+0000","last_became_active":"2024-05-10T04:02:35.647018+0000","last_became_peered":"2024-05-10T04:02:35.647018+0000","last_unstale":"2024-05-10T04:02:36.639965+0000","last_undegraded":"2024-05-10T04:02:36.639965+0000","last_fullsized":"2024-05-10T04:02:36.639965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:17.173497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199043+0000","last_change":"2024-05-10T04:02:31.632263+0000","last_active":"2024-05-10T04:03:29.199043+0000","last_peered":"2024-05-10T04:03:29.199043+0000","last_clean":"2024-05-10T04:03:29.199043+0000","last_became_active":"2024-05-10T04:02:31.631785+0000","last_became_peered":"2024-05-10T04:02:31.631785+0000","last_unstale":"2024-05-10T04:03:29.199043+0000","last_undegraded":"2024-05-10T04:03:29.199043+0000","last_fullsized":"2024-05-10T04:03:29.199043+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:30.585862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:57.191865+0000","last_change":"2024-05-10T04:02:31.182262+0000","last_active":"2024-05-10T04:02:57.191865+0000","last_peered":"2024-05-10T04:02:57.191865+0000","last_clean":"2024-05-10T04:02:57.191865+0000","last_became_active":"2024-05-10T04:02:31.181802+0000","last_became_peered":"2024-05-10T04:02:31.181802+0000","last_unstale":"2024-05-10T04:02:57.191865+0000","last_undegraded":"2024-05-10T04:02:57.191865+0000","last_fullsized":"2024-05-10T04:02:57.191865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:45:40.696571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":325,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640698+0000","last_change":"2024-05-10T04:01:41.866365+0000","last_active":"2024-05-10T04:02:36.640698+0000","last_peered":"2024-05-10T04:02:36.640698+0000","last_clean":"2024-05-10T04:02:36.640698+0000","last_became_active":"2024-05-10T04:01:41.865825+0000","last_became_peered":"2024-05-10T04:01:41.865825+0000","last_unstale":"2024-05-10T04:02:36.640698+0000","last_undegraded":"2024-05-10T04:02:36.640698+0000","last_fullsized":"2024-05-10T04:02:36.640698+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_clean_scrub_stamp":"2024-05-10T03:59:19.853473+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:28:29.598339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.069016+0000","last_change":"2024-05-10T04:02:35.900841+0000","last_active":"2024-05-10T04:02:36.069016+0000","last_peered":"2024-05-10T04:02:36.069016+0000","last_clean":"2024-05-10T04:02:36.069016+0000","last_became_active":"2024-05-10T04:02:35.900323+0000","last_became_peered":"2024-05-10T04:02:35.900323+0000","last_unstale":"2024-05-10T04:02:36.069016+0000","last_undegraded":"2024-05-10T04:02:36.069016+0000","last_fullsized":"2024-05-10T04:02:36.069016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:00.554804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630799+0000","last_change":"2024-05-10T04:02:31.641167+0000","last_active":"2024-05-10T04:02:35.630799+0000","last_peered":"2024-05-10T04:02:35.630799+0000","last_clean":"2024-05-10T04:02:35.630799+0000","last_became_active":"2024-05-10T04:02:31.640824+0000","last_became_peered":"2024-05-10T04:02:31.640824+0000","last_unstale":"2024-05-10T04:02:35.630799+0000","last_undegraded":"2024-05-10T04:02:35.630799+0000","last_fullsized":"2024-05-10T04:02:35.630799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:59:48.531970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.180041+0000","last_change":"2024-05-10T04:02:31.187834+0000","last_active":"2024-05-10T04:02:56.180041+0000","last_peered":"2024-05-10T04:02:56.180041+0000","last_clean":"2024-05-10T04:02:56.180041+0000","last_became_active":"2024-05-10T04:02:31.187440+0000","last_became_peered":"2024-05-10T04:02:31.187440+0000","last_unstale":"2024-05-10T04:02:56.180041+0000","last_undegraded":"2024-05-10T04:02:56.180041+0000","last_fullsized":"2024-05-10T04:02:56.180041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:50:37.678245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638789+0000","last_change":"2024-05-10T04:02:35.649664+0000","last_active":"2024-05-10T04:02:36.638789+0000","last_peered":"2024-05-10T04:02:36.638789+0000","last_clean":"2024-05-10T04:02:36.638789+0000","last_became_active":"2024-05-10T04:02:35.649076+0000","last_became_peered":"2024-05-10T04:02:35.649076+0000","last_unstale":"2024-05-10T04:02:36.638789+0000","last_undegraded":"2024-05-10T04:02:36.638789+0000","last_fullsized":"2024-05-10T04:02:36.638789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:14:42.864845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.173775+0000","last_change":"2024-05-10T04:02:31.192126+0000","last_active":"2024-05-10T04:02:55.173775+0000","last_peered":"2024-05-10T04:02:55.173775+0000","last_clean":"2024-05-10T04:02:55.173775+0000","last_became_active":"2024-05-10T04:02:31.191771+0000","last_became_peered":"2024-05-10T04:02:31.191771+0000","last_unstale":"2024-05-10T04:02:55.173775+0000","last_undegraded":"2024-05-10T04:02:55.173775+0000","last_fullsized":"2024-05-10T04:02:55.173775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:43.550535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632220+0000","last_change":"2024-05-10T04:02:31.620788+0000","last_active":"2024-05-10T04:02:35.632220+0000","last_peered":"2024-05-10T04:02:35.632220+0000","last_clean":"2024-05-10T04:02:35.632220+0000","last_became_active":"2024-05-10T04:02:31.620456+0000","last_became_peered":"2024-05-10T04:02:31.620456+0000","last_unstale":"2024-05-10T04:02:35.632220+0000","last_undegraded":"2024-05-10T04:02:35.632220+0000","last_fullsized":"2024-05-10T04:02:35.632220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:02.956030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198321+0000","last_change":"2024-05-10T04:02:35.899935+0000","last_active":"2024-05-10T04:03:29.198321+0000","last_peered":"2024-05-10T04:03:29.198321+0000","last_clean":"2024-05-10T04:03:29.198321+0000","last_became_active":"2024-05-10T04:02:35.899488+0000","last_became_peered":"2024-05-10T04:02:35.899488+0000","last_unstale":"2024-05-10T04:03:29.198321+0000","last_undegraded":"2024-05-10T04:03:29.198321+0000","last_fullsized":"2024-05-10T04:03:29.198321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:28.600463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:03:01.088621+0000","last_change":"2024-05-10T04:02:31.188605+0000","last_active":"2024-05-10T04:03:01.088621+0000","last_peered":"2024-05-10T04:03:01.088621+0000","last_clean":"2024-05-10T04:03:01.088621+0000","last_became_active":"2024-05-10T04:02:31.188283+0000","last_became_peered":"2024-05-10T04:02:31.188283+0000","last_unstale":"2024-05-10T04:03:01.088621+0000","last_undegraded":"2024-05-10T04:03:01.088621+0000","last_fullsized":"2024-05-10T04:03:01.088621+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:40:03.940269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199712+0000","last_change":"2024-05-10T04:02:31.629476+0000","last_active":"2024-05-10T04:03:29.199712+0000","last_peered":"2024-05-10T04:03:29.199712+0000","last_clean":"2024-05-10T04:03:29.199712+0000","last_became_active":"2024-05-10T04:02:31.629104+0000","last_became_peered":"2024-05-10T04:02:31.629104+0000","last_unstale":"2024-05-10T04:03:29.199712+0000","last_undegraded":"2024-05-10T04:03:29.199712+0000","last_fullsized":"2024-05-10T04:03:29.199712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:57.873755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.409730+0000","last_change":"2024-05-10T04:02:35.644094+0000","last_active":"2024-05-10T04:02:36.409730+0000","last_peered":"2024-05-10T04:02:36.409730+0000","last_clean":"2024-05-10T04:02:36.409730+0000","last_became_active":"2024-05-10T04:02:35.643382+0000","last_became_peered":"2024-05-10T04:02:35.643382+0000","last_unstale":"2024-05-10T04:02:36.409730+0000","last_undegraded":"2024-05-10T04:02:36.409730+0000","last_fullsized":"2024-05-10T04:02:36.409730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:33.250052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.175394+0000","last_change":"2024-05-10T04:02:31.189302+0000","last_active":"2024-05-10T04:02:55.175394+0000","last_peered":"2024-05-10T04:02:55.175394+0000","last_clean":"2024-05-10T04:02:55.175394+0000","last_became_active":"2024-05-10T04:02:31.188323+0000","last_became_peered":"2024-05-10T04:02:31.188323+0000","last_unstale":"2024-05-10T04:02:55.175394+0000","last_undegraded":"2024-05-10T04:02:55.175394+0000","last_fullsized":"2024-05-10T04:02:55.175394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:11:02.922884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637085+0000","last_change":"2024-05-10T04:02:31.635864+0000","last_active":"2024-05-10T04:02:35.637085+0000","last_peered":"2024-05-10T04:02:35.637085+0000","last_clean":"2024-05-10T04:02:35.637085+0000","last_became_active":"2024-05-10T04:02:31.635512+0000","last_became_peered":"2024-05-10T04:02:31.635512+0000","last_unstale":"2024-05-10T04:02:35.637085+0000","last_undegraded":"2024-05-10T04:02:35.637085+0000","last_fullsized":"2024-05-10T04:02:35.637085+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:26:10.957848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.642076+0000","last_change":"2024-05-10T04:02:35.900489+0000","last_active":"2024-05-10T04:02:36.642076+0000","last_peered":"2024-05-10T04:02:36.642076+0000","last_clean":"2024-05-10T04:02:36.642076+0000","last_became_active":"2024-05-10T04:02:35.899908+0000","last_became_peered":"2024-05-10T04:02:35.899908+0000","last_unstale":"2024-05-10T04:02:36.642076+0000","last_undegraded":"2024-05-10T04:02:36.642076+0000","last_fullsized":"2024-05-10T04:02:36.642076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:08.386128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.174482+0000","last_change":"2024-05-10T04:02:31.193956+0000","last_active":"2024-05-10T04:02:55.174482+0000","last_peered":"2024-05-10T04:02:55.174482+0000","last_clean":"2024-05-10T04:02:55.174482+0000","last_became_active":"2024-05-10T04:02:31.193574+0000","last_became_peered":"2024-05-10T04:02:31.193574+0000","last_unstale":"2024-05-10T04:02:55.174482+0000","last_undegraded":"2024-05-10T04:02:55.174482+0000","last_fullsized":"2024-05-10T04:02:55.174482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:46.369152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635522+0000","last_change":"2024-05-10T04:02:31.633263+0000","last_active":"2024-05-10T04:02:35.635522+0000","last_peered":"2024-05-10T04:02:35.635522+0000","last_clean":"2024-05-10T04:02:35.635522+0000","last_became_active":"2024-05-10T04:02:31.632903+0000","last_became_peered":"2024-05-10T04:02:31.632903+0000","last_unstale":"2024-05-10T04:02:35.635522+0000","last_undegraded":"2024-05-10T04:02:35.635522+0000","last_fullsized":"2024-05-10T04:02:35.635522+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:28.047473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.088498+0000","last_change":"2024-05-10T04:02:35.656003+0000","last_active":"2024-05-10T04:02:36.088498+0000","last_peered":"2024-05-10T04:02:36.088498+0000","last_clean":"2024-05-10T04:02:36.088498+0000","last_became_active":"2024-05-10T04:02:35.655518+0000","last_became_peered":"2024-05-10T04:02:35.655518+0000","last_unstale":"2024-05-10T04:02:36.088498+0000","last_undegraded":"2024-05-10T04:02:36.088498+0000","last_fullsized":"2024-05-10T04:02:36.088498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:49.696699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198559+0000","last_change":"2024-05-10T04:02:31.195297+0000","last_active":"2024-05-10T04:03:29.198559+0000","last_peered":"2024-05-10T04:03:29.198559+0000","last_clean":"2024-05-10T04:03:29.198559+0000","last_became_active":"2024-05-10T04:02:31.193960+0000","last_became_peered":"2024-05-10T04:02:31.193960+0000","last_unstale":"2024-05-10T04:03:29.198559+0000","last_undegraded":"2024-05-10T04:03:29.198559+0000","last_fullsized":"2024-05-10T04:03:29.198559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:04:10.314700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637944+0000","last_change":"2024-05-10T04:02:31.629247+0000","last_active":"2024-05-10T04:02:35.637944+0000","last_peered":"2024-05-10T04:02:35.637944+0000","last_clean":"2024-05-10T04:02:35.637944+0000","last_became_active":"2024-05-10T04:02:31.628838+0000","last_became_peered":"2024-05-10T04:02:31.628838+0000","last_unstale":"2024-05-10T04:02:35.637944+0000","last_undegraded":"2024-05-10T04:02:35.637944+0000","last_fullsized":"2024-05-10T04:02:35.637944+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:46:39.483264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:38.823384+0000","last_change":"2024-05-10T04:02:35.644502+0000","last_active":"2024-05-10T04:02:38.823384+0000","last_peered":"2024-05-10T04:02:38.823384+0000","last_clean":"2024-05-10T04:02:38.823384+0000","last_became_active":"2024-05-10T04:02:35.643942+0000","last_became_peered":"2024-05-10T04:02:35.643942+0000","last_unstale":"2024-05-10T04:02:38.823384+0000","last_undegraded":"2024-05-10T04:02:38.823384+0000","last_fullsized":"2024-05-10T04:02:38.823384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:22:42.394568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.409816+0000","last_change":"2024-05-10T04:02:31.193357+0000","last_active":"2024-05-10T04:02:36.409816+0000","last_peered":"2024-05-10T04:02:36.409816+0000","last_clean":"2024-05-10T04:02:36.409816+0000","last_became_active":"2024-05-10T04:02:31.190306+0000","last_became_peered":"2024-05-10T04:02:31.190306+0000","last_unstale":"2024-05-10T04:02:36.409816+0000","last_undegraded":"2024-05-10T04:02:36.409816+0000","last_fullsized":"2024-05-10T04:02:36.409816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:13.368960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198950+0000","last_change":"2024-05-10T04:02:31.628029+0000","last_active":"2024-05-10T04:03:29.198950+0000","last_peered":"2024-05-10T04:03:29.198950+0000","last_clean":"2024-05-10T04:03:29.198950+0000","last_became_active":"2024-05-10T04:02:31.627667+0000","last_became_peered":"2024-05-10T04:02:31.627667+0000","last_unstale":"2024-05-10T04:03:29.198950+0000","last_undegraded":"2024-05-10T04:03:29.198950+0000","last_fullsized":"2024-05-10T04:03:29.198950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:39:10.878894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641977+0000","last_change":"2024-05-10T04:02:35.900237+0000","last_active":"2024-05-10T04:02:36.641977+0000","last_peered":"2024-05-10T04:02:36.641977+0000","last_clean":"2024-05-10T04:02:36.641977+0000","last_became_active":"2024-05-10T04:02:35.899781+0000","last_became_peered":"2024-05-10T04:02:35.899781+0000","last_unstale":"2024-05-10T04:02:36.641977+0000","last_undegraded":"2024-05-10T04:02:36.641977+0000","last_fullsized":"2024-05-10T04:02:36.641977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:52:46.132557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.174762+0000","last_change":"2024-05-10T04:02:31.196920+0000","last_active":"2024-05-10T04:02:55.174762+0000","last_peered":"2024-05-10T04:02:55.174762+0000","last_clean":"2024-05-10T04:02:55.174762+0000","last_became_active":"2024-05-10T04:02:31.196191+0000","last_became_peered":"2024-05-10T04:02:31.196191+0000","last_unstale":"2024-05-10T04:02:55.174762+0000","last_undegraded":"2024-05-10T04:02:55.174762+0000","last_fullsized":"2024-05-10T04:02:55.174762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:45:17.980149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.639518+0000","last_change":"2024-05-10T04:02:31.623678+0000","last_active":"2024-05-10T04:02:35.639518+0000","last_peered":"2024-05-10T04:02:35.639518+0000","last_clean":"2024-05-10T04:02:35.639518+0000","last_became_active":"2024-05-10T04:02:31.623090+0000","last_became_peered":"2024-05-10T04:02:31.623090+0000","last_unstale":"2024-05-10T04:02:35.639518+0000","last_undegraded":"2024-05-10T04:02:35.639518+0000","last_fullsized":"2024-05-10T04:02:35.639518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:34.321959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640966+0000","last_change":"2024-05-10T04:02:35.648833+0000","last_active":"2024-05-10T04:02:36.640966+0000","last_peered":"2024-05-10T04:02:36.640966+0000","last_clean":"2024-05-10T04:02:36.640966+0000","last_became_active":"2024-05-10T04:02:35.648391+0000","last_became_peered":"2024-05-10T04:02:35.648391+0000","last_unstale":"2024-05-10T04:02:36.640966+0000","last_undegraded":"2024-05-10T04:02:36.640966+0000","last_fullsized":"2024-05-10T04:02:36.640966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:12.773284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182221+0000","last_change":"2024-05-10T04:02:31.192900+0000","last_active":"2024-05-10T04:02:56.182221+0000","last_peered":"2024-05-10T04:02:56.182221+0000","last_clean":"2024-05-10T04:02:56.182221+0000","last_became_active":"2024-05-10T04:02:31.192481+0000","last_became_peered":"2024-05-10T04:02:31.192481+0000","last_unstale":"2024-05-10T04:02:56.182221+0000","last_undegraded":"2024-05-10T04:02:56.182221+0000","last_fullsized":"2024-05-10T04:02:56.182221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:17.202383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641659+0000","last_change":"2024-05-10T04:02:31.628490+0000","last_active":"2024-05-10T04:02:35.641659+0000","last_peered":"2024-05-10T04:02:35.641659+0000","last_clean":"2024-05-10T04:02:35.641659+0000","last_became_active":"2024-05-10T04:02:31.627889+0000","last_became_peered":"2024-05-10T04:02:31.627889+0000","last_unstale":"2024-05-10T04:02:35.641659+0000","last_undegraded":"2024-05-10T04:02:35.641659+0000","last_fullsized":"2024-05-10T04:02:35.641659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:32:55.366759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638326+0000","last_change":"2024-05-10T04:02:35.653530+0000","last_active":"2024-05-10T04:02:36.638326+0000","last_peered":"2024-05-10T04:02:36.638326+0000","last_clean":"2024-05-10T04:02:36.638326+0000","last_became_active":"2024-05-10T04:02:35.653095+0000","last_became_peered":"2024-05-10T04:02:35.653095+0000","last_unstale":"2024-05-10T04:02:36.638326+0000","last_undegraded":"2024-05-10T04:02:36.638326+0000","last_fullsized":"2024-05-10T04:02:36.638326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:09.921713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.068914+0000","last_change":"2024-05-10T04:02:31.189434+0000","last_active":"2024-05-10T04:02:36.068914+0000","last_peered":"2024-05-10T04:02:36.068914+0000","last_clean":"2024-05-10T04:02:36.068914+0000","last_became_active":"2024-05-10T04:02:31.188843+0000","last_became_peered":"2024-05-10T04:02:31.188843+0000","last_unstale":"2024-05-10T04:02:36.068914+0000","last_undegraded":"2024-05-10T04:02:36.068914+0000","last_fullsized":"2024-05-10T04:02:36.068914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:31.508794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640750+0000","last_change":"2024-05-10T04:02:31.636775+0000","last_active":"2024-05-10T04:02:35.640750+0000","last_peered":"2024-05-10T04:02:35.640750+0000","last_clean":"2024-05-10T04:02:35.640750+0000","last_became_active":"2024-05-10T04:02:31.635946+0000","last_became_peered":"2024-05-10T04:02:31.635946+0000","last_unstale":"2024-05-10T04:02:35.640750+0000","last_undegraded":"2024-05-10T04:02:35.640750+0000","last_fullsized":"2024-05-10T04:02:35.640750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:48.830097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.068189+0000","last_change":"2024-05-10T04:02:35.646991+0000","last_active":"2024-05-10T04:02:36.068189+0000","last_peered":"2024-05-10T04:02:36.068189+0000","last_clean":"2024-05-10T04:02:36.068189+0000","last_became_active":"2024-05-10T04:02:35.646585+0000","last_became_peered":"2024-05-10T04:02:35.646585+0000","last_unstale":"2024-05-10T04:02:36.068189+0000","last_undegraded":"2024-05-10T04:02:36.068189+0000","last_fullsized":"2024-05-10T04:02:36.068189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:32:50.329746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.631077+0000","last_change":"2024-05-10T04:02:31.192103+0000","last_active":"2024-05-10T04:02:35.631077+0000","last_peered":"2024-05-10T04:02:35.631077+0000","last_clean":"2024-05-10T04:02:35.631077+0000","last_became_active":"2024-05-10T04:02:31.191378+0000","last_became_peered":"2024-05-10T04:02:31.191378+0000","last_unstale":"2024-05-10T04:02:35.631077+0000","last_undegraded":"2024-05-10T04:02:35.631077+0000","last_fullsized":"2024-05-10T04:02:35.631077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:55.298158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641428+0000","last_change":"2024-05-10T04:02:31.628587+0000","last_active":"2024-05-10T04:02:35.641428+0000","last_peered":"2024-05-10T04:02:35.641428+0000","last_clean":"2024-05-10T04:02:35.641428+0000","last_became_active":"2024-05-10T04:02:31.627296+0000","last_became_peered":"2024-05-10T04:02:31.627296+0000","last_unstale":"2024-05-10T04:02:35.641428+0000","last_undegraded":"2024-05-10T04:02:35.641428+0000","last_fullsized":"2024-05-10T04:02:35.641428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:04:09.783266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.638054+0000","last_change":"2024-05-10T04:02:31.634668+0000","last_active":"2024-05-10T04:02:35.638054+0000","last_peered":"2024-05-10T04:02:35.638054+0000","last_clean":"2024-05-10T04:02:35.638054+0000","last_became_active":"2024-05-10T04:02:31.634321+0000","last_became_peered":"2024-05-10T04:02:31.634321+0000","last_unstale":"2024-05-10T04:02:35.638054+0000","last_undegraded":"2024-05-10T04:02:35.638054+0000","last_fullsized":"2024-05-10T04:02:35.638054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:16:44.833804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.633396+0000","last_change":"2024-05-10T04:02:31.646002+0000","last_active":"2024-05-10T04:02:35.633396+0000","last_peered":"2024-05-10T04:02:35.633396+0000","last_clean":"2024-05-10T04:02:35.633396+0000","last_became_active":"2024-05-10T04:02:31.645673+0000","last_became_peered":"2024-05-10T04:02:31.645673+0000","last_unstale":"2024-05-10T04:02:35.633396+0000","last_undegraded":"2024-05-10T04:02:35.633396+0000","last_fullsized":"2024-05-10T04:02:35.633396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:26:26.814523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632295+0000","last_change":"2024-05-10T04:02:31.630606+0000","last_active":"2024-05-10T04:02:35.632295+0000","last_peered":"2024-05-10T04:02:35.632295+0000","last_clean":"2024-05-10T04:02:35.632295+0000","last_became_active":"2024-05-10T04:02:31.630313+0000","last_became_peered":"2024-05-10T04:02:31.630313+0000","last_unstale":"2024-05-10T04:02:35.632295+0000","last_undegraded":"2024-05-10T04:02:35.632295+0000","last_fullsized":"2024-05-10T04:02:35.632295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:39.876016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636526+0000","last_change":"2024-05-10T04:02:31.636239+0000","last_active":"2024-05-10T04:02:35.636526+0000","last_peered":"2024-05-10T04:02:35.636526+0000","last_clean":"2024-05-10T04:02:35.636526+0000","last_became_active":"2024-05-10T04:02:31.635377+0000","last_became_peered":"2024-05-10T04:02:31.635377+0000","last_unstale":"2024-05-10T04:02:35.636526+0000","last_undegraded":"2024-05-10T04:02:35.636526+0000","last_fullsized":"2024-05-10T04:02:35.636526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:02.612285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632276+0000","last_change":"2024-05-10T04:02:31.643231+0000","last_active":"2024-05-10T04:02:35.632276+0000","last_peered":"2024-05-10T04:02:35.632276+0000","last_clean":"2024-05-10T04:02:35.632276+0000","last_became_active":"2024-05-10T04:02:31.642797+0000","last_became_peered":"2024-05-10T04:02:31.642797+0000","last_unstale":"2024-05-10T04:02:35.632276+0000","last_undegraded":"2024-05-10T04:02:35.632276+0000","last_fullsized":"2024-05-10T04:02:35.632276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:01:57.365339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.638745+0000","last_change":"2024-05-10T04:02:31.633928+0000","last_active":"2024-05-10T04:02:35.638745+0000","last_peered":"2024-05-10T04:02:35.638745+0000","last_clean":"2024-05-10T04:02:35.638745+0000","last_became_active":"2024-05-10T04:02:31.633423+0000","last_became_peered":"2024-05-10T04:02:31.633423+0000","last_unstale":"2024-05-10T04:02:35.638745+0000","last_undegraded":"2024-05-10T04:02:35.638745+0000","last_fullsized":"2024-05-10T04:02:35.638745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:07.011354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632757+0000","last_change":"2024-05-10T04:02:31.627151+0000","last_active":"2024-05-10T04:02:35.632757+0000","last_peered":"2024-05-10T04:02:35.632757+0000","last_clean":"2024-05-10T04:02:35.632757+0000","last_became_active":"2024-05-10T04:02:31.626780+0000","last_became_peered":"2024-05-10T04:02:31.626780+0000","last_unstale":"2024-05-10T04:02:35.632757+0000","last_undegraded":"2024-05-10T04:02:35.632757+0000","last_fullsized":"2024-05-10T04:02:35.632757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:32.070530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199643+0000","last_change":"2024-05-10T04:02:31.640669+0000","last_active":"2024-05-10T04:03:29.199643+0000","last_peered":"2024-05-10T04:03:29.199643+0000","last_clean":"2024-05-10T04:03:29.199643+0000","last_became_active":"2024-05-10T04:02:31.640410+0000","last_became_peered":"2024-05-10T04:02:31.640410+0000","last_unstale":"2024-05-10T04:03:29.199643+0000","last_undegraded":"2024-05-10T04:03:29.199643+0000","last_fullsized":"2024-05-10T04:03:29.199643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:02.235391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.640243+0000","last_change":"2024-05-10T04:02:31.633624+0000","last_active":"2024-05-10T04:02:35.640243+0000","last_peered":"2024-05-10T04:02:35.640243+0000","last_clean":"2024-05-10T04:02:35.640243+0000","last_became_active":"2024-05-10T04:02:31.633241+0000","last_became_peered":"2024-05-10T04:02:31.633241+0000","last_unstale":"2024-05-10T04:02:35.640243+0000","last_undegraded":"2024-05-10T04:02:35.640243+0000","last_fullsized":"2024-05-10T04:02:35.640243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:58.750912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198385+0000","last_change":"2024-05-10T04:02:31.628632+0000","last_active":"2024-05-10T04:03:29.198385+0000","last_peered":"2024-05-10T04:03:29.198385+0000","last_clean":"2024-05-10T04:03:29.198385+0000","last_became_active":"2024-05-10T04:02:31.628349+0000","last_became_peered":"2024-05-10T04:02:31.628349+0000","last_unstale":"2024-05-10T04:03:29.198385+0000","last_undegraded":"2024-05-10T04:03:29.198385+0000","last_fullsized":"2024-05-10T04:03:29.198385+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:32.575168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641204+0000","last_change":"2024-05-10T04:02:31.631328+0000","last_active":"2024-05-10T04:02:36.641204+0000","last_peered":"2024-05-10T04:02:36.641204+0000","last_clean":"2024-05-10T04:02:36.641204+0000","last_became_active":"2024-05-10T04:02:31.630957+0000","last_became_peered":"2024-05-10T04:02:31.630957+0000","last_unstale":"2024-05-10T04:02:36.641204+0000","last_undegraded":"2024-05-10T04:02:36.641204+0000","last_fullsized":"2024-05-10T04:02:36.641204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:40.400911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630956+0000","last_change":"2024-05-10T04:02:31.617972+0000","last_active":"2024-05-10T04:02:35.630956+0000","last_peered":"2024-05-10T04:02:35.630956+0000","last_clean":"2024-05-10T04:02:35.630956+0000","last_became_active":"2024-05-10T04:02:31.617663+0000","last_became_peered":"2024-05-10T04:02:31.617663+0000","last_unstale":"2024-05-10T04:02:35.630956+0000","last_undegraded":"2024-05-10T04:02:35.630956+0000","last_fullsized":"2024-05-10T04:02:35.630956+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:50.121955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.641183+0000","last_change":"2024-05-10T04:02:31.644256+0000","last_active":"2024-05-10T04:02:35.641183+0000","last_peered":"2024-05-10T04:02:35.641183+0000","last_clean":"2024-05-10T04:02:35.641183+0000","last_became_active":"2024-05-10T04:02:31.643819+0000","last_became_peered":"2024-05-10T04:02:31.643819+0000","last_unstale":"2024-05-10T04:02:35.641183+0000","last_undegraded":"2024-05-10T04:02:35.641183+0000","last_fullsized":"2024-05-10T04:02:35.641183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:54.932547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279970,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66031,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":219,"num_read_kb":182,"num_write":154,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":198093,"data_compressed":4764,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514518,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":925696,"data_stored":461863,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.73299999999999998}]},{"osd":1,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.497}]},{"osd":2,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.753},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56399999999999995}]},{"osd":5,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.82399999999999995}]},{"osd":6,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.874},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71699999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":9,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.95599999999999996}]},{"osd":10,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.83999999999999997}]}]},{"osd":10,"up_from":68,"seq":292057776154,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":1060864,"data_stored":1110926,"data_compressed":48861,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":1,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.45100000000000001}]},{"osd":5,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76700000000000002}]},{"osd":6,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":8,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.747},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.66500000000000004}]},{"osd":9,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.53100000000000003}]},{"osd":11,"last update":"Fri May 10 04:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.80200000000000005}]}]},{"osd":9,"up_from":64,"seq":274877906973,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":946176,"data_stored":466195,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.435,"15min":0.443},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.46500000000000002,"15min":0.47599999999999998},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.89100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.314}]},{"osd":1,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80700000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.434},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.61499999999999999,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71399999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.623},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.69899999999999995}]},{"osd":3,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.34799999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.52100000000000002,"15min":0.54300000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.58799999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.33600000000000002}]},{"osd":4,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.67100000000000004,"15min":0.68400000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.751,"5min":1.149,"15min":1.149},"last":0.50900000000000001}]},{"osd":6,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.1699999999999999}]},{"osd":7,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.74299999999999999,"15min":0.76300000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2470000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.752,"15min":0.78000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88100000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.59299999999999997}]},{"osd":8,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.53000000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.749}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.56299999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]},{"osd":8,"up_from":56,"seq":240518168609,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":942080,"data_stored":462019,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":0.83499999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54300000000000004,"15min":0.54800000000000004},"min":{"1min":0.40699999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.65000000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90000000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82099999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.88300000000000001}]},{"osd":2,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48399999999999999,"15min":0.47299999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.47599999999999998},"min":{"1min":0.32700000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.84899999999999998}]},{"osd":4,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.66200000000000003,"15min":0.64600000000000002},"min":{"1min":0.47199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96699999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":1.0920000000000001}]},{"osd":5,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.41299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.82199999999999995}]},{"osd":6,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90000000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.92800000000000005}]},{"osd":7,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.49199999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.55300000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.94199999999999995}]},{"osd":9,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":1.042}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":1.02},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.135}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397540,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":991232,"data_stored":1058227,"data_compressed":47273,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.32100000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.65200000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.45800000000000002,"15min":0.45300000000000001},"min":{"1min":0.30499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.5}]},{"osd":2,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.93999999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.73999999999999999}]},{"osd":3,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.443,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84799999999999998,"5min":1.143,"15min":1.143},"last":0.63},{"interface":"front","average":{"1min":0.629,"5min":0.63,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80900000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76000000000000001}]},{"osd":4,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82199999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.53300000000000003}]},{"osd":5,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95799999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.69599999999999995,"15min":0.67400000000000004},"min":{"1min":0.438,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.5630000000000002,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.79000000000000004}]},{"osd":6,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.54600000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.88200000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.71999999999999997}]},{"osd":8,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.204,"15min":1.286},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88900000000000001,"5min":15.521000000000001,"15min":15.521000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":1.2829999999999999,"15min":1.3660000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0049999999999999,"5min":16.306999999999999,"15min":16.306999999999999},"last":0.77600000000000002}]},{"osd":9,"last update":"Fri May 10 04:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0509999999999999,"5min":1.167,"15min":1.167},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.753,"15min":0.751},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.81499999999999995}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561063,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":921600,"data_stored":457275,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.38800000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.38900000000000001},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.377}]},{"osd":1,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.48399999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71299999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.754,"5min":1.087,"15min":1.087},"last":0.624}]},{"osd":3,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.47499999999999998,"15min":0.46700000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.47599999999999998}]},{"osd":4,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88500000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.496}]},{"osd":5,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.63},"min":{"1min":0.44,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93700000000000006,"5min":1.018,"15min":1.018},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri May 10 04:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83299999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.67800000000000005,"15min":0.67300000000000004},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86399999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.57499999999999996}]},{"osd":8,"last update":"Fri May 10 04:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.55000000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.59199999999999997}]},{"osd":10,"last update":"Fri May 10 04:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68000000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":995328,"data_stored":528426,"data_compressed":16207,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80700000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56100000000000005,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.58199999999999996,"15min":0.53700000000000003},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.56299999999999994,"15min":0.52300000000000002},"min":{"1min":0.41599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri May 10 04:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.53800000000000003,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.54800000000000004,"15min":0.53100000000000003},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.442}]},{"osd":4,"last update":"Fri May 10 04:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.46800000000000003,"15min":0.45300000000000001},"min":{"1min":0.33900000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65300000000000002,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.54600000000000004,"15min":0.51600000000000001},"min":{"1min":0.35099999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri May 10 04:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.62},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.56499999999999995,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.496}]},{"osd":6,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.63200000000000001,"15min":0.61499999999999999},"min":{"1min":0.55500000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.56899999999999995}]},{"osd":7,"last update":"Fri May 10 04:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.58899999999999997,"15min":0.57399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.37},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.433,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.47399999999999998}]},{"osd":8,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69999999999999996,"15min":0.68799999999999994},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.55100000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.46300000000000002}]},{"osd":9,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.73099999999999998}]},{"osd":10,"last update":"Fri May 10 04:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.69099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":458663,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68100000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.71999999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.62,"15min":0.55000000000000004},"min":{"1min":0.443,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.9009999999999998,"5min":4.9009999999999998,"15min":4.9009999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82499999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69899999999999995}]},{"osd":3,"last update":"Fri May 10 04:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40799999999999997},"min":{"1min":0.26000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.58599999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.82699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri May 10 04:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75800000000000001,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72599999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.627}]},{"osd":5,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79700000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.64800000000000002,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri May 10 04:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54300000000000004}]},{"osd":7,"last update":"Fri May 10 04:01:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.748},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.84599999999999997}]},{"osd":8,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.67000000000000004,"15min":0.67800000000000005},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85099999999999998,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86299999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.80200000000000005}]},{"osd":9,"last update":"Fri May 10 04:02:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.83499999999999996}]},{"osd":10,"last update":"Fri May 10 04:02:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.91200000000000003}]},{"osd":11,"last update":"Fri May 10 04:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.76400000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411382,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":1007616,"data_stored":520611,"data_compressed":13252,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.54200000000000004,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.63},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56799999999999995,"15min":0.53500000000000003},"min":{"1min":0.41799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.59799999999999998,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri May 10 04:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.47899999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.44400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.78900000000000003}]},{"osd":4,"last update":"Fri May 10 04:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.41799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.318,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Fri May 10 04:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.51700000000000002,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79000000000000004,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.40799999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.83099999999999996}]},{"osd":6,"last update":"Fri May 10 04:02:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.44400000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.84899999999999998}]},{"osd":7,"last update":"Fri May 10 04:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.89700000000000002}]},{"osd":8,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.92900000000000005}]},{"osd":9,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.74299999999999999,"15min":0.72299999999999998},"min":{"1min":0.66800000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0189999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":1.01},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.59199999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.91900000000000004,"5min":1.016,"15min":1.016},"last":0.95599999999999996}]},{"osd":10,"last update":"Fri May 10 04:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.96699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":476247,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.35399999999999998},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.46400000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.437},{"interface":"front","average":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36099999999999999},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.41499999999999998}]},{"osd":1,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.505,"15min":0.51100000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.59999999999999998,"5min":0.747,"15min":0.747},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69799999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69899999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78000000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri May 10 04:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.70399999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri May 10 04:03:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80100000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.59199999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78800000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri May 10 04:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri May 10 04:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.80500000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68899999999999995}]},{"osd":8,"last update":"Fri May 10 04:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.439}]},{"osd":9,"last update":"Fri May 10 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69599999999999995,"15min":0.71199999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89900000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83999999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55200000000000005}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.755}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953519,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":462270,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.33900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50900000000000001,"15min":0.502},"min":{"1min":0.371,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.79000000000000004}]},{"osd":1,"last update":"Fri May 10 04:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.39000000000000001},"min":{"1min":0.29599999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.66400000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.499},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.40899999999999997,"15min":0.38800000000000001},"min":{"1min":0.31900000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.65200000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri May 10 04:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60399999999999998,"15min":0.56399999999999995},"min":{"1min":0.46600000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.95099999999999996,"5min":1.046,"15min":1.046},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75900000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.628}]},{"osd":3,"last update":"Fri May 10 04:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56999999999999995,"15min":0.54100000000000004},"min":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.47499999999999998}]},{"osd":5,"last update":"Fri May 10 04:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.46400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.92300000000000004,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81699999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri May 10 04:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.47799999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70899999999999996,"15min":0.69999999999999996},"min":{"1min":0.503,"5min":0.439,"15min":0.439},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96499999999999997,"5min":1.01,"15min":1.01},"last":0.69099999999999995}]},{"osd":8,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68100000000000005,"15min":0.68500000000000005},"min":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.79900000000000004,"5min":0.879,"15min":0.879},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.56899999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.81499999999999995}]},{"osd":9,"last update":"Fri May 10 04:03:26 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.80000000000000004,"15min":0.76700000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.6449999999999996,"5min":5.6449999999999996,"15min":5.6449999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.82899999999999996},"min":{"1min":0.54500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":5.6100000000000003,"5min":5.6100000000000003,"15min":5.6100000000000003},"last":0.54500000000000004}]},{"osd":10,"last update":"Fri May 10 04:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.78200000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]},{"osd":5,"up_from":37,"seq":158913789994,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":995328,"data_stored":1053139,"data_compressed":47273,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52400000000000002,"15min":0.498},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77800000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.433,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74299999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57499999999999996,"15min":0.55100000000000005},"min":{"1min":0.45400000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.47999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.40400000000000003},"min":{"1min":0.35799999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.495},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.44,"15min":0.42199999999999999},"min":{"1min":0.37,"5min":0.249,"15min":0.249},"max":{"1min":0.66100000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80500000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.74299999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61099999999999999,"15min":0.59199999999999997},"min":{"1min":0.48199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83399999999999996}]},{"osd":6,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.439,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.48199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.626}]},{"osd":7,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.69099999999999995,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.54400000000000004,"5min":0.505,"15min":0.505},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.85399999999999998}]},{"osd":8,"last update":"Fri May 10 04:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91900000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.46000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.873},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.55000000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri May 10 04:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.80400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61731,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63426,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63654,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:03:32.003 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T04:03:32.226 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:32.647 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:32.647 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:32.647 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:32 smithi150 ceph-mon[42477]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:03:32.647 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:03:32.648 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:03:32.648 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:32.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:32 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:32.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:32 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:32.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:32 smithi181 ceph-mon[46602]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:03:32.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:32 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:03:32.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:32 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:03:32.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:32 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:32.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:32 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:32.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:32 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:32.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:32 smithi161 ceph-mon[46918]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:03:32.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:32 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:03:32.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:32 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:03:32.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:32 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:03:33.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.208+0000 7fb0de64e700 1 -- 172.21.15.150:0/2109797620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d8105730 msgr2=0x7fb0d8105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:33.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.208+0000 7fb0de64e700 1 --2- 172.21.15.150:0/2109797620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d8105730 0x7fb0d8105b40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009990 tx=0x7fb0cc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:33.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.208+0000 7fb0de64e700 1 -- 172.21.15.150:0/2109797620 shutdown_connections 2024-05-10T04:03:33.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.208+0000 7fb0de64e700 1 --2- 172.21.15.150:0/2109797620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d8106080 0x7fb0d810d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:33.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.208+0000 7fb0de64e700 1 --2- 172.21.15.150:0/2109797620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0d8100e00 0x7fb0d8101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:33.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.208+0000 7fb0de64e700 1 --2- 172.21.15.150:0/2109797620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d8105730 0x7fb0d8105b40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:33.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.208+0000 7fb0de64e700 1 -- 172.21.15.150:0/2109797620 >> 172.21.15.150:0/2109797620 conn(0x7fb0d80fc7a0 msgr2=0x7fb0d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:33.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.208+0000 7fb0de64e700 1 -- 172.21.15.150:0/2109797620 shutdown_connections 2024-05-10T04:03:33.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.208+0000 7fb0de64e700 1 -- 172.21.15.150:0/2109797620 wait complete. 2024-05-10T04:03:33.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.208+0000 7fb0de64e700 1 Processor -- start 2024-05-10T04:03:33.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.208+0000 7fb0de64e700 1 -- start start 2024-05-10T04:03:33.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0de64e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0d8100e00 0x7fb0d8197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:33.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0de64e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d8105730 0x7fb0d81979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:33.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0de64e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d8106080 0x7fb0d819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:33.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0de64e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d819c420 con 0x7fb0d8106080 2024-05-10T04:03:33.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0de64e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d819c590 con 0x7fb0d8105730 2024-05-10T04:03:33.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0de64e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d819c750 con 0x7fb0d8100e00 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0dcbeb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d8106080 0x7fb0d819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0dcbeb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d8106080 0x7fb0d819beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41874/0 (socket says 172.21.15.150:41874) 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0dcbeb700 1 -- 172.21.15.150:0/2490218426 learned_addr learned my addr 172.21.15.150:0/2490218426 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0d77fe700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d8105730 0x7fb0d81979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0d7fff700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0d8100e00 0x7fb0d8197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0dcbeb700 1 -- 172.21.15.150:0/2490218426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0d8100e00 msgr2=0x7fb0d8197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0dcbeb700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0d8100e00 0x7fb0d8197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0dcbeb700 1 -- 172.21.15.150:0/2490218426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d8105730 msgr2=0x7fb0d81979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0dcbeb700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d8105730 0x7fb0d81979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.209+0000 7fb0dcbeb700 1 -- 172.21.15.150:0/2490218426 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc0095f0 con 0x7fb0d8106080 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.210+0000 7fb0dcbeb700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d8106080 0x7fb0d819beb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000ef30 tx=0x7fb0d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.210+0000 7fb0d57fa700 1 -- 172.21.15.150:0/2490218426 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0018070 con 0x7fb0d8106080 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.210+0000 7fb0de64e700 1 -- 172.21.15.150:0/2490218426 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d819c9d0 con 0x7fb0d8106080 2024-05-10T04:03:33.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.210+0000 7fb0d57fa700 1 -- 172.21.15.150:0/2490218426 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0d000f040 con 0x7fb0d8106080 2024-05-10T04:03:33.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.210+0000 7fb0d57fa700 1 -- 172.21.15.150:0/2490218426 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d000cd00 con 0x7fb0d8106080 2024-05-10T04:03:33.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.210+0000 7fb0de64e700 1 -- 172.21.15.150:0/2490218426 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d819cd00 con 0x7fb0d8106080 2024-05-10T04:03:33.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.211+0000 7fb0de64e700 1 -- 172.21.15.150:0/2490218426 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d8109130 con 0x7fb0d8106080 2024-05-10T04:03:33.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.212+0000 7fb0d57fa700 1 -- 172.21.15.150:0/2490218426 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb0d0003a00 con 0x7fb0d8106080 2024-05-10T04:03:33.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.213+0000 7fb0d57fa700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0c00a11b0 0x7fb0c00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:33.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.213+0000 7fb0d7fff700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0c00a11b0 0x7fb0c00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:33.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.213+0000 7fb0d57fa700 1 -- 172.21.15.150:0/2490218426 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb0d00cd2b0 con 0x7fb0d8106080 2024-05-10T04:03:33.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.213+0000 7fb0d7fff700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0c00a11b0 0x7fb0c00a36c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8007be0 tx=0x7fb0c80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:33.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.216+0000 7fb0d57fa700 1 -- 172.21.15.150:0/2490218426 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb0d0096600 con 0x7fb0d8106080 2024-05-10T04:03:33.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.403+0000 7fb0de64e700 1 -- 172.21.15.150:0/2490218426 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0d804edc0 con 0x7fb0c00a11b0 2024-05-10T04:03:33.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.415+0000 7fb0d57fa700 1 -- 172.21.15.150:0/2490218426 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+424263 (secure 0 0 0) 0x7fb0d804edc0 con 0x7fb0c00a11b0 2024-05-10T04:03:33.423 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:33.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.419+0000 7fb0de64e700 1 -- 172.21.15.150:0/2490218426 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0c00a11b0 msgr2=0x7fb0c00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:33.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.419+0000 7fb0de64e700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0c00a11b0 0x7fb0c00a36c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8007be0 tx=0x7fb0c80073d0 comp rx=0 tx=0).stop 2024-05-10T04:03:33.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.419+0000 7fb0de64e700 1 -- 172.21.15.150:0/2490218426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d8106080 msgr2=0x7fb0d819beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:33.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.419+0000 7fb0de64e700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d8106080 0x7fb0d819beb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000ef30 tx=0x7fb0d000c3a0 comp rx=0 tx=0).stop 2024-05-10T04:03:33.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.419+0000 7fb0de64e700 1 -- 172.21.15.150:0/2490218426 shutdown_connections 2024-05-10T04:03:33.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.419+0000 7fb0de64e700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0c00a11b0 0x7fb0c00a36c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:33.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.419+0000 7fb0de64e700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d8106080 0x7fb0d819beb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:33.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.419+0000 7fb0de64e700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0d8100e00 0x7fb0d8197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:33.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.419+0000 7fb0de64e700 1 --2- 172.21.15.150:0/2490218426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0d8105730 0x7fb0d81979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:33.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.419+0000 7fb0de64e700 1 -- 172.21.15.150:0/2490218426 >> 172.21.15.150:0/2490218426 conn(0x7fb0d80fc7a0 msgr2=0x7fb0d8106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:33.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.419+0000 7fb0de64e700 1 -- 172.21.15.150:0/2490218426 shutdown_connections 2024-05-10T04:03:33.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:33.419+0000 7fb0de64e700 1 -- 172.21.15.150:0/2490218426 wait complete. 2024-05-10T04:03:33.427 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:03:33.709 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:33 smithi150 ceph-mon[42477]: pgmap v259: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-10T04:03:33.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:33 smithi181 ceph-mon[46602]: pgmap v259: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-10T04:03:33.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:33 smithi161 ceph-mon[46918]: pgmap v259: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-10T04:03:33.802 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-05-10T04:03:33.096113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656633,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":350,"num_read_kb":291,"num_write":337,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":422,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372528,"kb_used_data":11444,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569296,"statfs":{"total":1151940427776,"available":1151558959104,"internally_reserved":0,"allocated":11718656,"data_stored":7586877,"data_compressed":284099,"data_compressed_allocated":389120,"data_compressed_original":2277376,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002226"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195663+0000","last_change":"2024-05-10T04:02:31.642089+0000","last_active":"2024-05-10T04:03:29.195663+0000","last_peered":"2024-05-10T04:03:29.195663+0000","last_clean":"2024-05-10T04:03:29.195663+0000","last_became_active":"2024-05-10T04:02:31.641080+0000","last_became_peered":"2024-05-10T04:02:31.641080+0000","last_unstale":"2024-05-10T04:03:29.195663+0000","last_undegraded":"2024-05-10T04:03:29.195663+0000","last_fullsized":"2024-05-10T04:03:29.195663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:29.693139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636966+0000","last_change":"2024-05-10T04:02:31.633056+0000","last_active":"2024-05-10T04:02:35.636966+0000","last_peered":"2024-05-10T04:02:35.636966+0000","last_clean":"2024-05-10T04:02:35.636966+0000","last_became_active":"2024-05-10T04:02:31.632725+0000","last_became_peered":"2024-05-10T04:02:31.632725+0000","last_unstale":"2024-05-10T04:02:35.636966+0000","last_undegraded":"2024-05-10T04:02:35.636966+0000","last_fullsized":"2024-05-10T04:02:35.636966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:08:40.365433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636999+0000","last_change":"2024-05-10T04:02:31.625219+0000","last_active":"2024-05-10T04:02:35.636999+0000","last_peered":"2024-05-10T04:02:35.636999+0000","last_clean":"2024-05-10T04:02:35.636999+0000","last_became_active":"2024-05-10T04:02:31.624889+0000","last_became_peered":"2024-05-10T04:02:31.624889+0000","last_unstale":"2024-05-10T04:02:35.636999+0000","last_undegraded":"2024-05-10T04:02:35.636999+0000","last_fullsized":"2024-05-10T04:02:35.636999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:10:59.081031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.199344+0000","last_change":"2024-05-10T04:03:30.198971+0000","last_active":"2024-05-10T04:03:30.199344+0000","last_peered":"2024-05-10T04:03:30.199344+0000","last_clean":"2024-05-10T04:03:30.199344+0000","last_became_active":"2024-05-10T04:03:30.198664+0000","last_became_peered":"2024-05-10T04:03:30.198664+0000","last_unstale":"2024-05-10T04:03:30.199344+0000","last_undegraded":"2024-05-10T04:03:30.199344+0000","last_fullsized":"2024-05-10T04:03:30.199344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:22.655875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635834+0000","last_change":"2024-05-10T04:02:31.630688+0000","last_active":"2024-05-10T04:02:35.635834+0000","last_peered":"2024-05-10T04:02:35.635834+0000","last_clean":"2024-05-10T04:02:35.635834+0000","last_became_active":"2024-05-10T04:02:31.629252+0000","last_became_peered":"2024-05-10T04:02:31.629252+0000","last_unstale":"2024-05-10T04:02:35.635834+0000","last_undegraded":"2024-05-10T04:02:35.635834+0000","last_fullsized":"2024-05-10T04:02:35.635834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:37.647516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640499+0000","last_change":"2024-05-10T04:02:31.642586+0000","last_active":"2024-05-10T04:02:36.640499+0000","last_peered":"2024-05-10T04:02:36.640499+0000","last_clean":"2024-05-10T04:02:36.640499+0000","last_became_active":"2024-05-10T04:02:31.642244+0000","last_became_peered":"2024-05-10T04:02:31.642244+0000","last_unstale":"2024-05-10T04:02:36.640499+0000","last_undegraded":"2024-05-10T04:02:36.640499+0000","last_fullsized":"2024-05-10T04:02:36.640499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:28.020151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191697+0000","last_change":"2024-05-10T04:02:31.636706+0000","last_active":"2024-05-10T04:03:29.191697+0000","last_peered":"2024-05-10T04:03:29.191697+0000","last_clean":"2024-05-10T04:03:29.191697+0000","last_became_active":"2024-05-10T04:02:31.636303+0000","last_became_peered":"2024-05-10T04:02:31.636303+0000","last_unstale":"2024-05-10T04:03:29.191697+0000","last_undegraded":"2024-05-10T04:03:29.191697+0000","last_fullsized":"2024-05-10T04:03:29.191697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:16.941400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198297+0000","last_change":"2024-05-10T04:02:31.640032+0000","last_active":"2024-05-10T04:03:29.198297+0000","last_peered":"2024-05-10T04:03:29.198297+0000","last_clean":"2024-05-10T04:03:29.198297+0000","last_became_active":"2024-05-10T04:02:31.639733+0000","last_became_peered":"2024-05-10T04:02:31.639733+0000","last_unstale":"2024-05-10T04:03:29.198297+0000","last_undegraded":"2024-05-10T04:03:29.198297+0000","last_fullsized":"2024-05-10T04:03:29.198297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:02:36.963414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640667+0000","last_change":"2024-05-10T04:02:31.638940+0000","last_active":"2024-05-10T04:02:36.640667+0000","last_peered":"2024-05-10T04:02:36.640667+0000","last_clean":"2024-05-10T04:02:36.640667+0000","last_became_active":"2024-05-10T04:02:31.638617+0000","last_became_peered":"2024-05-10T04:02:31.638617+0000","last_unstale":"2024-05-10T04:02:36.640667+0000","last_undegraded":"2024-05-10T04:02:36.640667+0000","last_fullsized":"2024-05-10T04:02:36.640667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:10:36.348117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.639933+0000","last_change":"2024-05-10T04:02:31.635702+0000","last_active":"2024-05-10T04:02:36.639933+0000","last_peered":"2024-05-10T04:02:36.639933+0000","last_clean":"2024-05-10T04:02:36.639933+0000","last_became_active":"2024-05-10T04:02:31.635330+0000","last_became_peered":"2024-05-10T04:02:31.635330+0000","last_unstale":"2024-05-10T04:02:36.639933+0000","last_undegraded":"2024-05-10T04:02:36.639933+0000","last_fullsized":"2024-05-10T04:02:36.639933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:21:15.160340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192388+0000","last_change":"2024-05-10T04:02:31.628252+0000","last_active":"2024-05-10T04:03:29.192388+0000","last_peered":"2024-05-10T04:03:29.192388+0000","last_clean":"2024-05-10T04:03:29.192388+0000","last_became_active":"2024-05-10T04:02:31.627394+0000","last_became_peered":"2024-05-10T04:02:31.627394+0000","last_unstale":"2024-05-10T04:03:29.192388+0000","last_undegraded":"2024-05-10T04:03:29.192388+0000","last_fullsized":"2024-05-10T04:03:29.192388+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:54.569786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635041+0000","last_change":"2024-05-10T04:02:31.641788+0000","last_active":"2024-05-10T04:02:35.635041+0000","last_peered":"2024-05-10T04:02:35.635041+0000","last_clean":"2024-05-10T04:02:35.635041+0000","last_became_active":"2024-05-10T04:02:31.641328+0000","last_became_peered":"2024-05-10T04:02:31.641328+0000","last_unstale":"2024-05-10T04:02:35.635041+0000","last_undegraded":"2024-05-10T04:02:35.635041+0000","last_fullsized":"2024-05-10T04:02:35.635041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:10:55.920245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.631051+0000","last_change":"2024-05-10T04:02:31.631919+0000","last_active":"2024-05-10T04:02:35.631051+0000","last_peered":"2024-05-10T04:02:35.631051+0000","last_clean":"2024-05-10T04:02:35.631051+0000","last_became_active":"2024-05-10T04:02:31.631592+0000","last_became_peered":"2024-05-10T04:02:31.631592+0000","last_unstale":"2024-05-10T04:02:35.631051+0000","last_undegraded":"2024-05-10T04:02:35.631051+0000","last_fullsized":"2024-05-10T04:02:35.631051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:59:03.792090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636548+0000","last_change":"2024-05-10T04:02:31.633288+0000","last_active":"2024-05-10T04:02:35.636548+0000","last_peered":"2024-05-10T04:02:35.636548+0000","last_clean":"2024-05-10T04:02:35.636548+0000","last_became_active":"2024-05-10T04:02:31.632582+0000","last_became_peered":"2024-05-10T04:02:31.632582+0000","last_unstale":"2024-05-10T04:02:35.636548+0000","last_undegraded":"2024-05-10T04:02:35.636548+0000","last_fullsized":"2024-05-10T04:02:35.636548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:27.991439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636432+0000","last_change":"2024-05-10T04:02:31.635833+0000","last_active":"2024-05-10T04:02:35.636432+0000","last_peered":"2024-05-10T04:02:35.636432+0000","last_clean":"2024-05-10T04:02:35.636432+0000","last_became_active":"2024-05-10T04:02:31.635520+0000","last_became_peered":"2024-05-10T04:02:31.635520+0000","last_unstale":"2024-05-10T04:02:35.636432+0000","last_undegraded":"2024-05-10T04:02:35.636432+0000","last_fullsized":"2024-05-10T04:02:35.636432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:40:06.312819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638804+0000","last_change":"2024-05-10T04:02:31.631734+0000","last_active":"2024-05-10T04:02:36.638804+0000","last_peered":"2024-05-10T04:02:36.638804+0000","last_clean":"2024-05-10T04:02:36.638804+0000","last_became_active":"2024-05-10T04:02:31.631375+0000","last_became_peered":"2024-05-10T04:02:31.631375+0000","last_unstale":"2024-05-10T04:02:36.638804+0000","last_undegraded":"2024-05-10T04:02:36.638804+0000","last_fullsized":"2024-05-10T04:02:36.638804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:25:13.815406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195179+0000","last_change":"2024-05-10T04:02:31.642292+0000","last_active":"2024-05-10T04:03:29.195179+0000","last_peered":"2024-05-10T04:03:29.195179+0000","last_clean":"2024-05-10T04:03:29.195179+0000","last_became_active":"2024-05-10T04:02:31.641570+0000","last_became_peered":"2024-05-10T04:02:31.641570+0000","last_unstale":"2024-05-10T04:03:29.195179+0000","last_undegraded":"2024-05-10T04:03:29.195179+0000","last_fullsized":"2024-05-10T04:03:29.195179+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:16:33.704809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195044+0000","last_change":"2024-05-10T04:02:31.625695+0000","last_active":"2024-05-10T04:03:29.195044+0000","last_peered":"2024-05-10T04:03:29.195044+0000","last_clean":"2024-05-10T04:03:29.195044+0000","last_became_active":"2024-05-10T04:02:31.625393+0000","last_became_peered":"2024-05-10T04:02:31.625393+0000","last_unstale":"2024-05-10T04:03:29.195044+0000","last_undegraded":"2024-05-10T04:03:29.195044+0000","last_fullsized":"2024-05-10T04:03:29.195044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:53.955921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.633141+0000","last_change":"2024-05-10T04:02:31.629152+0000","last_active":"2024-05-10T04:02:35.633141+0000","last_peered":"2024-05-10T04:02:35.633141+0000","last_clean":"2024-05-10T04:02:35.633141+0000","last_became_active":"2024-05-10T04:02:31.628778+0000","last_became_peered":"2024-05-10T04:02:31.628778+0000","last_unstale":"2024-05-10T04:02:35.633141+0000","last_undegraded":"2024-05-10T04:02:35.633141+0000","last_fullsized":"2024-05-10T04:02:35.633141+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:39.772500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.409997+0000","last_change":"2024-05-10T04:02:35.650536+0000","last_active":"2024-05-10T04:02:36.409997+0000","last_peered":"2024-05-10T04:02:36.409997+0000","last_clean":"2024-05-10T04:02:36.409997+0000","last_became_active":"2024-05-10T04:02:35.650102+0000","last_became_peered":"2024-05-10T04:02:35.650102+0000","last_unstale":"2024-05-10T04:02:36.409997+0000","last_undegraded":"2024-05-10T04:02:36.409997+0000","last_fullsized":"2024-05-10T04:02:36.409997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:19:35.605844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192581+0000","last_change":"2024-05-10T04:02:31.186846+0000","last_active":"2024-05-10T04:03:29.192581+0000","last_peered":"2024-05-10T04:03:29.192581+0000","last_clean":"2024-05-10T04:03:29.192581+0000","last_became_active":"2024-05-10T04:02:31.186497+0000","last_became_peered":"2024-05-10T04:02:31.186497+0000","last_unstale":"2024-05-10T04:03:29.192581+0000","last_undegraded":"2024-05-10T04:03:29.192581+0000","last_fullsized":"2024-05-10T04:03:29.192581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:15:40.832455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.252299+0000","last_change":"2024-05-10T04:03:30.251863+0000","last_active":"2024-05-10T04:03:30.252299+0000","last_peered":"2024-05-10T04:03:30.252299+0000","last_clean":"2024-05-10T04:03:30.252299+0000","last_became_active":"2024-05-10T04:03:30.251466+0000","last_became_peered":"2024-05-10T04:03:30.251466+0000","last_unstale":"2024-05-10T04:03:30.252299+0000","last_undegraded":"2024-05-10T04:03:30.252299+0000","last_fullsized":"2024-05-10T04:03:30.252299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:56:12.810512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641619+0000","last_change":"2024-05-10T04:02:35.651622+0000","last_active":"2024-05-10T04:02:36.641619+0000","last_peered":"2024-05-10T04:02:36.641619+0000","last_clean":"2024-05-10T04:02:36.641619+0000","last_became_active":"2024-05-10T04:02:35.651148+0000","last_became_peered":"2024-05-10T04:02:35.651148+0000","last_unstale":"2024-05-10T04:02:36.641619+0000","last_undegraded":"2024-05-10T04:02:36.641619+0000","last_fullsized":"2024-05-10T04:02:36.641619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:40:09.019286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199659+0000","last_change":"2024-05-10T04:02:31.187785+0000","last_active":"2024-05-10T04:03:29.199659+0000","last_peered":"2024-05-10T04:03:29.199659+0000","last_clean":"2024-05-10T04:03:29.199659+0000","last_became_active":"2024-05-10T04:02:31.187499+0000","last_became_peered":"2024-05-10T04:02:31.187499+0000","last_unstale":"2024-05-10T04:03:29.199659+0000","last_undegraded":"2024-05-10T04:03:29.199659+0000","last_fullsized":"2024-05-10T04:03:29.199659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:05:29.015767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636966+0000","last_change":"2024-05-10T04:02:31.624433+0000","last_active":"2024-05-10T04:02:35.636966+0000","last_peered":"2024-05-10T04:02:35.636966+0000","last_clean":"2024-05-10T04:02:35.636966+0000","last_became_active":"2024-05-10T04:02:31.624115+0000","last_became_peered":"2024-05-10T04:02:31.624115+0000","last_unstale":"2024-05-10T04:02:35.636966+0000","last_undegraded":"2024-05-10T04:02:35.636966+0000","last_fullsized":"2024-05-10T04:02:35.636966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:54.584681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641726+0000","last_change":"2024-05-10T04:02:35.651535+0000","last_active":"2024-05-10T04:02:36.641726+0000","last_peered":"2024-05-10T04:02:36.641726+0000","last_clean":"2024-05-10T04:02:36.641726+0000","last_became_active":"2024-05-10T04:02:35.651077+0000","last_became_peered":"2024-05-10T04:02:35.651077+0000","last_unstale":"2024-05-10T04:02:36.641726+0000","last_undegraded":"2024-05-10T04:02:36.641726+0000","last_fullsized":"2024-05-10T04:02:36.641726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:21:04.647730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.206678+0000","last_change":"2024-05-10T04:03:30.206145+0000","last_active":"2024-05-10T04:03:30.206678+0000","last_peered":"2024-05-10T04:03:30.206678+0000","last_clean":"2024-05-10T04:03:30.206678+0000","last_became_active":"2024-05-10T04:03:30.200138+0000","last_became_peered":"2024-05-10T04:03:30.200138+0000","last_unstale":"2024-05-10T04:03:30.206678+0000","last_undegraded":"2024-05-10T04:03:30.206678+0000","last_fullsized":"2024-05-10T04:03:30.206678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:49.885012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640147+0000","last_change":"2024-05-10T04:02:31.628124+0000","last_active":"2024-05-10T04:02:36.640147+0000","last_peered":"2024-05-10T04:02:36.640147+0000","last_clean":"2024-05-10T04:02:36.640147+0000","last_became_active":"2024-05-10T04:02:31.627789+0000","last_became_peered":"2024-05-10T04:02:31.627789+0000","last_unstale":"2024-05-10T04:02:36.640147+0000","last_undegraded":"2024-05-10T04:02:36.640147+0000","last_fullsized":"2024-05-10T04:02:36.640147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:40:11.240726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641791+0000","last_change":"2024-05-10T04:02:35.649536+0000","last_active":"2024-05-10T04:02:36.641791+0000","last_peered":"2024-05-10T04:02:36.641791+0000","last_clean":"2024-05-10T04:02:36.641791+0000","last_became_active":"2024-05-10T04:02:35.649130+0000","last_became_peered":"2024-05-10T04:02:35.649130+0000","last_unstale":"2024-05-10T04:02:36.641791+0000","last_undegraded":"2024-05-10T04:02:36.641791+0000","last_fullsized":"2024-05-10T04:02:36.641791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:03.870071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637193+0000","last_change":"2024-05-10T04:02:31.192627+0000","last_active":"2024-05-10T04:02:35.637193+0000","last_peered":"2024-05-10T04:02:35.637193+0000","last_clean":"2024-05-10T04:02:35.637193+0000","last_became_active":"2024-05-10T04:02:31.191433+0000","last_became_peered":"2024-05-10T04:02:31.191433+0000","last_unstale":"2024-05-10T04:02:35.637193+0000","last_undegraded":"2024-05-10T04:02:35.637193+0000","last_fullsized":"2024-05-10T04:02:35.637193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:29.235483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193100+0000","last_change":"2024-05-10T04:02:31.641192+0000","last_active":"2024-05-10T04:03:29.193100+0000","last_peered":"2024-05-10T04:03:29.193100+0000","last_clean":"2024-05-10T04:03:29.193100+0000","last_became_active":"2024-05-10T04:02:31.640540+0000","last_became_peered":"2024-05-10T04:02:31.640540+0000","last_unstale":"2024-05-10T04:03:29.193100+0000","last_undegraded":"2024-05-10T04:03:29.193100+0000","last_fullsized":"2024-05-10T04:03:29.193100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:36.734231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.410181+0000","last_change":"2024-05-10T04:02:35.644661+0000","last_active":"2024-05-10T04:02:36.410181+0000","last_peered":"2024-05-10T04:02:36.410181+0000","last_clean":"2024-05-10T04:02:36.410181+0000","last_became_active":"2024-05-10T04:02:35.644216+0000","last_became_peered":"2024-05-10T04:02:35.644216+0000","last_unstale":"2024-05-10T04:02:36.410181+0000","last_undegraded":"2024-05-10T04:02:36.410181+0000","last_fullsized":"2024-05-10T04:02:36.410181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:02.618922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.250689+0000","last_change":"2024-05-10T04:02:31.198037+0000","last_active":"2024-05-10T04:03:30.250689+0000","last_peered":"2024-05-10T04:03:30.250689+0000","last_clean":"2024-05-10T04:03:30.250689+0000","last_became_active":"2024-05-10T04:02:31.197679+0000","last_became_peered":"2024-05-10T04:02:31.197679+0000","last_unstale":"2024-05-10T04:03:30.250689+0000","last_undegraded":"2024-05-10T04:03:30.250689+0000","last_fullsized":"2024-05-10T04:03:30.250689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:19.059645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636057+0000","last_change":"2024-05-10T04:02:31.630020+0000","last_active":"2024-05-10T04:02:35.636057+0000","last_peered":"2024-05-10T04:02:35.636057+0000","last_clean":"2024-05-10T04:02:35.636057+0000","last_became_active":"2024-05-10T04:02:31.629738+0000","last_became_peered":"2024-05-10T04:02:31.629738+0000","last_unstale":"2024-05-10T04:02:35.636057+0000","last_undegraded":"2024-05-10T04:02:35.636057+0000","last_fullsized":"2024-05-10T04:02:35.636057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:58:42.317143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.410114+0000","last_change":"2024-05-10T04:02:35.647242+0000","last_active":"2024-05-10T04:02:36.410114+0000","last_peered":"2024-05-10T04:02:36.410114+0000","last_clean":"2024-05-10T04:02:36.410114+0000","last_became_active":"2024-05-10T04:02:35.646774+0000","last_became_peered":"2024-05-10T04:02:35.646774+0000","last_unstale":"2024-05-10T04:02:36.410114+0000","last_undegraded":"2024-05-10T04:02:36.410114+0000","last_fullsized":"2024-05-10T04:02:36.410114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:20:05.602820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.180856+0000","last_change":"2024-05-10T04:02:31.191981+0000","last_active":"2024-05-10T04:02:56.180856+0000","last_peered":"2024-05-10T04:02:56.180856+0000","last_clean":"2024-05-10T04:02:56.180856+0000","last_became_active":"2024-05-10T04:02:31.191632+0000","last_became_peered":"2024-05-10T04:02:31.191632+0000","last_unstale":"2024-05-10T04:02:56.180856+0000","last_undegraded":"2024-05-10T04:02:56.180856+0000","last_fullsized":"2024-05-10T04:02:56.180856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:44:52.744661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194359+0000","last_change":"2024-05-10T04:02:31.643240+0000","last_active":"2024-05-10T04:03:29.194359+0000","last_peered":"2024-05-10T04:03:29.194359+0000","last_clean":"2024-05-10T04:03:29.194359+0000","last_became_active":"2024-05-10T04:02:31.642891+0000","last_became_peered":"2024-05-10T04:02:31.642891+0000","last_unstale":"2024-05-10T04:03:29.194359+0000","last_undegraded":"2024-05-10T04:03:29.194359+0000","last_fullsized":"2024-05-10T04:03:29.194359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:41:42.901192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195070+0000","last_change":"2024-05-10T04:02:35.652370+0000","last_active":"2024-05-10T04:03:29.195070+0000","last_peered":"2024-05-10T04:03:29.195070+0000","last_clean":"2024-05-10T04:03:29.195070+0000","last_became_active":"2024-05-10T04:02:35.651803+0000","last_became_peered":"2024-05-10T04:02:35.651803+0000","last_unstale":"2024-05-10T04:03:29.195070+0000","last_undegraded":"2024-05-10T04:03:29.195070+0000","last_fullsized":"2024-05-10T04:03:29.195070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:33:36.049311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182414+0000","last_change":"2024-05-10T04:02:31.189053+0000","last_active":"2024-05-10T04:02:56.182414+0000","last_peered":"2024-05-10T04:02:56.182414+0000","last_clean":"2024-05-10T04:02:56.182414+0000","last_became_active":"2024-05-10T04:02:31.188614+0000","last_became_peered":"2024-05-10T04:02:31.188614+0000","last_unstale":"2024-05-10T04:02:56.182414+0000","last_undegraded":"2024-05-10T04:02:56.182414+0000","last_fullsized":"2024-05-10T04:02:56.182414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:59:37.750507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198409+0000","last_change":"2024-05-10T04:02:31.630379+0000","last_active":"2024-05-10T04:03:29.198409+0000","last_peered":"2024-05-10T04:03:29.198409+0000","last_clean":"2024-05-10T04:03:29.198409+0000","last_became_active":"2024-05-10T04:02:31.630094+0000","last_became_peered":"2024-05-10T04:02:31.630094+0000","last_unstale":"2024-05-10T04:03:29.198409+0000","last_undegraded":"2024-05-10T04:03:29.198409+0000","last_fullsized":"2024-05-10T04:03:29.198409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:05.699923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638588+0000","last_change":"2024-05-10T04:02:35.650770+0000","last_active":"2024-05-10T04:02:36.638588+0000","last_peered":"2024-05-10T04:02:36.638588+0000","last_clean":"2024-05-10T04:02:36.638588+0000","last_became_active":"2024-05-10T04:02:35.650351+0000","last_became_peered":"2024-05-10T04:02:35.650351+0000","last_unstale":"2024-05-10T04:02:36.638588+0000","last_undegraded":"2024-05-10T04:02:36.638588+0000","last_fullsized":"2024-05-10T04:02:36.638588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:41:06.728126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192986+0000","last_change":"2024-05-10T04:02:31.189557+0000","last_active":"2024-05-10T04:03:29.192986+0000","last_peered":"2024-05-10T04:03:29.192986+0000","last_clean":"2024-05-10T04:03:29.192986+0000","last_became_active":"2024-05-10T04:02:31.189206+0000","last_became_peered":"2024-05-10T04:02:31.189206+0000","last_unstale":"2024-05-10T04:03:29.192986+0000","last_undegraded":"2024-05-10T04:03:29.192986+0000","last_fullsized":"2024-05-10T04:03:29.192986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:11:08.225965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192249+0000","last_change":"2024-05-10T04:02:31.633561+0000","last_active":"2024-05-10T04:03:29.192249+0000","last_peered":"2024-05-10T04:03:29.192249+0000","last_clean":"2024-05-10T04:03:29.192249+0000","last_became_active":"2024-05-10T04:02:31.633224+0000","last_became_peered":"2024-05-10T04:02:31.633224+0000","last_unstale":"2024-05-10T04:03:29.192249+0000","last_undegraded":"2024-05-10T04:03:29.192249+0000","last_fullsized":"2024-05-10T04:03:29.192249+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:32.901146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198675+0000","last_change":"2024-05-10T04:02:35.649327+0000","last_active":"2024-05-10T04:03:29.198675+0000","last_peered":"2024-05-10T04:03:29.198675+0000","last_clean":"2024-05-10T04:03:29.198675+0000","last_became_active":"2024-05-10T04:02:35.648982+0000","last_became_peered":"2024-05-10T04:02:35.648982+0000","last_unstale":"2024-05-10T04:03:29.198675+0000","last_undegraded":"2024-05-10T04:03:29.198675+0000","last_fullsized":"2024-05-10T04:03:29.198675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:25.405748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198995+0000","last_change":"2024-05-10T04:02:31.186447+0000","last_active":"2024-05-10T04:03:29.198995+0000","last_peered":"2024-05-10T04:03:29.198995+0000","last_clean":"2024-05-10T04:03:29.198995+0000","last_became_active":"2024-05-10T04:02:31.186073+0000","last_became_peered":"2024-05-10T04:02:31.186073+0000","last_unstale":"2024-05-10T04:03:29.198995+0000","last_undegraded":"2024-05-10T04:03:29.198995+0000","last_fullsized":"2024-05-10T04:03:29.198995+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:53.381147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640890+0000","last_change":"2024-05-10T04:02:31.633772+0000","last_active":"2024-05-10T04:02:36.640890+0000","last_peered":"2024-05-10T04:02:36.640890+0000","last_clean":"2024-05-10T04:02:36.640890+0000","last_became_active":"2024-05-10T04:02:31.629592+0000","last_became_peered":"2024-05-10T04:02:31.629592+0000","last_unstale":"2024-05-10T04:02:36.640890+0000","last_undegraded":"2024-05-10T04:02:36.640890+0000","last_fullsized":"2024-05-10T04:02:36.640890+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:38:37.342351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191885+0000","last_change":"2024-05-10T04:02:35.646960+0000","last_active":"2024-05-10T04:03:29.191885+0000","last_peered":"2024-05-10T04:03:29.191885+0000","last_clean":"2024-05-10T04:03:29.191885+0000","last_became_active":"2024-05-10T04:02:35.646598+0000","last_became_peered":"2024-05-10T04:02:35.646598+0000","last_unstale":"2024-05-10T04:03:29.191885+0000","last_undegraded":"2024-05-10T04:03:29.191885+0000","last_fullsized":"2024-05-10T04:03:29.191885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:40:10.478745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192245+0000","last_change":"2024-05-10T04:02:31.192846+0000","last_active":"2024-05-10T04:03:29.192245+0000","last_peered":"2024-05-10T04:03:29.192245+0000","last_clean":"2024-05-10T04:03:29.192245+0000","last_became_active":"2024-05-10T04:02:31.192514+0000","last_became_peered":"2024-05-10T04:02:31.192514+0000","last_unstale":"2024-05-10T04:03:29.192245+0000","last_undegraded":"2024-05-10T04:03:29.192245+0000","last_fullsized":"2024-05-10T04:03:29.192245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:50:32.649440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637130+0000","last_change":"2024-05-10T04:02:31.641578+0000","last_active":"2024-05-10T04:02:35.637130+0000","last_peered":"2024-05-10T04:02:35.637130+0000","last_clean":"2024-05-10T04:02:35.637130+0000","last_became_active":"2024-05-10T04:02:31.641268+0000","last_became_peered":"2024-05-10T04:02:31.641268+0000","last_unstale":"2024-05-10T04:02:35.637130+0000","last_undegraded":"2024-05-10T04:02:35.637130+0000","last_fullsized":"2024-05-10T04:02:35.637130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:31:00.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641920+0000","last_change":"2024-05-10T04:02:35.646754+0000","last_active":"2024-05-10T04:02:36.641920+0000","last_peered":"2024-05-10T04:02:36.641920+0000","last_clean":"2024-05-10T04:02:36.641920+0000","last_became_active":"2024-05-10T04:02:35.646342+0000","last_became_peered":"2024-05-10T04:02:35.646342+0000","last_unstale":"2024-05-10T04:02:36.641920+0000","last_undegraded":"2024-05-10T04:02:36.641920+0000","last_fullsized":"2024-05-10T04:02:36.641920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:28:33.373312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192990+0000","last_change":"2024-05-10T04:02:31.186427+0000","last_active":"2024-05-10T04:03:29.192990+0000","last_peered":"2024-05-10T04:03:29.192990+0000","last_clean":"2024-05-10T04:03:29.192990+0000","last_became_active":"2024-05-10T04:02:31.185937+0000","last_became_peered":"2024-05-10T04:02:31.185937+0000","last_unstale":"2024-05-10T04:03:29.192990+0000","last_undegraded":"2024-05-10T04:03:29.192990+0000","last_fullsized":"2024-05-10T04:03:29.192990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.818126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640331+0000","last_change":"2024-05-10T04:02:31.645582+0000","last_active":"2024-05-10T04:02:36.640331+0000","last_peered":"2024-05-10T04:02:36.640331+0000","last_clean":"2024-05-10T04:02:36.640331+0000","last_became_active":"2024-05-10T04:02:31.645259+0000","last_became_peered":"2024-05-10T04:02:31.645259+0000","last_unstale":"2024-05-10T04:02:36.640331+0000","last_undegraded":"2024-05-10T04:02:36.640331+0000","last_fullsized":"2024-05-10T04:02:36.640331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:17:31.818116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192983+0000","last_change":"2024-05-10T04:02:35.647341+0000","last_active":"2024-05-10T04:03:29.192983+0000","last_peered":"2024-05-10T04:03:29.192983+0000","last_clean":"2024-05-10T04:03:29.192983+0000","last_became_active":"2024-05-10T04:02:35.646806+0000","last_became_peered":"2024-05-10T04:02:35.646806+0000","last_unstale":"2024-05-10T04:03:29.192983+0000","last_undegraded":"2024-05-10T04:03:29.192983+0000","last_fullsized":"2024-05-10T04:03:29.192983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:01:51.252683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'31","reported_seq":119,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.250684+0000","last_change":"2024-05-10T04:02:31.195377+0000","last_active":"2024-05-10T04:03:29.250684+0000","last_peered":"2024-05-10T04:03:29.250684+0000","last_clean":"2024-05-10T04:03:29.250684+0000","last_became_active":"2024-05-10T04:02:31.194965+0000","last_became_peered":"2024-05-10T04:02:31.194965+0000","last_unstale":"2024-05-10T04:03:29.250684+0000","last_undegraded":"2024-05-10T04:03:29.250684+0000","last_fullsized":"2024-05-10T04:03:29.250684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:11:09.237525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61731,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":73,"num_read_kb":97,"num_write":32,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636559+0000","last_change":"2024-05-10T04:02:31.629600+0000","last_active":"2024-05-10T04:02:35.636559+0000","last_peered":"2024-05-10T04:02:35.636559+0000","last_clean":"2024-05-10T04:02:35.636559+0000","last_became_active":"2024-05-10T04:02:31.629264+0000","last_became_peered":"2024-05-10T04:02:31.629264+0000","last_unstale":"2024-05-10T04:02:35.636559+0000","last_undegraded":"2024-05-10T04:02:35.636559+0000","last_fullsized":"2024-05-10T04:02:35.636559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:45:08.867783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193519+0000","last_change":"2024-05-10T04:02:35.649468+0000","last_active":"2024-05-10T04:03:29.193519+0000","last_peered":"2024-05-10T04:03:29.193519+0000","last_clean":"2024-05-10T04:03:29.193519+0000","last_became_active":"2024-05-10T04:02:35.649053+0000","last_became_peered":"2024-05-10T04:02:35.649053+0000","last_unstale":"2024-05-10T04:03:29.193519+0000","last_undegraded":"2024-05-10T04:03:29.193519+0000","last_fullsized":"2024-05-10T04:03:29.193519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:01:05.761287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182116+0000","last_change":"2024-05-10T04:02:31.195169+0000","last_active":"2024-05-10T04:02:56.182116+0000","last_peered":"2024-05-10T04:02:56.182116+0000","last_clean":"2024-05-10T04:02:56.182116+0000","last_became_active":"2024-05-10T04:02:31.194720+0000","last_became_peered":"2024-05-10T04:02:31.194720+0000","last_unstale":"2024-05-10T04:02:56.182116+0000","last_undegraded":"2024-05-10T04:02:56.182116+0000","last_fullsized":"2024-05-10T04:02:56.182116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:42.727841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630936+0000","last_change":"2024-05-10T04:02:31.631739+0000","last_active":"2024-05-10T04:02:35.630936+0000","last_peered":"2024-05-10T04:02:35.630936+0000","last_clean":"2024-05-10T04:02:35.630936+0000","last_became_active":"2024-05-10T04:02:31.631401+0000","last_became_peered":"2024-05-10T04:02:31.631401+0000","last_unstale":"2024-05-10T04:02:35.630936+0000","last_undegraded":"2024-05-10T04:02:35.630936+0000","last_fullsized":"2024-05-10T04:02:35.630936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:06.958328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196041+0000","last_change":"2024-05-10T04:02:35.651484+0000","last_active":"2024-05-10T04:03:29.196041+0000","last_peered":"2024-05-10T04:03:29.196041+0000","last_clean":"2024-05-10T04:03:29.196041+0000","last_became_active":"2024-05-10T04:02:35.651069+0000","last_became_peered":"2024-05-10T04:02:35.651069+0000","last_unstale":"2024-05-10T04:03:29.196041+0000","last_undegraded":"2024-05-10T04:03:29.196041+0000","last_fullsized":"2024-05-10T04:03:29.196041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:26:45.229101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192929+0000","last_change":"2024-05-10T04:02:31.186314+0000","last_active":"2024-05-10T04:03:29.192929+0000","last_peered":"2024-05-10T04:03:29.192929+0000","last_clean":"2024-05-10T04:03:29.192929+0000","last_became_active":"2024-05-10T04:02:31.185861+0000","last_became_peered":"2024-05-10T04:02:31.185861+0000","last_unstale":"2024-05-10T04:03:29.192929+0000","last_undegraded":"2024-05-10T04:03:29.192929+0000","last_fullsized":"2024-05-10T04:03:29.192929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:07.386978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191954+0000","last_change":"2024-05-10T04:02:31.637151+0000","last_active":"2024-05-10T04:03:29.191954+0000","last_peered":"2024-05-10T04:03:29.191954+0000","last_clean":"2024-05-10T04:03:29.191954+0000","last_became_active":"2024-05-10T04:02:31.636745+0000","last_became_peered":"2024-05-10T04:02:31.636745+0000","last_unstale":"2024-05-10T04:03:29.191954+0000","last_undegraded":"2024-05-10T04:03:29.191954+0000","last_fullsized":"2024-05-10T04:03:29.191954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:59.804391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192369+0000","last_change":"2024-05-10T04:02:35.656108+0000","last_active":"2024-05-10T04:03:29.192369+0000","last_peered":"2024-05-10T04:03:29.192369+0000","last_clean":"2024-05-10T04:03:29.192369+0000","last_became_active":"2024-05-10T04:02:35.655664+0000","last_became_peered":"2024-05-10T04:02:35.655664+0000","last_unstale":"2024-05-10T04:03:29.192369+0000","last_undegraded":"2024-05-10T04:03:29.192369+0000","last_fullsized":"2024-05-10T04:03:29.192369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:10.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:03:27.241862+0000","last_change":"2024-05-10T04:02:31.189432+0000","last_active":"2024-05-10T04:03:27.241862+0000","last_peered":"2024-05-10T04:03:27.241862+0000","last_clean":"2024-05-10T04:03:27.241862+0000","last_became_active":"2024-05-10T04:02:31.189062+0000","last_became_peered":"2024-05-10T04:02:31.189062+0000","last_unstale":"2024-05-10T04:03:27.241862+0000","last_undegraded":"2024-05-10T04:03:27.241862+0000","last_fullsized":"2024-05-10T04:03:27.241862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:28.452100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191766+0000","last_change":"2024-05-10T04:02:31.637517+0000","last_active":"2024-05-10T04:03:29.191766+0000","last_peered":"2024-05-10T04:03:29.191766+0000","last_clean":"2024-05-10T04:03:29.191766+0000","last_became_active":"2024-05-10T04:02:31.636518+0000","last_became_peered":"2024-05-10T04:02:31.636518+0000","last_unstale":"2024-05-10T04:03:29.191766+0000","last_undegraded":"2024-05-10T04:03:29.191766+0000","last_fullsized":"2024-05-10T04:03:29.191766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:07:09.732117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.067746+0000","last_change":"2024-05-10T04:02:35.649693+0000","last_active":"2024-05-10T04:02:36.067746+0000","last_peered":"2024-05-10T04:02:36.067746+0000","last_clean":"2024-05-10T04:02:36.067746+0000","last_became_active":"2024-05-10T04:02:35.649175+0000","last_became_peered":"2024-05-10T04:02:35.649175+0000","last_unstale":"2024-05-10T04:02:36.067746+0000","last_undegraded":"2024-05-10T04:02:36.067746+0000","last_fullsized":"2024-05-10T04:02:36.067746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:03:52.340444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.181556+0000","last_change":"2024-05-10T04:02:31.196909+0000","last_active":"2024-05-10T04:02:56.181556+0000","last_peered":"2024-05-10T04:02:56.181556+0000","last_clean":"2024-05-10T04:02:56.181556+0000","last_became_active":"2024-05-10T04:02:31.196599+0000","last_became_peered":"2024-05-10T04:02:31.196599+0000","last_unstale":"2024-05-10T04:02:56.181556+0000","last_undegraded":"2024-05-10T04:02:56.181556+0000","last_fullsized":"2024-05-10T04:02:56.181556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:16:51.483498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194588+0000","last_change":"2024-05-10T04:02:31.641426+0000","last_active":"2024-05-10T04:03:29.194588+0000","last_peered":"2024-05-10T04:03:29.194588+0000","last_clean":"2024-05-10T04:03:29.194588+0000","last_became_active":"2024-05-10T04:02:31.641054+0000","last_became_peered":"2024-05-10T04:02:31.641054+0000","last_unstale":"2024-05-10T04:03:29.194588+0000","last_undegraded":"2024-05-10T04:03:29.194588+0000","last_fullsized":"2024-05-10T04:03:29.194588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:12:48.472674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194503+0000","last_change":"2024-05-10T04:02:35.653258+0000","last_active":"2024-05-10T04:03:29.194503+0000","last_peered":"2024-05-10T04:03:29.194503+0000","last_clean":"2024-05-10T04:03:29.194503+0000","last_became_active":"2024-05-10T04:02:35.652859+0000","last_became_peered":"2024-05-10T04:02:35.652859+0000","last_unstale":"2024-05-10T04:03:29.194503+0000","last_undegraded":"2024-05-10T04:03:29.194503+0000","last_fullsized":"2024-05-10T04:03:29.194503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:15:57.814963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:57.193415+0000","last_change":"2024-05-10T04:02:31.192231+0000","last_active":"2024-05-10T04:02:57.193415+0000","last_peered":"2024-05-10T04:02:57.193415+0000","last_clean":"2024-05-10T04:02:57.193415+0000","last_became_active":"2024-05-10T04:02:31.191902+0000","last_became_peered":"2024-05-10T04:02:31.191902+0000","last_unstale":"2024-05-10T04:02:57.193415+0000","last_undegraded":"2024-05-10T04:02:57.193415+0000","last_fullsized":"2024-05-10T04:02:57.193415+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:18.637112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195378+0000","last_change":"2024-05-10T04:02:31.635514+0000","last_active":"2024-05-10T04:03:29.195378+0000","last_peered":"2024-05-10T04:03:29.195378+0000","last_clean":"2024-05-10T04:03:29.195378+0000","last_became_active":"2024-05-10T04:02:31.635171+0000","last_became_peered":"2024-05-10T04:02:31.635171+0000","last_unstale":"2024-05-10T04:03:29.195378+0000","last_undegraded":"2024-05-10T04:03:29.195378+0000","last_fullsized":"2024-05-10T04:03:29.195378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:48.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.090937+0000","last_change":"2024-05-10T04:02:35.900174+0000","last_active":"2024-05-10T04:02:36.090937+0000","last_peered":"2024-05-10T04:02:36.090937+0000","last_clean":"2024-05-10T04:02:36.090937+0000","last_became_active":"2024-05-10T04:02:35.899653+0000","last_became_peered":"2024-05-10T04:02:35.899653+0000","last_unstale":"2024-05-10T04:02:36.090937+0000","last_undegraded":"2024-05-10T04:02:36.090937+0000","last_fullsized":"2024-05-10T04:02:36.090937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:32:56.962361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194614+0000","last_change":"2024-05-10T04:02:31.190173+0000","last_active":"2024-05-10T04:03:29.194614+0000","last_peered":"2024-05-10T04:03:29.194614+0000","last_clean":"2024-05-10T04:03:29.194614+0000","last_became_active":"2024-05-10T04:02:31.189840+0000","last_became_peered":"2024-05-10T04:02:31.189840+0000","last_unstale":"2024-05-10T04:03:29.194614+0000","last_undegraded":"2024-05-10T04:03:29.194614+0000","last_fullsized":"2024-05-10T04:03:29.194614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:31:49.382482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.639305+0000","last_change":"2024-05-10T04:02:31.629497+0000","last_active":"2024-05-10T04:02:36.639305+0000","last_peered":"2024-05-10T04:02:36.639305+0000","last_clean":"2024-05-10T04:02:36.639305+0000","last_became_active":"2024-05-10T04:02:31.629182+0000","last_became_peered":"2024-05-10T04:02:31.629182+0000","last_unstale":"2024-05-10T04:02:36.639305+0000","last_undegraded":"2024-05-10T04:02:36.639305+0000","last_fullsized":"2024-05-10T04:02:36.639305+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:11:33.478737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.067644+0000","last_change":"2024-05-10T04:02:35.647249+0000","last_active":"2024-05-10T04:02:36.067644+0000","last_peered":"2024-05-10T04:02:36.067644+0000","last_clean":"2024-05-10T04:02:36.067644+0000","last_became_active":"2024-05-10T04:02:35.646716+0000","last_became_peered":"2024-05-10T04:02:35.646716+0000","last_unstale":"2024-05-10T04:02:36.067644+0000","last_undegraded":"2024-05-10T04:02:36.067644+0000","last_fullsized":"2024-05-10T04:02:36.067644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:44.651836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191715+0000","last_change":"2024-05-10T04:02:31.191241+0000","last_active":"2024-05-10T04:03:29.191715+0000","last_peered":"2024-05-10T04:03:29.191715+0000","last_clean":"2024-05-10T04:03:29.191715+0000","last_became_active":"2024-05-10T04:02:31.190888+0000","last_became_peered":"2024-05-10T04:02:31.190888+0000","last_unstale":"2024-05-10T04:03:29.191715+0000","last_undegraded":"2024-05-10T04:03:29.191715+0000","last_fullsized":"2024-05-10T04:03:29.191715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:55.256119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.639659+0000","last_change":"2024-05-10T04:02:31.638394+0000","last_active":"2024-05-10T04:02:36.639659+0000","last_peered":"2024-05-10T04:02:36.639659+0000","last_clean":"2024-05-10T04:02:36.639659+0000","last_became_active":"2024-05-10T04:02:31.638025+0000","last_became_peered":"2024-05-10T04:02:31.638025+0000","last_unstale":"2024-05-10T04:02:36.639659+0000","last_undegraded":"2024-05-10T04:02:36.639659+0000","last_fullsized":"2024-05-10T04:02:36.639659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:13.088393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.067640+0000","last_change":"2024-05-10T04:02:35.899898+0000","last_active":"2024-05-10T04:02:36.067640+0000","last_peered":"2024-05-10T04:02:36.067640+0000","last_clean":"2024-05-10T04:02:36.067640+0000","last_became_active":"2024-05-10T04:02:35.899485+0000","last_became_peered":"2024-05-10T04:02:35.899485+0000","last_unstale":"2024-05-10T04:02:36.067640+0000","last_undegraded":"2024-05-10T04:02:36.067640+0000","last_fullsized":"2024-05-10T04:02:36.067640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:26:50.909122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193219+0000","last_change":"2024-05-10T04:02:31.640923+0000","last_active":"2024-05-10T04:03:29.193219+0000","last_peered":"2024-05-10T04:03:29.193219+0000","last_clean":"2024-05-10T04:03:29.193219+0000","last_became_active":"2024-05-10T04:02:31.640490+0000","last_became_peered":"2024-05-10T04:02:31.640490+0000","last_unstale":"2024-05-10T04:03:29.193219+0000","last_undegraded":"2024-05-10T04:03:29.193219+0000","last_fullsized":"2024-05-10T04:03:29.193219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:31:58.585789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:03:27.241092+0000","last_change":"2024-05-10T04:02:31.191326+0000","last_active":"2024-05-10T04:03:27.241092+0000","last_peered":"2024-05-10T04:03:27.241092+0000","last_clean":"2024-05-10T04:03:27.241092+0000","last_became_active":"2024-05-10T04:02:31.190974+0000","last_became_peered":"2024-05-10T04:02:31.190974+0000","last_unstale":"2024-05-10T04:03:27.241092+0000","last_undegraded":"2024-05-10T04:03:27.241092+0000","last_fullsized":"2024-05-10T04:03:27.241092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:43.670619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192264+0000","last_change":"2024-05-10T04:02:35.647526+0000","last_active":"2024-05-10T04:03:29.192264+0000","last_peered":"2024-05-10T04:03:29.192264+0000","last_clean":"2024-05-10T04:03:29.192264+0000","last_became_active":"2024-05-10T04:02:35.647018+0000","last_became_peered":"2024-05-10T04:02:35.647018+0000","last_unstale":"2024-05-10T04:03:29.192264+0000","last_undegraded":"2024-05-10T04:03:29.192264+0000","last_fullsized":"2024-05-10T04:03:29.192264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:17.173497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199043+0000","last_change":"2024-05-10T04:02:31.632263+0000","last_active":"2024-05-10T04:03:29.199043+0000","last_peered":"2024-05-10T04:03:29.199043+0000","last_clean":"2024-05-10T04:03:29.199043+0000","last_became_active":"2024-05-10T04:02:31.631785+0000","last_became_peered":"2024-05-10T04:02:31.631785+0000","last_unstale":"2024-05-10T04:03:29.199043+0000","last_undegraded":"2024-05-10T04:03:29.199043+0000","last_fullsized":"2024-05-10T04:03:29.199043+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:30.585862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:57.191865+0000","last_change":"2024-05-10T04:02:31.182262+0000","last_active":"2024-05-10T04:02:57.191865+0000","last_peered":"2024-05-10T04:02:57.191865+0000","last_clean":"2024-05-10T04:02:57.191865+0000","last_became_active":"2024-05-10T04:02:31.181802+0000","last_became_peered":"2024-05-10T04:02:31.181802+0000","last_unstale":"2024-05-10T04:02:57.191865+0000","last_undegraded":"2024-05-10T04:02:57.191865+0000","last_fullsized":"2024-05-10T04:02:57.191865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:45:40.696571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":327,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191496+0000","last_change":"2024-05-10T04:01:41.866365+0000","last_active":"2024-05-10T04:03:29.191496+0000","last_peered":"2024-05-10T04:03:29.191496+0000","last_clean":"2024-05-10T04:03:29.191496+0000","last_became_active":"2024-05-10T04:01:41.865825+0000","last_became_peered":"2024-05-10T04:01:41.865825+0000","last_unstale":"2024-05-10T04:03:29.191496+0000","last_undegraded":"2024-05-10T04:03:29.191496+0000","last_fullsized":"2024-05-10T04:03:29.191496+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_clean_scrub_stamp":"2024-05-10T03:59:19.853473+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:28:29.598339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193669+0000","last_change":"2024-05-10T04:02:35.900841+0000","last_active":"2024-05-10T04:03:29.193669+0000","last_peered":"2024-05-10T04:03:29.193669+0000","last_clean":"2024-05-10T04:03:29.193669+0000","last_became_active":"2024-05-10T04:02:35.900323+0000","last_became_peered":"2024-05-10T04:02:35.900323+0000","last_unstale":"2024-05-10T04:03:29.193669+0000","last_undegraded":"2024-05-10T04:03:29.193669+0000","last_fullsized":"2024-05-10T04:03:29.193669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:00.554804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630799+0000","last_change":"2024-05-10T04:02:31.641167+0000","last_active":"2024-05-10T04:02:35.630799+0000","last_peered":"2024-05-10T04:02:35.630799+0000","last_clean":"2024-05-10T04:02:35.630799+0000","last_became_active":"2024-05-10T04:02:31.640824+0000","last_became_peered":"2024-05-10T04:02:31.640824+0000","last_unstale":"2024-05-10T04:02:35.630799+0000","last_undegraded":"2024-05-10T04:02:35.630799+0000","last_fullsized":"2024-05-10T04:02:35.630799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:59:48.531970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191634+0000","last_change":"2024-05-10T04:02:31.187834+0000","last_active":"2024-05-10T04:03:29.191634+0000","last_peered":"2024-05-10T04:03:29.191634+0000","last_clean":"2024-05-10T04:03:29.191634+0000","last_became_active":"2024-05-10T04:02:31.187440+0000","last_became_peered":"2024-05-10T04:02:31.187440+0000","last_unstale":"2024-05-10T04:03:29.191634+0000","last_undegraded":"2024-05-10T04:03:29.191634+0000","last_fullsized":"2024-05-10T04:03:29.191634+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:50:37.678245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194942+0000","last_change":"2024-05-10T04:02:35.649664+0000","last_active":"2024-05-10T04:03:29.194942+0000","last_peered":"2024-05-10T04:03:29.194942+0000","last_clean":"2024-05-10T04:03:29.194942+0000","last_became_active":"2024-05-10T04:02:35.649076+0000","last_became_peered":"2024-05-10T04:02:35.649076+0000","last_unstale":"2024-05-10T04:03:29.194942+0000","last_undegraded":"2024-05-10T04:03:29.194942+0000","last_fullsized":"2024-05-10T04:03:29.194942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:14:42.864845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.173775+0000","last_change":"2024-05-10T04:02:31.192126+0000","last_active":"2024-05-10T04:02:55.173775+0000","last_peered":"2024-05-10T04:02:55.173775+0000","last_clean":"2024-05-10T04:02:55.173775+0000","last_became_active":"2024-05-10T04:02:31.191771+0000","last_became_peered":"2024-05-10T04:02:31.191771+0000","last_unstale":"2024-05-10T04:02:55.173775+0000","last_undegraded":"2024-05-10T04:02:55.173775+0000","last_fullsized":"2024-05-10T04:02:55.173775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:43.550535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632220+0000","last_change":"2024-05-10T04:02:31.620788+0000","last_active":"2024-05-10T04:02:35.632220+0000","last_peered":"2024-05-10T04:02:35.632220+0000","last_clean":"2024-05-10T04:02:35.632220+0000","last_became_active":"2024-05-10T04:02:31.620456+0000","last_became_peered":"2024-05-10T04:02:31.620456+0000","last_unstale":"2024-05-10T04:02:35.632220+0000","last_undegraded":"2024-05-10T04:02:35.632220+0000","last_fullsized":"2024-05-10T04:02:35.632220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:02.956030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198321+0000","last_change":"2024-05-10T04:02:35.899935+0000","last_active":"2024-05-10T04:03:29.198321+0000","last_peered":"2024-05-10T04:03:29.198321+0000","last_clean":"2024-05-10T04:03:29.198321+0000","last_became_active":"2024-05-10T04:02:35.899488+0000","last_became_peered":"2024-05-10T04:02:35.899488+0000","last_unstale":"2024-05-10T04:03:29.198321+0000","last_undegraded":"2024-05-10T04:03:29.198321+0000","last_fullsized":"2024-05-10T04:03:29.198321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:28.600463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195179+0000","last_change":"2024-05-10T04:02:31.188605+0000","last_active":"2024-05-10T04:03:29.195179+0000","last_peered":"2024-05-10T04:03:29.195179+0000","last_clean":"2024-05-10T04:03:29.195179+0000","last_became_active":"2024-05-10T04:02:31.188283+0000","last_became_peered":"2024-05-10T04:02:31.188283+0000","last_unstale":"2024-05-10T04:03:29.195179+0000","last_undegraded":"2024-05-10T04:03:29.195179+0000","last_fullsized":"2024-05-10T04:03:29.195179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:40:03.940269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199712+0000","last_change":"2024-05-10T04:02:31.629476+0000","last_active":"2024-05-10T04:03:29.199712+0000","last_peered":"2024-05-10T04:03:29.199712+0000","last_clean":"2024-05-10T04:03:29.199712+0000","last_became_active":"2024-05-10T04:02:31.629104+0000","last_became_peered":"2024-05-10T04:02:31.629104+0000","last_unstale":"2024-05-10T04:03:29.199712+0000","last_undegraded":"2024-05-10T04:03:29.199712+0000","last_fullsized":"2024-05-10T04:03:29.199712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:57.873755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.409730+0000","last_change":"2024-05-10T04:02:35.644094+0000","last_active":"2024-05-10T04:02:36.409730+0000","last_peered":"2024-05-10T04:02:36.409730+0000","last_clean":"2024-05-10T04:02:36.409730+0000","last_became_active":"2024-05-10T04:02:35.643382+0000","last_became_peered":"2024-05-10T04:02:35.643382+0000","last_unstale":"2024-05-10T04:02:36.409730+0000","last_undegraded":"2024-05-10T04:02:36.409730+0000","last_fullsized":"2024-05-10T04:02:36.409730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:33.250052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.175394+0000","last_change":"2024-05-10T04:02:31.189302+0000","last_active":"2024-05-10T04:02:55.175394+0000","last_peered":"2024-05-10T04:02:55.175394+0000","last_clean":"2024-05-10T04:02:55.175394+0000","last_became_active":"2024-05-10T04:02:31.188323+0000","last_became_peered":"2024-05-10T04:02:31.188323+0000","last_unstale":"2024-05-10T04:02:55.175394+0000","last_undegraded":"2024-05-10T04:02:55.175394+0000","last_fullsized":"2024-05-10T04:02:55.175394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:11:02.922884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.637085+0000","last_change":"2024-05-10T04:02:31.635864+0000","last_active":"2024-05-10T04:02:35.637085+0000","last_peered":"2024-05-10T04:02:35.637085+0000","last_clean":"2024-05-10T04:02:35.637085+0000","last_became_active":"2024-05-10T04:02:31.635512+0000","last_became_peered":"2024-05-10T04:02:31.635512+0000","last_unstale":"2024-05-10T04:02:35.637085+0000","last_undegraded":"2024-05-10T04:02:35.637085+0000","last_fullsized":"2024-05-10T04:02:35.637085+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:26:10.957848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.642076+0000","last_change":"2024-05-10T04:02:35.900489+0000","last_active":"2024-05-10T04:02:36.642076+0000","last_peered":"2024-05-10T04:02:36.642076+0000","last_clean":"2024-05-10T04:02:36.642076+0000","last_became_active":"2024-05-10T04:02:35.899908+0000","last_became_peered":"2024-05-10T04:02:35.899908+0000","last_unstale":"2024-05-10T04:02:36.642076+0000","last_undegraded":"2024-05-10T04:02:36.642076+0000","last_fullsized":"2024-05-10T04:02:36.642076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:08.386128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.174482+0000","last_change":"2024-05-10T04:02:31.193956+0000","last_active":"2024-05-10T04:02:55.174482+0000","last_peered":"2024-05-10T04:02:55.174482+0000","last_clean":"2024-05-10T04:02:55.174482+0000","last_became_active":"2024-05-10T04:02:31.193574+0000","last_became_peered":"2024-05-10T04:02:31.193574+0000","last_unstale":"2024-05-10T04:02:55.174482+0000","last_undegraded":"2024-05-10T04:02:55.174482+0000","last_fullsized":"2024-05-10T04:02:55.174482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:46.369152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.635522+0000","last_change":"2024-05-10T04:02:31.633263+0000","last_active":"2024-05-10T04:02:35.635522+0000","last_peered":"2024-05-10T04:02:35.635522+0000","last_clean":"2024-05-10T04:02:35.635522+0000","last_became_active":"2024-05-10T04:02:31.632903+0000","last_became_peered":"2024-05-10T04:02:31.632903+0000","last_unstale":"2024-05-10T04:02:35.635522+0000","last_undegraded":"2024-05-10T04:02:35.635522+0000","last_fullsized":"2024-05-10T04:02:35.635522+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:28.047473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192521+0000","last_change":"2024-05-10T04:02:35.656003+0000","last_active":"2024-05-10T04:03:29.192521+0000","last_peered":"2024-05-10T04:03:29.192521+0000","last_clean":"2024-05-10T04:03:29.192521+0000","last_became_active":"2024-05-10T04:02:35.655518+0000","last_became_peered":"2024-05-10T04:02:35.655518+0000","last_unstale":"2024-05-10T04:03:29.192521+0000","last_undegraded":"2024-05-10T04:03:29.192521+0000","last_fullsized":"2024-05-10T04:03:29.192521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:49.696699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198559+0000","last_change":"2024-05-10T04:02:31.195297+0000","last_active":"2024-05-10T04:03:29.198559+0000","last_peered":"2024-05-10T04:03:29.198559+0000","last_clean":"2024-05-10T04:03:29.198559+0000","last_became_active":"2024-05-10T04:02:31.193960+0000","last_became_peered":"2024-05-10T04:02:31.193960+0000","last_unstale":"2024-05-10T04:03:29.198559+0000","last_undegraded":"2024-05-10T04:03:29.198559+0000","last_fullsized":"2024-05-10T04:03:29.198559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:04:10.314700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193216+0000","last_change":"2024-05-10T04:02:31.629247+0000","last_active":"2024-05-10T04:03:29.193216+0000","last_peered":"2024-05-10T04:03:29.193216+0000","last_clean":"2024-05-10T04:03:29.193216+0000","last_became_active":"2024-05-10T04:02:31.628838+0000","last_became_peered":"2024-05-10T04:02:31.628838+0000","last_unstale":"2024-05-10T04:03:29.193216+0000","last_undegraded":"2024-05-10T04:03:29.193216+0000","last_fullsized":"2024-05-10T04:03:29.193216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:46:39.483264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:38.823384+0000","last_change":"2024-05-10T04:02:35.644502+0000","last_active":"2024-05-10T04:02:38.823384+0000","last_peered":"2024-05-10T04:02:38.823384+0000","last_clean":"2024-05-10T04:02:38.823384+0000","last_became_active":"2024-05-10T04:02:35.643942+0000","last_became_peered":"2024-05-10T04:02:35.643942+0000","last_unstale":"2024-05-10T04:02:38.823384+0000","last_undegraded":"2024-05-10T04:02:38.823384+0000","last_fullsized":"2024-05-10T04:02:38.823384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:22:42.394568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.202901+0000","last_change":"2024-05-10T04:03:30.202456+0000","last_active":"2024-05-10T04:03:30.202901+0000","last_peered":"2024-05-10T04:03:30.202901+0000","last_clean":"2024-05-10T04:03:30.202901+0000","last_became_active":"2024-05-10T04:03:30.197484+0000","last_became_peered":"2024-05-10T04:03:30.197484+0000","last_unstale":"2024-05-10T04:03:30.202901+0000","last_undegraded":"2024-05-10T04:03:30.202901+0000","last_fullsized":"2024-05-10T04:03:30.202901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:15:07.899758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198950+0000","last_change":"2024-05-10T04:02:31.628029+0000","last_active":"2024-05-10T04:03:29.198950+0000","last_peered":"2024-05-10T04:03:29.198950+0000","last_clean":"2024-05-10T04:03:29.198950+0000","last_became_active":"2024-05-10T04:02:31.627667+0000","last_became_peered":"2024-05-10T04:02:31.627667+0000","last_unstale":"2024-05-10T04:03:29.198950+0000","last_undegraded":"2024-05-10T04:03:29.198950+0000","last_fullsized":"2024-05-10T04:03:29.198950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:39:10.878894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641977+0000","last_change":"2024-05-10T04:02:35.900237+0000","last_active":"2024-05-10T04:02:36.641977+0000","last_peered":"2024-05-10T04:02:36.641977+0000","last_clean":"2024-05-10T04:02:36.641977+0000","last_became_active":"2024-05-10T04:02:35.899781+0000","last_became_peered":"2024-05-10T04:02:35.899781+0000","last_unstale":"2024-05-10T04:02:36.641977+0000","last_undegraded":"2024-05-10T04:02:36.641977+0000","last_fullsized":"2024-05-10T04:02:36.641977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:52:46.132557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:55.174762+0000","last_change":"2024-05-10T04:02:31.196920+0000","last_active":"2024-05-10T04:02:55.174762+0000","last_peered":"2024-05-10T04:02:55.174762+0000","last_clean":"2024-05-10T04:02:55.174762+0000","last_became_active":"2024-05-10T04:02:31.196191+0000","last_became_peered":"2024-05-10T04:02:31.196191+0000","last_unstale":"2024-05-10T04:02:55.174762+0000","last_undegraded":"2024-05-10T04:02:55.174762+0000","last_fullsized":"2024-05-10T04:02:55.174762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:45:17.980149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191841+0000","last_change":"2024-05-10T04:02:31.623678+0000","last_active":"2024-05-10T04:03:29.191841+0000","last_peered":"2024-05-10T04:03:29.191841+0000","last_clean":"2024-05-10T04:03:29.191841+0000","last_became_active":"2024-05-10T04:02:31.623090+0000","last_became_peered":"2024-05-10T04:02:31.623090+0000","last_unstale":"2024-05-10T04:03:29.191841+0000","last_undegraded":"2024-05-10T04:03:29.191841+0000","last_fullsized":"2024-05-10T04:03:29.191841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:34.321959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.640966+0000","last_change":"2024-05-10T04:02:35.648833+0000","last_active":"2024-05-10T04:02:36.640966+0000","last_peered":"2024-05-10T04:02:36.640966+0000","last_clean":"2024-05-10T04:02:36.640966+0000","last_became_active":"2024-05-10T04:02:35.648391+0000","last_became_peered":"2024-05-10T04:02:35.648391+0000","last_unstale":"2024-05-10T04:02:36.640966+0000","last_undegraded":"2024-05-10T04:02:36.640966+0000","last_fullsized":"2024-05-10T04:02:36.640966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:12.773284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:56.182221+0000","last_change":"2024-05-10T04:02:31.192900+0000","last_active":"2024-05-10T04:02:56.182221+0000","last_peered":"2024-05-10T04:02:56.182221+0000","last_clean":"2024-05-10T04:02:56.182221+0000","last_became_active":"2024-05-10T04:02:31.192481+0000","last_became_peered":"2024-05-10T04:02:31.192481+0000","last_unstale":"2024-05-10T04:02:56.182221+0000","last_undegraded":"2024-05-10T04:02:56.182221+0000","last_fullsized":"2024-05-10T04:02:56.182221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:17.202383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192511+0000","last_change":"2024-05-10T04:02:31.628490+0000","last_active":"2024-05-10T04:03:29.192511+0000","last_peered":"2024-05-10T04:03:29.192511+0000","last_clean":"2024-05-10T04:03:29.192511+0000","last_became_active":"2024-05-10T04:02:31.627889+0000","last_became_peered":"2024-05-10T04:02:31.627889+0000","last_unstale":"2024-05-10T04:03:29.192511+0000","last_undegraded":"2024-05-10T04:03:29.192511+0000","last_fullsized":"2024-05-10T04:03:29.192511+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:32:55.366759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.638326+0000","last_change":"2024-05-10T04:02:35.653530+0000","last_active":"2024-05-10T04:02:36.638326+0000","last_peered":"2024-05-10T04:02:36.638326+0000","last_clean":"2024-05-10T04:02:36.638326+0000","last_became_active":"2024-05-10T04:02:35.653095+0000","last_became_peered":"2024-05-10T04:02:35.653095+0000","last_unstale":"2024-05-10T04:02:36.638326+0000","last_undegraded":"2024-05-10T04:02:36.638326+0000","last_fullsized":"2024-05-10T04:02:36.638326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:09.921713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193200+0000","last_change":"2024-05-10T04:02:31.189434+0000","last_active":"2024-05-10T04:03:29.193200+0000","last_peered":"2024-05-10T04:03:29.193200+0000","last_clean":"2024-05-10T04:03:29.193200+0000","last_became_active":"2024-05-10T04:02:31.188843+0000","last_became_peered":"2024-05-10T04:02:31.188843+0000","last_unstale":"2024-05-10T04:03:29.193200+0000","last_undegraded":"2024-05-10T04:03:29.193200+0000","last_fullsized":"2024-05-10T04:03:29.193200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:31.508794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191797+0000","last_change":"2024-05-10T04:02:31.636775+0000","last_active":"2024-05-10T04:03:29.191797+0000","last_peered":"2024-05-10T04:03:29.191797+0000","last_clean":"2024-05-10T04:03:29.191797+0000","last_became_active":"2024-05-10T04:02:31.635946+0000","last_became_peered":"2024-05-10T04:02:31.635946+0000","last_unstale":"2024-05-10T04:03:29.191797+0000","last_undegraded":"2024-05-10T04:03:29.191797+0000","last_fullsized":"2024-05-10T04:03:29.191797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:48.830097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192629+0000","last_change":"2024-05-10T04:02:35.646991+0000","last_active":"2024-05-10T04:03:29.192629+0000","last_peered":"2024-05-10T04:03:29.192629+0000","last_clean":"2024-05-10T04:03:29.192629+0000","last_became_active":"2024-05-10T04:02:35.646585+0000","last_became_peered":"2024-05-10T04:02:35.646585+0000","last_unstale":"2024-05-10T04:03:29.192629+0000","last_undegraded":"2024-05-10T04:03:29.192629+0000","last_fullsized":"2024-05-10T04:03:29.192629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:32:50.329746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.631077+0000","last_change":"2024-05-10T04:02:31.192103+0000","last_active":"2024-05-10T04:02:35.631077+0000","last_peered":"2024-05-10T04:02:35.631077+0000","last_clean":"2024-05-10T04:02:35.631077+0000","last_became_active":"2024-05-10T04:02:31.191378+0000","last_became_peered":"2024-05-10T04:02:31.191378+0000","last_unstale":"2024-05-10T04:02:35.631077+0000","last_undegraded":"2024-05-10T04:02:35.631077+0000","last_fullsized":"2024-05-10T04:02:35.631077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:55.298158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192829+0000","last_change":"2024-05-10T04:02:31.628587+0000","last_active":"2024-05-10T04:03:29.192829+0000","last_peered":"2024-05-10T04:03:29.192829+0000","last_clean":"2024-05-10T04:03:29.192829+0000","last_became_active":"2024-05-10T04:02:31.627296+0000","last_became_peered":"2024-05-10T04:02:31.627296+0000","last_unstale":"2024-05-10T04:03:29.192829+0000","last_undegraded":"2024-05-10T04:03:29.192829+0000","last_fullsized":"2024-05-10T04:03:29.192829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:04:09.783266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193327+0000","last_change":"2024-05-10T04:02:31.634668+0000","last_active":"2024-05-10T04:03:29.193327+0000","last_peered":"2024-05-10T04:03:29.193327+0000","last_clean":"2024-05-10T04:03:29.193327+0000","last_became_active":"2024-05-10T04:02:31.634321+0000","last_became_peered":"2024-05-10T04:02:31.634321+0000","last_unstale":"2024-05-10T04:03:29.193327+0000","last_undegraded":"2024-05-10T04:03:29.193327+0000","last_fullsized":"2024-05-10T04:03:29.193327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:16:44.833804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.633396+0000","last_change":"2024-05-10T04:02:31.646002+0000","last_active":"2024-05-10T04:02:35.633396+0000","last_peered":"2024-05-10T04:02:35.633396+0000","last_clean":"2024-05-10T04:02:35.633396+0000","last_became_active":"2024-05-10T04:02:31.645673+0000","last_became_peered":"2024-05-10T04:02:31.645673+0000","last_unstale":"2024-05-10T04:02:35.633396+0000","last_undegraded":"2024-05-10T04:02:35.633396+0000","last_fullsized":"2024-05-10T04:02:35.633396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:26:26.814523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632295+0000","last_change":"2024-05-10T04:02:31.630606+0000","last_active":"2024-05-10T04:02:35.632295+0000","last_peered":"2024-05-10T04:02:35.632295+0000","last_clean":"2024-05-10T04:02:35.632295+0000","last_became_active":"2024-05-10T04:02:31.630313+0000","last_became_peered":"2024-05-10T04:02:31.630313+0000","last_unstale":"2024-05-10T04:02:35.632295+0000","last_undegraded":"2024-05-10T04:02:35.632295+0000","last_fullsized":"2024-05-10T04:02:35.632295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:39.876016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.636526+0000","last_change":"2024-05-10T04:02:31.636239+0000","last_active":"2024-05-10T04:02:35.636526+0000","last_peered":"2024-05-10T04:02:35.636526+0000","last_clean":"2024-05-10T04:02:35.636526+0000","last_became_active":"2024-05-10T04:02:31.635377+0000","last_became_peered":"2024-05-10T04:02:31.635377+0000","last_unstale":"2024-05-10T04:02:35.636526+0000","last_undegraded":"2024-05-10T04:02:35.636526+0000","last_fullsized":"2024-05-10T04:02:35.636526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:02.612285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632276+0000","last_change":"2024-05-10T04:02:31.643231+0000","last_active":"2024-05-10T04:02:35.632276+0000","last_peered":"2024-05-10T04:02:35.632276+0000","last_clean":"2024-05-10T04:02:35.632276+0000","last_became_active":"2024-05-10T04:02:31.642797+0000","last_became_peered":"2024-05-10T04:02:31.642797+0000","last_unstale":"2024-05-10T04:02:35.632276+0000","last_undegraded":"2024-05-10T04:02:35.632276+0000","last_fullsized":"2024-05-10T04:02:35.632276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:01:57.365339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196100+0000","last_change":"2024-05-10T04:02:31.633928+0000","last_active":"2024-05-10T04:03:29.196100+0000","last_peered":"2024-05-10T04:03:29.196100+0000","last_clean":"2024-05-10T04:03:29.196100+0000","last_became_active":"2024-05-10T04:02:31.633423+0000","last_became_peered":"2024-05-10T04:02:31.633423+0000","last_unstale":"2024-05-10T04:03:29.196100+0000","last_undegraded":"2024-05-10T04:03:29.196100+0000","last_fullsized":"2024-05-10T04:03:29.196100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:07.011354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.632757+0000","last_change":"2024-05-10T04:02:31.627151+0000","last_active":"2024-05-10T04:02:35.632757+0000","last_peered":"2024-05-10T04:02:35.632757+0000","last_clean":"2024-05-10T04:02:35.632757+0000","last_became_active":"2024-05-10T04:02:31.626780+0000","last_became_peered":"2024-05-10T04:02:31.626780+0000","last_unstale":"2024-05-10T04:02:35.632757+0000","last_undegraded":"2024-05-10T04:02:35.632757+0000","last_fullsized":"2024-05-10T04:02:35.632757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:32.070530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199643+0000","last_change":"2024-05-10T04:02:31.640669+0000","last_active":"2024-05-10T04:03:29.199643+0000","last_peered":"2024-05-10T04:03:29.199643+0000","last_clean":"2024-05-10T04:03:29.199643+0000","last_became_active":"2024-05-10T04:02:31.640410+0000","last_became_peered":"2024-05-10T04:02:31.640410+0000","last_unstale":"2024-05-10T04:03:29.199643+0000","last_undegraded":"2024-05-10T04:03:29.199643+0000","last_fullsized":"2024-05-10T04:03:29.199643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:02.235391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193090+0000","last_change":"2024-05-10T04:02:31.633624+0000","last_active":"2024-05-10T04:03:29.193090+0000","last_peered":"2024-05-10T04:03:29.193090+0000","last_clean":"2024-05-10T04:03:29.193090+0000","last_became_active":"2024-05-10T04:02:31.633241+0000","last_became_peered":"2024-05-10T04:02:31.633241+0000","last_unstale":"2024-05-10T04:03:29.193090+0000","last_undegraded":"2024-05-10T04:03:29.193090+0000","last_fullsized":"2024-05-10T04:03:29.193090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:58.750912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198385+0000","last_change":"2024-05-10T04:02:31.628632+0000","last_active":"2024-05-10T04:03:29.198385+0000","last_peered":"2024-05-10T04:03:29.198385+0000","last_clean":"2024-05-10T04:03:29.198385+0000","last_became_active":"2024-05-10T04:02:31.628349+0000","last_became_peered":"2024-05-10T04:02:31.628349+0000","last_unstale":"2024-05-10T04:03:29.198385+0000","last_undegraded":"2024-05-10T04:03:29.198385+0000","last_fullsized":"2024-05-10T04:03:29.198385+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:32.575168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:02:36.641204+0000","last_change":"2024-05-10T04:02:31.631328+0000","last_active":"2024-05-10T04:02:36.641204+0000","last_peered":"2024-05-10T04:02:36.641204+0000","last_clean":"2024-05-10T04:02:36.641204+0000","last_became_active":"2024-05-10T04:02:31.630957+0000","last_became_peered":"2024-05-10T04:02:31.630957+0000","last_unstale":"2024-05-10T04:02:36.641204+0000","last_undegraded":"2024-05-10T04:02:36.641204+0000","last_fullsized":"2024-05-10T04:02:36.641204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:40.400911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-10T04:02:35.630956+0000","last_change":"2024-05-10T04:02:31.617972+0000","last_active":"2024-05-10T04:02:35.630956+0000","last_peered":"2024-05-10T04:02:35.630956+0000","last_clean":"2024-05-10T04:02:35.630956+0000","last_became_active":"2024-05-10T04:02:31.617663+0000","last_became_peered":"2024-05-10T04:02:31.617663+0000","last_unstale":"2024-05-10T04:02:35.630956+0000","last_undegraded":"2024-05-10T04:02:35.630956+0000","last_fullsized":"2024-05-10T04:02:35.630956+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:50.121955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195144+0000","last_change":"2024-05-10T04:02:31.644256+0000","last_active":"2024-05-10T04:03:29.195144+0000","last_peered":"2024-05-10T04:03:29.195144+0000","last_clean":"2024-05-10T04:03:29.195144+0000","last_became_active":"2024-05-10T04:02:31.643819+0000","last_became_peered":"2024-05-10T04:02:31.643819+0000","last_unstale":"2024-05-10T04:03:29.195144+0000","last_undegraded":"2024-05-10T04:03:29.195144+0000","last_fullsized":"2024-05-10T04:03:29.195144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:54.932547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":28672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279970,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66031,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":224,"num_read_kb":182,"num_write":154,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":198093,"data_compressed":4764,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514519,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":950272,"data_stored":480641,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.51300000000000001}]},{"osd":2,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57999999999999996}]},{"osd":5,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73499999999999999}]},{"osd":8,"last update":"Fri May 10 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68300000000000005}]},{"osd":9,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.97899999999999998}]},{"osd":10,"last update":"Fri May 10 04:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.88400000000000001}]}]},{"osd":10,"up_from":68,"seq":292057776154,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":1060864,"data_stored":1110926,"data_compressed":48861,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":1,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.45100000000000001}]},{"osd":5,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76700000000000002}]},{"osd":6,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":8,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.747},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.66500000000000004}]},{"osd":9,"last update":"Fri May 10 04:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.53100000000000003}]},{"osd":11,"last update":"Fri May 10 04:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.80200000000000005}]}]},{"osd":9,"up_from":64,"seq":274877906974,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":966656,"data_stored":480877,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.435,"15min":0.443},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.46500000000000002,"15min":0.47599999999999998},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.89100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80700000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.61499999999999999,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71399999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.34799999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.52100000000000002,"15min":0.54300000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.58799999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.51200000000000001}]},{"osd":4,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.67100000000000004,"15min":0.68400000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.751,"5min":1.149,"15min":1.149},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.1699999999999999}]},{"osd":7,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.74299999999999999,"15min":0.76300000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2470000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.752,"15min":0.78000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88100000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.60599999999999998}]},{"osd":8,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.53000000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.77000000000000002}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.749}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":8,"up_from":56,"seq":240518168609,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":942080,"data_stored":462019,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":0.83499999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54300000000000004,"15min":0.54800000000000004},"min":{"1min":0.40699999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.65000000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90000000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82099999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.88300000000000001}]},{"osd":2,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48399999999999999,"15min":0.47299999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.47599999999999998},"min":{"1min":0.32700000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.84899999999999998}]},{"osd":4,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.66200000000000003,"15min":0.64600000000000002},"min":{"1min":0.47199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96699999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":1.0920000000000001}]},{"osd":5,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.41299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.82199999999999995}]},{"osd":6,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90000000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.92800000000000005}]},{"osd":7,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.49199999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.55300000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.94199999999999995}]},{"osd":9,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":1.042}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":1.02},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.135}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397541,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1015808,"data_stored":1077005,"data_compressed":53190,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.32100000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.65200000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.45800000000000002,"15min":0.45300000000000001},"min":{"1min":0.30499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.626},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.93999999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75900000000000001}]},{"osd":3,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.443,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84799999999999998,"5min":1.143,"15min":1.143},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.63,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80900000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82199999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95799999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.69599999999999995,"15min":0.67400000000000004},"min":{"1min":0.438,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.5630000000000002,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.54600000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.88200000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.55000000000000004}]},{"osd":8,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.204,"15min":1.286},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88900000000000001,"5min":15.521000000000001,"15min":15.521000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":1.2829999999999999,"15min":1.3660000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0049999999999999,"5min":16.306999999999999,"15min":16.306999999999999},"last":0.82299999999999995}]},{"osd":9,"last update":"Fri May 10 04:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0509999999999999,"5min":1.167,"15min":1.167},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.753,"15min":0.751},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.60399999999999998}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.66300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":6,"up_from":44,"seq":188978561063,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":921600,"data_stored":457275,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.38800000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.38900000000000001},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.377}]},{"osd":1,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.48399999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71299999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.754,"5min":1.087,"15min":1.087},"last":0.624}]},{"osd":3,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.47499999999999998,"15min":0.46700000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.47599999999999998}]},{"osd":4,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88500000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.496}]},{"osd":5,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.63},"min":{"1min":0.44,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93700000000000006,"5min":1.018,"15min":1.018},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri May 10 04:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83299999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.67800000000000005,"15min":0.67300000000000004},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86399999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.57499999999999996}]},{"osd":8,"last update":"Fri May 10 04:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.55000000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.59199999999999997}]},{"osd":10,"last update":"Fri May 10 04:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68000000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":995328,"data_stored":528426,"data_compressed":16207,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80700000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56100000000000005,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.58199999999999996,"15min":0.53700000000000003},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.56299999999999994,"15min":0.52300000000000002},"min":{"1min":0.41599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri May 10 04:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.53800000000000003,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.54800000000000004,"15min":0.53100000000000003},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.442}]},{"osd":4,"last update":"Fri May 10 04:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.46800000000000003,"15min":0.45300000000000001},"min":{"1min":0.33900000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65300000000000002,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.54600000000000004,"15min":0.51600000000000001},"min":{"1min":0.35099999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri May 10 04:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.62},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.56499999999999995,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.496}]},{"osd":6,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.63200000000000001,"15min":0.61499999999999999},"min":{"1min":0.55500000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.56899999999999995}]},{"osd":7,"last update":"Fri May 10 04:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.58899999999999997,"15min":0.57399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.37},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.433,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.47399999999999998}]},{"osd":8,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69999999999999996,"15min":0.68799999999999994},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.55100000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.46300000000000002}]},{"osd":9,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.73099999999999998}]},{"osd":10,"last update":"Fri May 10 04:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.69099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":458663,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68100000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.71999999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.62,"15min":0.55000000000000004},"min":{"1min":0.443,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.9009999999999998,"5min":4.9009999999999998,"15min":4.9009999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82499999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69899999999999995}]},{"osd":3,"last update":"Fri May 10 04:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40799999999999997},"min":{"1min":0.26000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.58599999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.82699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri May 10 04:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75800000000000001,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72599999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.627}]},{"osd":5,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79700000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.64800000000000002,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri May 10 04:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54300000000000004}]},{"osd":7,"last update":"Fri May 10 04:01:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.748},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.84599999999999997}]},{"osd":8,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.67000000000000004,"15min":0.67800000000000005},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85099999999999998,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86299999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.80200000000000005}]},{"osd":9,"last update":"Fri May 10 04:02:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.83499999999999996}]},{"osd":10,"last update":"Fri May 10 04:02:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.91200000000000003}]},{"osd":11,"last update":"Fri May 10 04:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.76400000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411382,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":1007616,"data_stored":520611,"data_compressed":13252,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.54200000000000004,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.63},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56799999999999995,"15min":0.53500000000000003},"min":{"1min":0.41799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.59799999999999998,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri May 10 04:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58899999999999997,"15min":0.57199999999999995},"min":{"1min":0.47899999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.44400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.78900000000000003}]},{"osd":4,"last update":"Fri May 10 04:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.41799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.318,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Fri May 10 04:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.51700000000000002,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79000000000000004,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.40799999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.83099999999999996}]},{"osd":6,"last update":"Fri May 10 04:02:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.44400000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.84899999999999998}]},{"osd":7,"last update":"Fri May 10 04:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.89700000000000002}]},{"osd":8,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.92900000000000005}]},{"osd":9,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.74299999999999999,"15min":0.72299999999999998},"min":{"1min":0.66800000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0189999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":1.01},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.59199999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.91900000000000004,"5min":1.016,"15min":1.016},"last":0.95599999999999996}]},{"osd":10,"last update":"Fri May 10 04:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.96699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":476247,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.35399999999999998},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.46400000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.437},{"interface":"front","average":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36099999999999999},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.41499999999999998}]},{"osd":1,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.505,"15min":0.51100000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.59999999999999998,"5min":0.747,"15min":0.747},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.374,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69799999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69899999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78000000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri May 10 04:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.70399999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri May 10 04:03:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80100000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.59199999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78800000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri May 10 04:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri May 10 04:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.80500000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68899999999999995}]},{"osd":8,"last update":"Fri May 10 04:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.439}]},{"osd":9,"last update":"Fri May 10 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69599999999999995,"15min":0.71199999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89900000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83999999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55200000000000005}]},{"osd":10,"last update":"Fri May 10 04:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.755}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953519,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":462270,"data_compressed":11664,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.33900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50900000000000001,"15min":0.502},"min":{"1min":0.371,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.79000000000000004}]},{"osd":1,"last update":"Fri May 10 04:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.39000000000000001},"min":{"1min":0.29599999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.66400000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.499},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.40899999999999997,"15min":0.38800000000000001},"min":{"1min":0.31900000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.65200000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri May 10 04:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60399999999999998,"15min":0.56399999999999995},"min":{"1min":0.46600000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.95099999999999996,"5min":1.046,"15min":1.046},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75900000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.628}]},{"osd":3,"last update":"Fri May 10 04:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56999999999999995,"15min":0.54100000000000004},"min":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.47499999999999998}]},{"osd":5,"last update":"Fri May 10 04:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.46400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.92300000000000004,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81699999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri May 10 04:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.47799999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70899999999999996,"15min":0.69999999999999996},"min":{"1min":0.503,"5min":0.439,"15min":0.439},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96499999999999997,"5min":1.01,"15min":1.01},"last":0.69099999999999995}]},{"osd":8,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68100000000000005,"15min":0.68500000000000005},"min":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.79900000000000004,"5min":0.879,"15min":0.879},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.56899999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.81499999999999995}]},{"osd":9,"last update":"Fri May 10 04:03:26 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.80000000000000004,"15min":0.76700000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.6449999999999996,"5min":5.6449999999999996,"15min":5.6449999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.82899999999999996},"min":{"1min":0.54500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":5.6100000000000003,"5min":5.6100000000000003,"15min":5.6100000000000003},"last":0.54500000000000004}]},{"osd":10,"last update":"Fri May 10 04:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.78200000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]},{"osd":5,"up_from":37,"seq":158913789995,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":1019904,"data_stored":1071917,"data_compressed":53190,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52400000000000002,"15min":0.498},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77800000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.433,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74299999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57499999999999996,"15min":0.55100000000000005},"min":{"1min":0.45400000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.47999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.40400000000000003},"min":{"1min":0.35799999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.495},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.44,"15min":0.42199999999999999},"min":{"1min":0.37,"5min":0.249,"15min":0.249},"max":{"1min":0.66100000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80500000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.74299999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61099999999999999,"15min":0.59199999999999997},"min":{"1min":0.48199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83399999999999996}]},{"osd":6,"last update":"Fri May 10 04:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.439,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.48199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.626}]},{"osd":7,"last update":"Fri May 10 04:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.69099999999999995,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.54400000000000004,"5min":0.505,"15min":0.505},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.85399999999999998}]},{"osd":8,"last update":"Fri May 10 04:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91900000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.46000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.873},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.55000000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri May 10 04:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.80400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61731,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63426,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63654,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:03:33.809 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-10T04:03:33.810 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-10T04:03:33.810 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-10T04:03:33.810 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-10T04:03:34.032 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:34.447 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:34 smithi150 ceph-mon[42477]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:03:34.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:34 smithi181 ceph-mon[46602]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:03:34.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:34 smithi161 ceph-mon[46918]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:03:35.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.014+0000 7f5b370ab700 1 -- 172.21.15.150:0/3418551970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b30100e50 msgr2=0x7f5b30101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:35.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.014+0000 7f5b370ab700 1 --2- 172.21.15.150:0/3418551970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b30100e50 0x7f5b30101260 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800b0a0 tx=0x7f5b2800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.014+0000 7f5b370ab700 1 -- 172.21.15.150:0/3418551970 shutdown_connections 2024-05-10T04:03:35.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.014+0000 7f5b370ab700 1 --2- 172.21.15.150:0/3418551970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b301060d0 0x7f5b3010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.014+0000 7f5b370ab700 1 --2- 172.21.15.150:0/3418551970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b30100e50 0x7f5b30101260 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.014+0000 7f5b370ab700 1 --2- 172.21.15.150:0/3418551970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b30105780 0x7f5b30105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.014+0000 7f5b370ab700 1 -- 172.21.15.150:0/3418551970 >> 172.21.15.150:0/3418551970 conn(0x7f5b300fc7c0 msgr2=0x7f5b300febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:35.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.014+0000 7f5b370ab700 1 -- 172.21.15.150:0/3418551970 shutdown_connections 2024-05-10T04:03:35.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.014+0000 7f5b370ab700 1 -- 172.21.15.150:0/3418551970 wait complete. 2024-05-10T04:03:35.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.014+0000 7f5b370ab700 1 Processor -- start 2024-05-10T04:03:35.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.014+0000 7f5b370ab700 1 -- start start 2024-05-10T04:03:35.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b370ab700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b30100e50 0x7f5b30199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:35.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b370ab700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b30105780 0x7f5b30199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:35.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b370ab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b301060d0 0x7f5b301a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b370ab700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3019a2f0 con 0x7f5b30100e50 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b370ab700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b301a4a30 con 0x7f5b30105780 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b370ab700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b301a4bf0 con 0x7f5b301060d0 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b34e47700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b30100e50 0x7f5b30199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b34e47700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b30100e50 0x7f5b30199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41884/0 (socket says 172.21.15.150:41884) 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b34e47700 1 -- 172.21.15.150:0/1476336997 learned_addr learned my addr 172.21.15.150:0/1476336997 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b35648700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b301060d0 0x7f5b301a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b2ffff700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b30105780 0x7f5b30199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b34e47700 1 -- 172.21.15.150:0/1476336997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b301060d0 msgr2=0x7f5b301a44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b34e47700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b301060d0 0x7f5b301a44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b34e47700 1 -- 172.21.15.150:0/1476336997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b30105780 msgr2=0x7f5b30199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b34e47700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b30105780 0x7f5b30199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b34e47700 1 -- 172.21.15.150:0/1476336997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28009d00 con 0x7f5b30100e50 2024-05-10T04:03:35.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.015+0000 7f5b34e47700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b30100e50 0x7f5b30199640 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5b280062a0 tx=0x7f5b28007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:35.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.016+0000 7f5b2dffb700 1 -- 172.21.15.150:0/1476336997 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800e040 con 0x7f5b30100e50 2024-05-10T04:03:35.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.016+0000 7f5b370ab700 1 -- 172.21.15.150:0/1476336997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b301a4e70 con 0x7f5b30100e50 2024-05-10T04:03:35.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.016+0000 7f5b2dffb700 1 -- 172.21.15.150:0/1476336997 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b28004c60 con 0x7f5b30100e50 2024-05-10T04:03:35.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.016+0000 7f5b2dffb700 1 -- 172.21.15.150:0/1476336997 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b280052a0 con 0x7f5b30100e50 2024-05-10T04:03:35.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.016+0000 7f5b370ab700 1 -- 172.21.15.150:0/1476336997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b301a5290 con 0x7f5b30100e50 2024-05-10T04:03:35.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.017+0000 7f5b370ab700 1 -- 172.21.15.150:0/1476336997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b30109090 con 0x7f5b30100e50 2024-05-10T04:03:35.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.018+0000 7f5b2dffb700 1 -- 172.21.15.150:0/1476336997 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5b28003a00 con 0x7f5b30100e50 2024-05-10T04:03:35.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.018+0000 7f5b2dffb700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5b1c0a11b0 0x7f5b1c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:35.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.019+0000 7f5b2ffff700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5b1c0a11b0 0x7f5b1c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:35.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.019+0000 7f5b2dffb700 1 -- 172.21.15.150:0/1476336997 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5b280ce4e0 con 0x7f5b30100e50 2024-05-10T04:03:35.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.019+0000 7f5b2ffff700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5b1c0a11b0 0x7f5b1c0a36c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5b300ffcf0 tx=0x7f5b1800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:35.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.022+0000 7f5b2dffb700 1 -- 172.21.15.150:0/1476336997 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5b28097890 con 0x7f5b30100e50 2024-05-10T04:03:35.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.264+0000 7f5b370ab700 1 -- 172.21.15.150:0/1476336997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5b30102300 con 0x7f5b30100e50 2024-05-10T04:03:35.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.264+0000 7f5b2dffb700 1 -- 172.21.15.150:0/1476336997 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5b28003cb0 con 0x7f5b30100e50 2024-05-10T04:03:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:35.268 INFO:teuthology.orchestra.run.smithi150.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-10T04:03:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.266+0000 7f5b370ab700 1 -- 172.21.15.150:0/1476336997 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5b1c0a11b0 msgr2=0x7f5b1c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.266+0000 7f5b370ab700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5b1c0a11b0 0x7f5b1c0a36c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5b300ffcf0 tx=0x7f5b1800b410 comp rx=0 tx=0).stop 2024-05-10T04:03:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.266+0000 7f5b370ab700 1 -- 172.21.15.150:0/1476336997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b30100e50 msgr2=0x7f5b30199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.266+0000 7f5b370ab700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b30100e50 0x7f5b30199640 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5b280062a0 tx=0x7f5b28007870 comp rx=0 tx=0).stop 2024-05-10T04:03:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.266+0000 7f5b370ab700 1 -- 172.21.15.150:0/1476336997 shutdown_connections 2024-05-10T04:03:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.266+0000 7f5b370ab700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5b1c0a11b0 0x7f5b1c0a36c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.266+0000 7f5b370ab700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b301060d0 0x7f5b301a44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.266+0000 7f5b370ab700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b30100e50 0x7f5b30199640 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.266+0000 7f5b370ab700 1 --2- 172.21.15.150:0/1476336997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b30105780 0x7f5b30199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.266+0000 7f5b370ab700 1 -- 172.21.15.150:0/1476336997 >> 172.21.15.150:0/1476336997 conn(0x7f5b300fc7c0 msgr2=0x7f5b30104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.266+0000 7f5b370ab700 1 -- 172.21.15.150:0/1476336997 shutdown_connections 2024-05-10T04:03:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.267+0000 7f5b370ab700 1 -- 172.21.15.150:0/1476336997 wait complete. 2024-05-10T04:03:35.566 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:35 smithi150 ceph-mon[42477]: pgmap v260: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 28 B/s, 1 keys/s, 1 objects/s recovering 2024-05-10T04:03:35.567 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:35 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1476336997' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T04:03:35.646 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-10T04:03:35.646 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:03:35.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:35 smithi181 ceph-mon[46602]: pgmap v260: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 28 B/s, 1 keys/s, 1 objects/s recovering 2024-05-10T04:03:35.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:35 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1476336997' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T04:03:35.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:35 smithi161 ceph-mon[46918]: pgmap v260: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 28 B/s, 1 keys/s, 1 objects/s recovering 2024-05-10T04:03:35.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:35 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1476336997' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T04:03:35.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.780+0000 7f39e4aed700 1 -- 172.21.15.150:0/2075607697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39e0102580 msgr2=0x7f39e0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:35.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.780+0000 7f39e4aed700 1 --2- 172.21.15.150:0/2075607697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39e0102580 0x7f39e0108e30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f39d800b0a0 tx=0x7f39d800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.780+0000 7f39e4aed700 1 -- 172.21.15.150:0/2075607697 shutdown_connections 2024-05-10T04:03:35.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.780+0000 7f39e4aed700 1 --2- 172.21.15.150:0/2075607697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39e0102580 0x7f39e0108e30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.780+0000 7f39e4aed700 1 --2- 172.21.15.150:0/2075607697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e0100ac0 0x7f39e0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.780+0000 7f39e4aed700 1 --2- 172.21.15.150:0/2075607697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e0101be0 0x7f39e0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.780+0000 7f39e4aed700 1 -- 172.21.15.150:0/2075607697 >> 172.21.15.150:0/2075607697 conn(0x7f39e00fc0d0 msgr2=0x7f39e00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:35.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.781+0000 7f39e4aed700 1 -- 172.21.15.150:0/2075607697 shutdown_connections 2024-05-10T04:03:35.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.781+0000 7f39e4aed700 1 -- 172.21.15.150:0/2075607697 wait complete. 2024-05-10T04:03:35.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.781+0000 7f39e4aed700 1 Processor -- start 2024-05-10T04:03:35.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.781+0000 7f39e4aed700 1 -- start start 2024-05-10T04:03:35.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.781+0000 7f39e4aed700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e0100ac0 0x7f39e0068e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:35.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.781+0000 7f39e4aed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e0101be0 0x7f39e019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:35.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39e4aed700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39e0102580 0x7f39e019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:35.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39e4aed700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e019fb30 con 0x7f39e0102580 2024-05-10T04:03:35.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39e4aed700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e019fca0 con 0x7f39e0100ac0 2024-05-10T04:03:35.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39e4aed700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e019fe60 con 0x7f39e0101be0 2024-05-10T04:03:35.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39ded9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39e0102580 0x7f39e019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:35.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39ded9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39e0102580 0x7f39e019f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41914/0 (socket says 172.21.15.150:41914) 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39ded9d700 1 -- 172.21.15.150:0/2905413625 learned_addr learned my addr 172.21.15.150:0/2905413625 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39ddd9b700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e0101be0 0x7f39e019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39de59c700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e0100ac0 0x7f39e0068e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39ded9d700 1 -- 172.21.15.150:0/2905413625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e0101be0 msgr2=0x7f39e019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39ded9d700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e0101be0 0x7f39e019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39ded9d700 1 -- 172.21.15.150:0/2905413625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e0100ac0 msgr2=0x7f39e0068e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39ded9d700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e0100ac0 0x7f39e0068e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39ded9d700 1 -- 172.21.15.150:0/2905413625 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d8009d00 con 0x7f39e0102580 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39ded9d700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39e0102580 0x7f39e019f430 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f39d800b070 tx=0x7f39d80079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39cf7fe700 1 -- 172.21.15.150:0/2905413625 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d800e040 con 0x7f39e0102580 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39cf7fe700 1 -- 172.21.15.150:0/2905413625 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39d8003c00 con 0x7f39e0102580 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39e4aed700 1 -- 172.21.15.150:0/2905413625 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e01a00e0 con 0x7f39e0102580 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39cf7fe700 1 -- 172.21.15.150:0/2905413625 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d801b6c0 con 0x7f39e0102580 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.782+0000 7f39e4aed700 1 -- 172.21.15.150:0/2905413625 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e01a0560 con 0x7f39e0102580 2024-05-10T04:03:35.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.783+0000 7f39e4aed700 1 -- 172.21.15.150:0/2905413625 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e0107cd0 con 0x7f39e0102580 2024-05-10T04:03:35.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.784+0000 7f39cf7fe700 1 -- 172.21.15.150:0/2905413625 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f39d801d470 con 0x7f39e0102580 2024-05-10T04:03:35.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.785+0000 7f39cf7fe700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f39c80a1230 0x7f39c80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:35.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.785+0000 7f39cf7fe700 1 -- 172.21.15.150:0/2905413625 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f39d80ce000 con 0x7f39e0102580 2024-05-10T04:03:35.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.785+0000 7f39de59c700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f39c80a1230 0x7f39c80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:35.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.786+0000 7f39de59c700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f39c80a1230 0x7f39c80a3740 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f39d4005950 tx=0x7f39d40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:35.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.789+0000 7f39cf7fe700 1 -- 172.21.15.150:0/2905413625 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f39d8097430 con 0x7f39e0102580 2024-05-10T04:03:35.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.988+0000 7f39e4aed700 1 -- 172.21.15.150:0/2905413625 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39e010c2d0 con 0x7f39e0102580 2024-05-10T04:03:35.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.989+0000 7f39cf7fe700 1 -- 172.21.15.150:0/2905413625 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24554 (secure 0 0 0) 0x7f39d809aa50 con 0x7f39e0102580 2024-05-10T04:03:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:35.992 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":87,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:03:30.185138+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:03:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.992+0000 7f39e4aed700 1 -- 172.21.15.150:0/2905413625 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f39c80a1230 msgr2=0x7f39c80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.992+0000 7f39e4aed700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f39c80a1230 0x7f39c80a3740 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f39d4005950 tx=0x7f39d40058e0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.992+0000 7f39e4aed700 1 -- 172.21.15.150:0/2905413625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39e0102580 msgr2=0x7f39e019f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.992+0000 7f39e4aed700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39e0102580 0x7f39e019f430 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f39d800b070 tx=0x7f39d80079b0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.992+0000 7f39e4aed700 1 -- 172.21.15.150:0/2905413625 shutdown_connections 2024-05-10T04:03:35.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.992+0000 7f39e4aed700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f39c80a1230 0x7f39c80a3740 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.992+0000 7f39e4aed700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39e0102580 0x7f39e019f430 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.992+0000 7f39e4aed700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e0100ac0 0x7f39e0068e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.992+0000 7f39e4aed700 1 --2- 172.21.15.150:0/2905413625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39e0101be0 0x7f39e019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:35.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.992+0000 7f39e4aed700 1 -- 172.21.15.150:0/2905413625 >> 172.21.15.150:0/2905413625 conn(0x7f39e00fc0d0 msgr2=0x7f39e0104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:35.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.992+0000 7f39e4aed700 1 -- 172.21.15.150:0/2905413625 shutdown_connections 2024-05-10T04:03:35.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:35.992+0000 7f39e4aed700 1 -- 172.21.15.150:0/2905413625 wait complete. 2024-05-10T04:03:36.008 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:02:30.076694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:02:30.760710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:02:33.850714+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:03:36.008 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:03:36.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.144+0000 7fb095391700 1 -- 172.21.15.150:0/725776088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0900fff30 msgr2=0x7fb090100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:36.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.144+0000 7fb095391700 1 --2- 172.21.15.150:0/725776088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0900fff30 0x7fb090100340 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb084009990 tx=0x7fb084009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.144+0000 7fb095391700 1 -- 172.21.15.150:0/725776088 shutdown_connections 2024-05-10T04:03:36.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.144+0000 7fb095391700 1 --2- 172.21.15.150:0/725776088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb090107a00 0x7fb090109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.144+0000 7fb095391700 1 --2- 172.21.15.150:0/725776088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0900fff30 0x7fb090100340 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.144+0000 7fb095391700 1 --2- 172.21.15.150:0/725776088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0901050a0 0x7fb0901074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.144+0000 7fb095391700 1 -- 172.21.15.150:0/725776088 >> 172.21.15.150:0/725776088 conn(0x7fb0900fb880 msgr2=0x7fb0900fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:36.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.144+0000 7fb095391700 1 -- 172.21.15.150:0/725776088 shutdown_connections 2024-05-10T04:03:36.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.144+0000 7fb095391700 1 -- 172.21.15.150:0/725776088 wait complete. 2024-05-10T04:03:36.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.144+0000 7fb095391700 1 Processor -- start 2024-05-10T04:03:36.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.145+0000 7fb095391700 1 -- start start 2024-05-10T04:03:36.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.145+0000 7fb095391700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0900fff30 0x7fb090197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:36.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.145+0000 7fb095391700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0901050a0 0x7fb090198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:36.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.145+0000 7fb095391700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb090107a00 0x7fb09019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:36.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.145+0000 7fb095391700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09019f9a0 con 0x7fb0901050a0 2024-05-10T04:03:36.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.145+0000 7fb095391700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09019fb10 con 0x7fb0900fff30 2024-05-10T04:03:36.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.145+0000 7fb095391700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09019fcd0 con 0x7fb090107a00 2024-05-10T04:03:36.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.145+0000 7fb08e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0901050a0 0x7fb090198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:36.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.145+0000 7fb08e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0901050a0 0x7fb090198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41936/0 (socket says 172.21.15.150:41936) 2024-05-10T04:03:36.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.145+0000 7fb08e7fc700 1 -- 172.21.15.150:0/982682794 learned_addr learned my addr 172.21.15.150:0/982682794 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:36.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.145+0000 7fb08effd700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0900fff30 0x7fb090197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:36.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.145+0000 7fb08f7fe700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb090107a00 0x7fb09019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:36.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.146+0000 7fb08e7fc700 1 -- 172.21.15.150:0/982682794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb090107a00 msgr2=0x7fb09019f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:36.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.146+0000 7fb08e7fc700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb090107a00 0x7fb09019f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.146+0000 7fb08e7fc700 1 -- 172.21.15.150:0/982682794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0900fff30 msgr2=0x7fb090197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:36.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.146+0000 7fb08e7fc700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0900fff30 0x7fb090197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.146+0000 7fb08e7fc700 1 -- 172.21.15.150:0/982682794 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0840095f0 con 0x7fb0901050a0 2024-05-10T04:03:36.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.146+0000 7fb08f7fe700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb090107a00 0x7fb09019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:36.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.146+0000 7fb08e7fc700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0901050a0 0x7fb090198400 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb08000d950 tx=0x7fb08000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:36.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.146+0000 7fb077fff700 1 -- 172.21.15.150:0/982682794 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080009940 con 0x7fb0901050a0 2024-05-10T04:03:36.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.146+0000 7fb077fff700 1 -- 172.21.15.150:0/982682794 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb080010460 con 0x7fb0901050a0 2024-05-10T04:03:36.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.146+0000 7fb095391700 1 -- 172.21.15.150:0/982682794 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09019ffb0 con 0x7fb0901050a0 2024-05-10T04:03:36.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.146+0000 7fb077fff700 1 -- 172.21.15.150:0/982682794 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08000f620 con 0x7fb0901050a0 2024-05-10T04:03:36.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.146+0000 7fb095391700 1 -- 172.21.15.150:0/982682794 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0901a03e0 con 0x7fb0901050a0 2024-05-10T04:03:36.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.147+0000 7fb095391700 1 -- 172.21.15.150:0/982682794 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0900fd430 con 0x7fb0901050a0 2024-05-10T04:03:36.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.149+0000 7fb077fff700 1 -- 172.21.15.150:0/982682794 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb0800108d0 con 0x7fb0901050a0 2024-05-10T04:03:36.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.149+0000 7fb077fff700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0780a1260 0x7fb0780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:36.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.149+0000 7fb077fff700 1 -- 172.21.15.150:0/982682794 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb0800ccc20 con 0x7fb0901050a0 2024-05-10T04:03:36.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.149+0000 7fb08effd700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0780a1260 0x7fb0780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:36.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.150+0000 7fb08effd700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0780a1260 0x7fb0780a3770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb084003730 tx=0x7fb084005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:36.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.153+0000 7fb077fff700 1 -- 172.21.15.150:0/982682794 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb080095f40 con 0x7fb0901050a0 2024-05-10T04:03:36.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.351+0000 7fb095391700 1 -- 172.21.15.150:0/982682794 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb09010d3b0 con 0x7fb0901050a0 2024-05-10T04:03:36.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.351+0000 7fb077fff700 1 -- 172.21.15.150:0/982682794 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0800160a0 con 0x7fb0901050a0 2024-05-10T04:03:36.354 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:03:36.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.354+0000 7fb095391700 1 -- 172.21.15.150:0/982682794 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0780a1260 msgr2=0x7fb0780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:36.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.354+0000 7fb095391700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0780a1260 0x7fb0780a3770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb084003730 tx=0x7fb084005fb0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.354+0000 7fb095391700 1 -- 172.21.15.150:0/982682794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0901050a0 msgr2=0x7fb090198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:36.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.354+0000 7fb095391700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0901050a0 0x7fb090198400 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb08000d950 tx=0x7fb08000dc60 comp rx=0 tx=0).stop 2024-05-10T04:03:36.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.354+0000 7fb095391700 1 -- 172.21.15.150:0/982682794 shutdown_connections 2024-05-10T04:03:36.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.354+0000 7fb095391700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb0780a1260 0x7fb0780a3770 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.354+0000 7fb095391700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb090107a00 0x7fb09019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.354+0000 7fb095391700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0900fff30 0x7fb090197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.354+0000 7fb095391700 1 --2- 172.21.15.150:0/982682794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0901050a0 0x7fb090198400 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.354+0000 7fb095391700 1 -- 172.21.15.150:0/982682794 >> 172.21.15.150:0/982682794 conn(0x7fb0900fb880 msgr2=0x7fb090105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:36.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.354+0000 7fb095391700 1 -- 172.21.15.150:0/982682794 shutdown_connections 2024-05-10T04:03:36.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.354+0000 7fb095391700 1 -- 172.21.15.150:0/982682794 wait complete. 2024-05-10T04:03:36.369 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:03:36.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.503+0000 7f70a1b5c700 1 -- 172.21.15.150:0/172778472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c102580 msgr2=0x7f709c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:36.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.503+0000 7f70a1b5c700 1 --2- 172.21.15.150:0/172778472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c102580 0x7f709c108e30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f709400b0a0 tx=0x7f709400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.503+0000 7f70a1b5c700 1 -- 172.21.15.150:0/172778472 shutdown_connections 2024-05-10T04:03:36.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.503+0000 7f70a1b5c700 1 --2- 172.21.15.150:0/172778472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c102580 0x7f709c108e30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.503+0000 7f70a1b5c700 1 --2- 172.21.15.150:0/172778472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f709c100ac0 0x7f709c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.503+0000 7f70a1b5c700 1 --2- 172.21.15.150:0/172778472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f709c101be0 0x7f709c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.503+0000 7f70a1b5c700 1 -- 172.21.15.150:0/172778472 >> 172.21.15.150:0/172778472 conn(0x7f709c0fc0d0 msgr2=0x7f709c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:36.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.503+0000 7f70a1b5c700 1 -- 172.21.15.150:0/172778472 shutdown_connections 2024-05-10T04:03:36.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.503+0000 7f70a1b5c700 1 -- 172.21.15.150:0/172778472 wait complete. 2024-05-10T04:03:36.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.504+0000 7f70a1b5c700 1 Processor -- start 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.504+0000 7f70a1b5c700 1 -- start start 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.504+0000 7f70a1b5c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f709c100ac0 0x7f709c068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.504+0000 7f70a1b5c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f709c101be0 0x7f709c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.504+0000 7f70a1b5c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c102580 0x7f709c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.504+0000 7f70a1b5c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c19fab0 con 0x7f709c102580 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.504+0000 7f70a1b5c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c19fc20 con 0x7f709c100ac0 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.504+0000 7f70a1b5c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c19fde0 con 0x7f709c101be0 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f709bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c102580 0x7f709c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f709bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c102580 0x7f709c19f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41964/0 (socket says 172.21.15.150:41964) 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f709bfff700 1 -- 172.21.15.150:0/913649106 learned_addr learned my addr 172.21.15.150:0/913649106 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f709b7fe700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f709c100ac0 0x7f709c068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f709affd700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f709c101be0 0x7f709c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f709bfff700 1 -- 172.21.15.150:0/913649106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f709c101be0 msgr2=0x7f709c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:36.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f709bfff700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f709c101be0 0x7f709c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f709bfff700 1 -- 172.21.15.150:0/913649106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f709c100ac0 msgr2=0x7f709c068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:36.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f709bfff700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f709c100ac0 0x7f709c068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f709bfff700 1 -- 172.21.15.150:0/913649106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7094009d00 con 0x7f709c102580 2024-05-10T04:03:36.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f709bfff700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c102580 0x7f709c19f3b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7094006070 tx=0x7f7094009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:36.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f7098ff9700 1 -- 172.21.15.150:0/913649106 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709400e040 con 0x7f709c102580 2024-05-10T04:03:36.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f70a1b5c700 1 -- 172.21.15.150:0/913649106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c1a0060 con 0x7f709c102580 2024-05-10T04:03:36.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f7098ff9700 1 -- 172.21.15.150:0/913649106 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7094004ce0 con 0x7f709c102580 2024-05-10T04:03:36.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f7098ff9700 1 -- 172.21.15.150:0/913649106 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709401d480 con 0x7f709c102580 2024-05-10T04:03:36.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.505+0000 7f70a1b5c700 1 -- 172.21.15.150:0/913649106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c1a04e0 con 0x7f709c102580 2024-05-10T04:03:36.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.506+0000 7f70a1b5c700 1 -- 172.21.15.150:0/913649106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709c107cd0 con 0x7f709c102580 2024-05-10T04:03:36.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.508+0000 7f7098ff9700 1 -- 172.21.15.150:0/913649106 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7094003a00 con 0x7f709c102580 2024-05-10T04:03:36.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.508+0000 7f7098ff9700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f70880a1260 0x7f70880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:36.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.508+0000 7f7098ff9700 1 -- 172.21.15.150:0/913649106 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f70940ce730 con 0x7f709c102580 2024-05-10T04:03:36.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.508+0000 7f709b7fe700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f70880a1260 0x7f70880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:36.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.509+0000 7f709b7fe700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f70880a1260 0x7f70880a3770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f709c101ae0 tx=0x7f708c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:36.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.512+0000 7f7098ff9700 1 -- 172.21.15.150:0/913649106 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7094097ae0 con 0x7f709c102580 2024-05-10T04:03:36.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.709+0000 7f70a1b5c700 1 -- 172.21.15.150:0/913649106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f709c10c2f0 con 0x7f709c102580 2024-05-10T04:03:36.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.709+0000 7f7098ff9700 1 -- 172.21.15.150:0/913649106 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f709409b100 con 0x7f709c102580 2024-05-10T04:03:36.712 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:03:36.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:36 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2905413625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:36.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:36 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/982682794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:03:36.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.712+0000 7f70a1b5c700 1 -- 172.21.15.150:0/913649106 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f70880a1260 msgr2=0x7f70880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:36.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.712+0000 7f70a1b5c700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f70880a1260 0x7f70880a3770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f709c101ae0 tx=0x7f708c00b410 comp rx=0 tx=0).stop 2024-05-10T04:03:36.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.712+0000 7f70a1b5c700 1 -- 172.21.15.150:0/913649106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c102580 msgr2=0x7f709c19f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:36.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.712+0000 7f70a1b5c700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c102580 0x7f709c19f3b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7094006070 tx=0x7f7094009440 comp rx=0 tx=0).stop 2024-05-10T04:03:36.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.712+0000 7f70a1b5c700 1 -- 172.21.15.150:0/913649106 shutdown_connections 2024-05-10T04:03:36.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.712+0000 7f70a1b5c700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f70880a1260 0x7f70880a3770 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.712+0000 7f70a1b5c700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c102580 0x7f709c19f3b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.712+0000 7f70a1b5c700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f709c100ac0 0x7f709c068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.712+0000 7f70a1b5c700 1 --2- 172.21.15.150:0/913649106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f709c101be0 0x7f709c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.712+0000 7f70a1b5c700 1 -- 172.21.15.150:0/913649106 >> 172.21.15.150:0/913649106 conn(0x7f709c0fc0d0 msgr2=0x7f709c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:36.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.712+0000 7f70a1b5c700 1 -- 172.21.15.150:0/913649106 shutdown_connections 2024-05-10T04:03:36.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.712+0000 7f70a1b5c700 1 -- 172.21.15.150:0/913649106 wait complete. 2024-05-10T04:03:36.727 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:03:36.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:36 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2905413625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:36.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:36 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/982682794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:03:36.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:36 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2905413625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:36.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:36 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/982682794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:03:36.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.860+0000 7f1ed5337700 1 -- 172.21.15.150:0/2583157077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed0102590 msgr2=0x7f1ed0108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:36.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.860+0000 7f1ed5337700 1 --2- 172.21.15.150:0/2583157077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed0102590 0x7f1ed0108e40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800b0a0 tx=0x7f1ec800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.860+0000 7f1ed5337700 1 -- 172.21.15.150:0/2583157077 shutdown_connections 2024-05-10T04:03:36.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.860+0000 7f1ed5337700 1 --2- 172.21.15.150:0/2583157077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed0102590 0x7f1ed0108e40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.860+0000 7f1ed5337700 1 --2- 172.21.15.150:0/2583157077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ed0100ad0 0x7f1ed0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.860+0000 7f1ed5337700 1 --2- 172.21.15.150:0/2583157077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ed0101bf0 0x7f1ed0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.860+0000 7f1ed5337700 1 -- 172.21.15.150:0/2583157077 >> 172.21.15.150:0/2583157077 conn(0x7f1ed00fc0e0 msgr2=0x7f1ed00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:36.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.860+0000 7f1ed5337700 1 -- 172.21.15.150:0/2583157077 shutdown_connections 2024-05-10T04:03:36.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.860+0000 7f1ed5337700 1 -- 172.21.15.150:0/2583157077 wait complete. 2024-05-10T04:03:36.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.861+0000 7f1ed5337700 1 Processor -- start 2024-05-10T04:03:36.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.861+0000 7f1ed5337700 1 -- start start 2024-05-10T04:03:36.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.861+0000 7f1ed5337700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed0100ad0 0x7f1ed01980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:36.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.861+0000 7f1ed5337700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ed0101bf0 0x7f1ed01985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:36.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.861+0000 7f1ed5337700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ed0102590 0x7f1ed019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:36.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.861+0000 7f1ed5337700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed019fb80 con 0x7f1ed0100ad0 2024-05-10T04:03:36.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.861+0000 7f1ed5337700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed019fcf0 con 0x7f1ed0101bf0 2024-05-10T04:03:36.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.861+0000 7f1ed5337700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed019feb0 con 0x7f1ed0102590 2024-05-10T04:03:36.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.861+0000 7f1eceffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed0100ad0 0x7f1ed01980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:36.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1eceffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed0100ad0 0x7f1ed01980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41974/0 (socket says 172.21.15.150:41974) 2024-05-10T04:03:36.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1eceffd700 1 -- 172.21.15.150:0/1961368384 learned_addr learned my addr 172.21.15.150:0/1961368384 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1ece7fc700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ed0101bf0 0x7f1ed01985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1ecf7fe700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ed0102590 0x7f1ed019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1eceffd700 1 -- 172.21.15.150:0/1961368384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ed0102590 msgr2=0x7f1ed019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1eceffd700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ed0102590 0x7f1ed019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1eceffd700 1 -- 172.21.15.150:0/1961368384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ed0101bf0 msgr2=0x7f1ed01985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1eceffd700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ed0101bf0 0x7f1ed01985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1eceffd700 1 -- 172.21.15.150:0/1961368384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec8009d00 con 0x7f1ed0100ad0 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1ecf7fe700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ed0102590 0x7f1ed019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1eceffd700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed0100ad0 0x7f1ed01980a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400c970 tx=0x7f1ec400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1eb7fff700 1 -- 172.21.15.150:0/1961368384 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4004220 con 0x7f1ed0100ad0 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1eb7fff700 1 -- 172.21.15.150:0/1961368384 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ec400ce60 con 0x7f1ed0100ad0 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1ed5337700 1 -- 172.21.15.150:0/1961368384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed01a0190 con 0x7f1ed0100ad0 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1eb7fff700 1 -- 172.21.15.150:0/1961368384 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4003c60 con 0x7f1ed0100ad0 2024-05-10T04:03:36.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.862+0000 7f1ed5337700 1 -- 172.21.15.150:0/1961368384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed01a06a0 con 0x7f1ed0100ad0 2024-05-10T04:03:36.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.863+0000 7f1ed5337700 1 -- 172.21.15.150:0/1961368384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed0068730 con 0x7f1ed0100ad0 2024-05-10T04:03:36.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.865+0000 7f1eb7fff700 1 -- 172.21.15.150:0/1961368384 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1ec4007900 con 0x7f1ed0100ad0 2024-05-10T04:03:36.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.865+0000 7f1eb7fff700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1eb80a1260 0x7f1eb80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:36.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.865+0000 7f1ece7fc700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1eb80a1260 0x7f1eb80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:36.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.865+0000 7f1eb7fff700 1 -- 172.21.15.150:0/1961368384 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1ec40cccb0 con 0x7f1ed0100ad0 2024-05-10T04:03:36.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.866+0000 7f1ece7fc700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1eb80a1260 0x7f1eb80a3770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0199140 tx=0x7f1ec000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:36.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:36.869+0000 7f1eb7fff700 1 -- 172.21.15.150:0/1961368384 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1ec4095fd0 con 0x7f1ed0100ad0 2024-05-10T04:03:37.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.070+0000 7f1ed5337700 1 -- 172.21.15.150:0/1961368384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ed010c470 con 0x7f1ed0100ad0 2024-05-10T04:03:37.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.070+0000 7f1eb7fff700 1 -- 172.21.15.150:0/1961368384 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1ec401d0a0 con 0x7f1ed0100ad0 2024-05-10T04:03:37.073 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:03:37.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.072+0000 7f1ed5337700 1 -- 172.21.15.150:0/1961368384 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1eb80a1260 msgr2=0x7f1eb80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.072+0000 7f1ed5337700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1eb80a1260 0x7f1eb80a3770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0199140 tx=0x7f1ec000b410 comp rx=0 tx=0).stop 2024-05-10T04:03:37.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.073+0000 7f1ed5337700 1 -- 172.21.15.150:0/1961368384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed0100ad0 msgr2=0x7f1ed01980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.073+0000 7f1ed5337700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed0100ad0 0x7f1ed01980a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400c970 tx=0x7f1ec400cc80 comp rx=0 tx=0).stop 2024-05-10T04:03:37.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.073+0000 7f1ed5337700 1 -- 172.21.15.150:0/1961368384 shutdown_connections 2024-05-10T04:03:37.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.073+0000 7f1ed5337700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1eb80a1260 0x7f1eb80a3770 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.073+0000 7f1ed5337700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ed0102590 0x7f1ed019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.073+0000 7f1ed5337700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed0100ad0 0x7f1ed01980a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.073+0000 7f1ed5337700 1 --2- 172.21.15.150:0/1961368384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ed0101bf0 0x7f1ed01985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.073+0000 7f1ed5337700 1 -- 172.21.15.150:0/1961368384 >> 172.21.15.150:0/1961368384 conn(0x7f1ed00fc0e0 msgr2=0x7f1ed00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:37.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.073+0000 7f1ed5337700 1 -- 172.21.15.150:0/1961368384 shutdown_connections 2024-05-10T04:03:37.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.073+0000 7f1ed5337700 1 -- 172.21.15.150:0/1961368384 wait complete. 2024-05-10T04:03:37.088 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:03:37.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.221+0000 7f24296a2700 1 -- 172.21.15.150:0/911590982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2424101640 msgr2=0x7f2424108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.221+0000 7f24296a2700 1 --2- 172.21.15.150:0/911590982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2424101640 0x7f2424108f50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f241c00b0a0 tx=0x7f241c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.221+0000 7f24296a2700 1 -- 172.21.15.150:0/911590982 shutdown_connections 2024-05-10T04:03:37.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.221+0000 7f24296a2700 1 --2- 172.21.15.150:0/911590982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2424101640 0x7f2424108f50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.221+0000 7f24296a2700 1 --2- 172.21.15.150:0/911590982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2424101ff0 0x7f2424102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.221+0000 7f24296a2700 1 --2- 172.21.15.150:0/911590982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2424100cf0 0x7f2424101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.221+0000 7f24296a2700 1 -- 172.21.15.150:0/911590982 >> 172.21.15.150:0/911590982 conn(0x7f24240fb8b0 msgr2=0x7f24240fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:37.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.221+0000 7f24296a2700 1 -- 172.21.15.150:0/911590982 shutdown_connections 2024-05-10T04:03:37.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.221+0000 7f24296a2700 1 -- 172.21.15.150:0/911590982 wait complete. 2024-05-10T04:03:37.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.222+0000 7f24296a2700 1 Processor -- start 2024-05-10T04:03:37.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.222+0000 7f24296a2700 1 -- start start 2024-05-10T04:03:37.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.222+0000 7f24296a2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2424100cf0 0x7f2424100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:37.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.222+0000 7f24296a2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2424101640 0x7f242419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:37.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.222+0000 7f24296a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2424101ff0 0x7f24241a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:37.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.222+0000 7f24296a2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24241a1bc0 con 0x7f2424100cf0 2024-05-10T04:03:37.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.222+0000 7f24296a2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24241a1d30 con 0x7f2424101640 2024-05-10T04:03:37.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.222+0000 7f24296a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24241a1ef0 con 0x7f2424101ff0 2024-05-10T04:03:37.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.222+0000 7f2422ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2424100cf0 0x7f2424100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:37.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.222+0000 7f2422ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2424100cf0 0x7f2424100650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41982/0 (socket says 172.21.15.150:41982) 2024-05-10T04:03:37.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.222+0000 7f2422ffd700 1 -- 172.21.15.150:0/3954220802 learned_addr learned my addr 172.21.15.150:0/3954220802 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f24237fe700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2424101ff0 0x7f24241a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f24227fc700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2424101640 0x7f242419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f2422ffd700 1 -- 172.21.15.150:0/3954220802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2424101ff0 msgr2=0x7f24241a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f2422ffd700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2424101ff0 0x7f24241a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f2422ffd700 1 -- 172.21.15.150:0/3954220802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2424101640 msgr2=0x7f242419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f2422ffd700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2424101640 0x7f242419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f2422ffd700 1 -- 172.21.15.150:0/3954220802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c009d00 con 0x7f2424100cf0 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f24227fc700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2424101640 0x7f242419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f2422ffd700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2424100cf0 0x7f2424100650 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f241800c9e0 tx=0x7f241800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f240bfff700 1 -- 172.21.15.150:0/3954220802 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418007aa0 con 0x7f2424100cf0 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f240bfff700 1 -- 172.21.15.150:0/3954220802 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2418007c00 con 0x7f2424100cf0 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f24296a2700 1 -- 172.21.15.150:0/3954220802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24241a21d0 con 0x7f2424100cf0 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f240bfff700 1 -- 172.21.15.150:0/3954220802 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418018660 con 0x7f2424100cf0 2024-05-10T04:03:37.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.223+0000 7f24296a2700 1 -- 172.21.15.150:0/3954220802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24241a26b0 con 0x7f2424100cf0 2024-05-10T04:03:37.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.224+0000 7f24296a2700 1 -- 172.21.15.150:0/3954220802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2424107df0 con 0x7f2424100cf0 2024-05-10T04:03:37.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.225+0000 7f240bfff700 1 -- 172.21.15.150:0/3954220802 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f241800f450 con 0x7f2424100cf0 2024-05-10T04:03:37.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.226+0000 7f240bfff700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f240c0a1260 0x7f240c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:37.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.226+0000 7f24227fc700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f240c0a1260 0x7f240c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:37.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.226+0000 7f240bfff700 1 -- 172.21.15.150:0/3954220802 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f24180ccf20 con 0x7f2424100cf0 2024-05-10T04:03:37.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.227+0000 7f24227fc700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f240c0a1260 0x7f240c0a3770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f24240ffff0 tx=0x7f2414009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:37.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.230+0000 7f240bfff700 1 -- 172.21.15.150:0/3954220802 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f24180962d0 con 0x7f2424100cf0 2024-05-10T04:03:37.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.430+0000 7f24296a2700 1 -- 172.21.15.150:0/3954220802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f242410c5a0 con 0x7f2424100cf0 2024-05-10T04:03:37.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.430+0000 7f240bfff700 1 -- 172.21.15.150:0/3954220802 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f24180998f0 con 0x7f2424100cf0 2024-05-10T04:03:37.433 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:03:37.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.433+0000 7f24296a2700 1 -- 172.21.15.150:0/3954220802 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f240c0a1260 msgr2=0x7f240c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.433+0000 7f24296a2700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f240c0a1260 0x7f240c0a3770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f24240ffff0 tx=0x7f2414009260 comp rx=0 tx=0).stop 2024-05-10T04:03:37.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.433+0000 7f24296a2700 1 -- 172.21.15.150:0/3954220802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2424100cf0 msgr2=0x7f2424100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.433+0000 7f24296a2700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2424100cf0 0x7f2424100650 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f241800c9e0 tx=0x7f241800ccf0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.433+0000 7f24296a2700 1 -- 172.21.15.150:0/3954220802 shutdown_connections 2024-05-10T04:03:37.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.433+0000 7f24296a2700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f240c0a1260 0x7f240c0a3770 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.433+0000 7f24296a2700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2424101ff0 0x7f24241a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.433+0000 7f24296a2700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2424100cf0 0x7f2424100650 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.433+0000 7f24296a2700 1 --2- 172.21.15.150:0/3954220802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2424101640 0x7f242419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.433+0000 7f24296a2700 1 -- 172.21.15.150:0/3954220802 >> 172.21.15.150:0/3954220802 conn(0x7f24240fb8b0 msgr2=0x7f2424104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:37.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.433+0000 7f24296a2700 1 -- 172.21.15.150:0/3954220802 shutdown_connections 2024-05-10T04:03:37.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.433+0000 7f24296a2700 1 -- 172.21.15.150:0/3954220802 wait complete. 2024-05-10T04:03:37.449 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:03:37.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.579+0000 7fa1390db700 1 -- 172.21.15.150:0/3259021812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134101090 msgr2=0x7fa1341014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.579+0000 7fa1390db700 1 --2- 172.21.15.150:0/3259021812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134101090 0x7fa1341014a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa124009990 tx=0x7fa124009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.579+0000 7fa1390db700 1 -- 172.21.15.150:0/3259021812 shutdown_connections 2024-05-10T04:03:37.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.579+0000 7fa1390db700 1 --2- 172.21.15.150:0/3259021812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1341025d0 0x7fa134109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.579+0000 7fa1390db700 1 --2- 172.21.15.150:0/3259021812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134101090 0x7fa1341014a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.579+0000 7fa1390db700 1 --2- 172.21.15.150:0/3259021812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa134101a70 0x7fa134101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.579+0000 7fa1390db700 1 -- 172.21.15.150:0/3259021812 >> 172.21.15.150:0/3259021812 conn(0x7fa1340fb8f0 msgr2=0x7fa1340fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:37.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.579+0000 7fa1390db700 1 -- 172.21.15.150:0/3259021812 shutdown_connections 2024-05-10T04:03:37.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.579+0000 7fa1390db700 1 -- 172.21.15.150:0/3259021812 wait complete. 2024-05-10T04:03:37.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.580+0000 7fa1390db700 1 Processor -- start 2024-05-10T04:03:37.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.580+0000 7fa1390db700 1 -- start start 2024-05-10T04:03:37.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.580+0000 7fa1390db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa134101090 0x7fa134100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:37.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.580+0000 7fa1390db700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa134101a70 0x7fa13419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:37.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.580+0000 7fa1390db700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1341025d0 0x7fa13419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:37.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.580+0000 7fa1390db700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13419faa0 con 0x7fa1341025d0 2024-05-10T04:03:37.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.580+0000 7fa1390db700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13419fc10 con 0x7fa134101a70 2024-05-10T04:03:37.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.580+0000 7fa1390db700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13419fdd0 con 0x7fa134101090 2024-05-10T04:03:37.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.580+0000 7fa13359e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1341025d0 0x7fa13419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:37.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.580+0000 7fa13359e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1341025d0 0x7fa13419f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42012/0 (socket says 172.21.15.150:42012) 2024-05-10T04:03:37.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.580+0000 7fa13359e700 1 -- 172.21.15.150:0/3690292807 learned_addr learned my addr 172.21.15.150:0/3690292807 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:37.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.581+0000 7fa13259c700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa134101a70 0x7fa13419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:37.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.581+0000 7fa132d9d700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa134101090 0x7fa134100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:37.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.581+0000 7fa13359e700 1 -- 172.21.15.150:0/3690292807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa134101090 msgr2=0x7fa134100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.581+0000 7fa13359e700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa134101090 0x7fa134100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.581+0000 7fa13359e700 1 -- 172.21.15.150:0/3690292807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa134101a70 msgr2=0x7fa13419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.581+0000 7fa13359e700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa134101a70 0x7fa13419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.581+0000 7fa13359e700 1 -- 172.21.15.150:0/3690292807 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1240095f0 con 0x7fa1341025d0 2024-05-10T04:03:37.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.581+0000 7fa13359e700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1341025d0 0x7fa13419f350 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa12c007ee0 tx=0x7fa12c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:37.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.581+0000 7fa12bfff700 1 -- 172.21.15.150:0/3690292807 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c017070 con 0x7fa1341025d0 2024-05-10T04:03:37.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.581+0000 7fa12bfff700 1 -- 172.21.15.150:0/3690292807 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa12c00f040 con 0x7fa1341025d0 2024-05-10T04:03:37.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.581+0000 7fa1390db700 1 -- 172.21.15.150:0/3690292807 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13419fff0 con 0x7fa1341025d0 2024-05-10T04:03:37.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.581+0000 7fa12bfff700 1 -- 172.21.15.150:0/3690292807 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c005580 con 0x7fa1341025d0 2024-05-10T04:03:37.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.581+0000 7fa1390db700 1 -- 172.21.15.150:0/3690292807 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1341a0450 con 0x7fa1341025d0 2024-05-10T04:03:37.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.582+0000 7fa1390db700 1 -- 172.21.15.150:0/3690292807 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa134107ef0 con 0x7fa1341025d0 2024-05-10T04:03:37.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.583+0000 7fa12bfff700 1 -- 172.21.15.150:0/3690292807 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa12c00db20 con 0x7fa1341025d0 2024-05-10T04:03:37.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.584+0000 7fa12bfff700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa1200a11b0 0x7fa1200a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:37.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.584+0000 7fa132d9d700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa1200a11b0 0x7fa1200a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:37.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.584+0000 7fa12bfff700 1 -- 172.21.15.150:0/3690292807 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa12c0ccaa0 con 0x7fa1341025d0 2024-05-10T04:03:37.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.585+0000 7fa132d9d700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa1200a11b0 0x7fa1200a36c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa12400b5c0 tx=0x7fa124005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:37.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.588+0000 7fa12bfff700 1 -- 172.21.15.150:0/3690292807 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa12c095d60 con 0x7fa1341025d0 2024-05-10T04:03:37.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:37 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/913649106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:03:37.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:37 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1961368384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:03:37.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:37 smithi150 ceph-mon[42477]: pgmap v261: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 28 B/s, 1 keys/s, 1 objects/s recovering 2024-05-10T04:03:37.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:37 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/913649106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:03:37.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:37 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1961368384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:03:37.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:37 smithi181 ceph-mon[46602]: pgmap v261: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 28 B/s, 1 keys/s, 1 objects/s recovering 2024-05-10T04:03:37.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:37 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/913649106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:03:37.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:37 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1961368384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:03:37.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:37 smithi161 ceph-mon[46918]: pgmap v261: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 28 B/s, 1 keys/s, 1 objects/s recovering 2024-05-10T04:03:37.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.829+0000 7fa1390db700 1 -- 172.21.15.150:0/3690292807 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa13410c2b0 con 0x7fa1341025d0 2024-05-10T04:03:37.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.829+0000 7fa12bfff700 1 -- 172.21.15.150:0/3690292807 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10886 (secure 0 0 0) 0x7fa12c099380 con 0x7fa1341025d0 2024-05-10T04:03:37.832 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:37.832 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:02:57.183845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:03:37.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.832+0000 7fa1390db700 1 -- 172.21.15.150:0/3690292807 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa1200a11b0 msgr2=0x7fa1200a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.832+0000 7fa1390db700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa1200a11b0 0x7fa1200a36c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa12400b5c0 tx=0x7fa124005fb0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.832+0000 7fa1390db700 1 -- 172.21.15.150:0/3690292807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1341025d0 msgr2=0x7fa13419f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.832+0000 7fa1390db700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1341025d0 0x7fa13419f350 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa12c007ee0 tx=0x7fa12c00d3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.832+0000 7fa1390db700 1 -- 172.21.15.150:0/3690292807 shutdown_connections 2024-05-10T04:03:37.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.832+0000 7fa1390db700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa1200a11b0 0x7fa1200a36c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.832+0000 7fa1390db700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1341025d0 0x7fa13419f350 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.832+0000 7fa1390db700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa134101090 0x7fa134100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.832+0000 7fa1390db700 1 --2- 172.21.15.150:0/3690292807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa134101a70 0x7fa13419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.832+0000 7fa1390db700 1 -- 172.21.15.150:0/3690292807 >> 172.21.15.150:0/3690292807 conn(0x7fa1340fb8f0 msgr2=0x7fa134105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:37.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.832+0000 7fa1390db700 1 -- 172.21.15.150:0/3690292807 shutdown_connections 2024-05-10T04:03:37.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.832+0000 7fa1390db700 1 -- 172.21.15.150:0/3690292807 wait complete. 2024-05-10T04:03:37.836 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 24 2024-05-10T04:03:37.848 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:03:37.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.978+0000 7f6b466ec700 1 -- 172.21.15.150:0/2198763697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b40102060 msgr2=0x7f6b40102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.978+0000 7f6b466ec700 1 --2- 172.21.15.150:0/2198763697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b40102060 0x7f6b40102470 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009990 tx=0x7f6b30009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.978+0000 7f6b466ec700 1 -- 172.21.15.150:0/2198763697 shutdown_connections 2024-05-10T04:03:37.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.978+0000 7f6b466ec700 1 --2- 172.21.15.150:0/2198763697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b401016c0 0x7f6b40108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.978+0000 7f6b466ec700 1 --2- 172.21.15.150:0/2198763697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b40102060 0x7f6b40102470 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.978+0000 7f6b466ec700 1 --2- 172.21.15.150:0/2198763697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b40100d70 0x7f6b40101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.978+0000 7f6b466ec700 1 -- 172.21.15.150:0/2198763697 >> 172.21.15.150:0/2198763697 conn(0x7f6b400fb8b0 msgr2=0x7f6b400fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:37.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.978+0000 7f6b466ec700 1 -- 172.21.15.150:0/2198763697 shutdown_connections 2024-05-10T04:03:37.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.978+0000 7f6b466ec700 1 -- 172.21.15.150:0/2198763697 wait complete. 2024-05-10T04:03:37.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.979+0000 7f6b466ec700 1 Processor -- start 2024-05-10T04:03:37.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.979+0000 7f6b466ec700 1 -- start start 2024-05-10T04:03:37.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.979+0000 7f6b466ec700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b40100d70 0x7f6b401006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:37.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.979+0000 7f6b466ec700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b401016c0 0x7f6b4019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:37.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.979+0000 7f6b466ec700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b40102060 0x7f6b4019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:37.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.979+0000 7f6b466ec700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4019fa80 con 0x7f6b40100d70 2024-05-10T04:03:37.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.979+0000 7f6b466ec700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4019fbf0 con 0x7f6b401016c0 2024-05-10T04:03:37.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.979+0000 7f6b466ec700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4019fdb0 con 0x7f6b40102060 2024-05-10T04:03:37.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.979+0000 7f6b3ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b40100d70 0x7f6b401006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:37.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.979+0000 7f6b3ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b40100d70 0x7f6b401006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42026/0 (socket says 172.21.15.150:42026) 2024-05-10T04:03:37.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.979+0000 7f6b3ffff700 1 -- 172.21.15.150:0/2938504059 learned_addr learned my addr 172.21.15.150:0/2938504059 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:37.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.979+0000 7f6b3f7fe700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b401016c0 0x7f6b4019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:37.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.979+0000 7f6b44c89700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b40102060 0x7f6b4019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:37.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.980+0000 7f6b3ffff700 1 -- 172.21.15.150:0/2938504059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b40102060 msgr2=0x7f6b4019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.980+0000 7f6b3ffff700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b40102060 0x7f6b4019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.980+0000 7f6b3ffff700 1 -- 172.21.15.150:0/2938504059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b401016c0 msgr2=0x7f6b4019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:37.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.980+0000 7f6b3ffff700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b401016c0 0x7f6b4019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:37.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.980+0000 7f6b3ffff700 1 -- 172.21.15.150:0/2938504059 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b300095f0 con 0x7f6b40100d70 2024-05-10T04:03:37.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.980+0000 7f6b44c89700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b40102060 0x7f6b4019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:37.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.980+0000 7f6b3ffff700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b40100d70 0x7f6b401006f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000b6f0 tx=0x7f6b30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:37.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.980+0000 7f6b3d7fa700 1 -- 172.21.15.150:0/2938504059 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3001d070 con 0x7f6b40100d70 2024-05-10T04:03:37.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.980+0000 7f6b3d7fa700 1 -- 172.21.15.150:0/2938504059 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b3000fb90 con 0x7f6b40100d70 2024-05-10T04:03:37.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.980+0000 7f6b466ec700 1 -- 172.21.15.150:0/2938504059 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4019ffd0 con 0x7f6b40100d70 2024-05-10T04:03:37.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.980+0000 7f6b3d7fa700 1 -- 172.21.15.150:0/2938504059 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30017650 con 0x7f6b40100d70 2024-05-10T04:03:37.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.980+0000 7f6b466ec700 1 -- 172.21.15.150:0/2938504059 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b401a0340 con 0x7f6b40100d70 2024-05-10T04:03:37.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.981+0000 7f6b466ec700 1 -- 172.21.15.150:0/2938504059 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b40107e70 con 0x7f6b40100d70 2024-05-10T04:03:37.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.982+0000 7f6b3d7fa700 1 -- 172.21.15.150:0/2938504059 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6b30022470 con 0x7f6b40100d70 2024-05-10T04:03:37.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.983+0000 7f6b3d7fa700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6b280a11b0 0x7f6b280a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:37.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.983+0000 7f6b3d7fa700 1 -- 172.21.15.150:0/2938504059 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6b300ce570 con 0x7f6b40100d70 2024-05-10T04:03:37.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.983+0000 7f6b3f7fe700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6b280a11b0 0x7f6b280a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:37.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.983+0000 7f6b3f7fe700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6b280a11b0 0x7f6b280a36c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6b40100090 tx=0x7f6b34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:37.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:37.987+0000 7f6b3d7fa700 1 -- 172.21.15.150:0/2938504059 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6b30097920 con 0x7f6b40100d70 2024-05-10T04:03:38.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.185+0000 7f6b466ec700 1 -- 172.21.15.150:0/2938504059 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b4010c2e0 con 0x7f6b40100d70 2024-05-10T04:03:38.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.186+0000 7f6b3d7fa700 1 -- 172.21.15.150:0/2938504059 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24554 (secure 0 0 0) 0x7f6b3009af40 con 0x7f6b40100d70 2024-05-10T04:03:38.189 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":87,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:03:30.185138+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:03:38.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.189+0000 7f6b466ec700 1 -- 172.21.15.150:0/2938504059 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6b280a11b0 msgr2=0x7f6b280a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:38.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.189+0000 7f6b466ec700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6b280a11b0 0x7f6b280a36c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6b40100090 tx=0x7f6b34009260 comp rx=0 tx=0).stop 2024-05-10T04:03:38.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.189+0000 7f6b466ec700 1 -- 172.21.15.150:0/2938504059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b40100d70 msgr2=0x7f6b401006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:38.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.189+0000 7f6b466ec700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b40100d70 0x7f6b401006f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000b6f0 tx=0x7f6b30004730 comp rx=0 tx=0).stop 2024-05-10T04:03:38.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.189+0000 7f6b466ec700 1 -- 172.21.15.150:0/2938504059 shutdown_connections 2024-05-10T04:03:38.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.189+0000 7f6b466ec700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6b280a11b0 0x7f6b280a36c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.189+0000 7f6b466ec700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b40102060 0x7f6b4019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.189+0000 7f6b466ec700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b40100d70 0x7f6b401006f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.189+0000 7f6b466ec700 1 --2- 172.21.15.150:0/2938504059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b401016c0 0x7f6b4019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.189+0000 7f6b466ec700 1 -- 172.21.15.150:0/2938504059 >> 172.21.15.150:0/2938504059 conn(0x7f6b400fb8b0 msgr2=0x7f6b40104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:38.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.189+0000 7f6b466ec700 1 -- 172.21.15.150:0/2938504059 shutdown_connections 2024-05-10T04:03:38.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.189+0000 7f6b466ec700 1 -- 172.21.15.150:0/2938504059 wait complete. 2024-05-10T04:03:38.206 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:02:30.076694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:02:30.760710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:02:33.850714+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:03:38.206 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:03:38.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.339+0000 7efdad13d700 1 -- 172.21.15.150:0/2163501936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efda80fff80 msgr2=0x7efda8100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.339+0000 7efdad13d700 1 --2- 172.21.15.150:0/2163501936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efda80fff80 0x7efda8100390 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efd98009990 tx=0x7efd98009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.339+0000 7efdad13d700 1 -- 172.21.15.150:0/2163501936 shutdown_connections 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.339+0000 7efdad13d700 1 --2- 172.21.15.150:0/2163501936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efda8107a50 0x7efda8109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.339+0000 7efdad13d700 1 --2- 172.21.15.150:0/2163501936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efda80fff80 0x7efda8100390 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.339+0000 7efdad13d700 1 --2- 172.21.15.150:0/2163501936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda81050f0 0x7efda8107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.339+0000 7efdad13d700 1 -- 172.21.15.150:0/2163501936 >> 172.21.15.150:0/2163501936 conn(0x7efda80fb8f0 msgr2=0x7efda80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.340+0000 7efdad13d700 1 -- 172.21.15.150:0/2163501936 shutdown_connections 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.340+0000 7efdad13d700 1 -- 172.21.15.150:0/2163501936 wait complete. 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.340+0000 7efdad13d700 1 Processor -- start 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.340+0000 7efdad13d700 1 -- start start 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efdad13d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efda80fff80 0x7efda8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efdad13d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda81050f0 0x7efda81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efdad13d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efda8107a50 0x7efda819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:38.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efdad13d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda819fa90 con 0x7efda8107a50 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efdad13d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda819fc00 con 0x7efda81050f0 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efdad13d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda819fdc0 con 0x7efda80fff80 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efda759e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efda8107a50 0x7efda819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efda759e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efda8107a50 0x7efda819f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42044/0 (socket says 172.21.15.150:42044) 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efda759e700 1 -- 172.21.15.150:0/1905303821 learned_addr learned my addr 172.21.15.150:0/1905303821 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efda659c700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda81050f0 0x7efda81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efda6d9d700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efda80fff80 0x7efda8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efda759e700 1 -- 172.21.15.150:0/1905303821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efda80fff80 msgr2=0x7efda8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efda759e700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efda80fff80 0x7efda8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efda759e700 1 -- 172.21.15.150:0/1905303821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda81050f0 msgr2=0x7efda81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efda759e700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda81050f0 0x7efda81984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efda759e700 1 -- 172.21.15.150:0/1905303821 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd980095f0 con 0x7efda8107a50 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.341+0000 7efda759e700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efda8107a50 0x7efda819f480 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efda000b970 tx=0x7efda000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.342+0000 7efd8ffff700 1 -- 172.21.15.150:0/1905303821 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda000d860 con 0x7efda8107a50 2024-05-10T04:03:38.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.342+0000 7efd8ffff700 1 -- 172.21.15.150:0/1905303821 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efda000de80 con 0x7efda8107a50 2024-05-10T04:03:38.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.342+0000 7efdad13d700 1 -- 172.21.15.150:0/1905303821 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda81a00a0 con 0x7efda8107a50 2024-05-10T04:03:38.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.342+0000 7efd8ffff700 1 -- 172.21.15.150:0/1905303821 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda0017400 con 0x7efda8107a50 2024-05-10T04:03:38.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.342+0000 7efdad13d700 1 -- 172.21.15.150:0/1905303821 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda81a0560 con 0x7efda8107a50 2024-05-10T04:03:38.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.343+0000 7efdad13d700 1 -- 172.21.15.150:0/1905303821 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda80fd4a0 con 0x7efda8107a50 2024-05-10T04:03:38.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.344+0000 7efd8ffff700 1 -- 172.21.15.150:0/1905303821 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7efda000e630 con 0x7efda8107a50 2024-05-10T04:03:38.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.345+0000 7efd8ffff700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7efd900a1260 0x7efd900a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:38.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.345+0000 7efda6d9d700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7efd900a1260 0x7efd900a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:38.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.345+0000 7efd8ffff700 1 -- 172.21.15.150:0/1905303821 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efda0006e40 con 0x7efda8107a50 2024-05-10T04:03:38.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.345+0000 7efda6d9d700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7efd900a1260 0x7efd900a3770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efd98003730 tx=0x7efd98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:38.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.349+0000 7efd8ffff700 1 -- 172.21.15.150:0/1905303821 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efda0095e30 con 0x7efda8107a50 2024-05-10T04:03:38.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:38 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3954220802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:03:38.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:38 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3690292807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:38.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:38 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2938504059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:38.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.549+0000 7efdad13d700 1 -- 172.21.15.150:0/1905303821 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efda810d410 con 0x7efda8107a50 2024-05-10T04:03:38.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.549+0000 7efd8ffff700 1 -- 172.21.15.150:0/1905303821 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7efda0099450 con 0x7efda8107a50 2024-05-10T04:03:38.552 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:03:38.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.552+0000 7efdad13d700 1 -- 172.21.15.150:0/1905303821 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7efd900a1260 msgr2=0x7efd900a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:38.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.552+0000 7efdad13d700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7efd900a1260 0x7efd900a3770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efd98003730 tx=0x7efd98005fb0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.552+0000 7efdad13d700 1 -- 172.21.15.150:0/1905303821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efda8107a50 msgr2=0x7efda819f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:38.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.552+0000 7efdad13d700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efda8107a50 0x7efda819f480 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efda000b970 tx=0x7efda000bc80 comp rx=0 tx=0).stop 2024-05-10T04:03:38.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.552+0000 7efdad13d700 1 -- 172.21.15.150:0/1905303821 shutdown_connections 2024-05-10T04:03:38.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.552+0000 7efdad13d700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7efd900a1260 0x7efd900a3770 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.552+0000 7efdad13d700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efda8107a50 0x7efda819f480 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.552+0000 7efdad13d700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efda80fff80 0x7efda8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.552+0000 7efdad13d700 1 --2- 172.21.15.150:0/1905303821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda81050f0 0x7efda81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.552+0000 7efdad13d700 1 -- 172.21.15.150:0/1905303821 >> 172.21.15.150:0/1905303821 conn(0x7efda80fb8f0 msgr2=0x7efda8106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:38.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.552+0000 7efdad13d700 1 -- 172.21.15.150:0/1905303821 shutdown_connections 2024-05-10T04:03:38.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.552+0000 7efdad13d700 1 -- 172.21.15.150:0/1905303821 wait complete. 2024-05-10T04:03:38.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:03:38.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.701+0000 7fc4c37eb700 1 -- 172.21.15.150:0/1466717065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4bc101110 msgr2=0x7fc4bc101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:38.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.701+0000 7fc4c37eb700 1 --2- 172.21.15.150:0/1466717065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4bc101110 0x7fc4bc101520 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009990 tx=0x7fc4ac009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.701+0000 7fc4c37eb700 1 -- 172.21.15.150:0/1466717065 shutdown_connections 2024-05-10T04:03:38.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.701+0000 7fc4c37eb700 1 --2- 172.21.15.150:0/1466717065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4bc102950 0x7fc4bc109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.701+0000 7fc4c37eb700 1 --2- 172.21.15.150:0/1466717065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4bc101110 0x7fc4bc101520 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.701+0000 7fc4c37eb700 1 --2- 172.21.15.150:0/1466717065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc102000 0x7fc4bc102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.701+0000 7fc4c37eb700 1 -- 172.21.15.150:0/1466717065 >> 172.21.15.150:0/1466717065 conn(0x7fc4bc0fb8f0 msgr2=0x7fc4bc0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:38.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.701+0000 7fc4c37eb700 1 -- 172.21.15.150:0/1466717065 shutdown_connections 2024-05-10T04:03:38.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.701+0000 7fc4c37eb700 1 -- 172.21.15.150:0/1466717065 wait complete. 2024-05-10T04:03:38.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.702+0000 7fc4c37eb700 1 Processor -- start 2024-05-10T04:03:38.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.702+0000 7fc4c37eb700 1 -- start start 2024-05-10T04:03:38.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.702+0000 7fc4c37eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4bc101110 0x7fc4bc100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:38.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.702+0000 7fc4c37eb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4bc102000 0x7fc4bc19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:38.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.702+0000 7fc4c37eb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc102950 0x7fc4bc19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:38.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.702+0000 7fc4c37eb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc19fbe0 con 0x7fc4bc102000 2024-05-10T04:03:38.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.702+0000 7fc4c37eb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc19fd50 con 0x7fc4bc102950 2024-05-10T04:03:38.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.702+0000 7fc4c37eb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc19ff10 con 0x7fc4bc101110 2024-05-10T04:03:38.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.702+0000 7fc4c0d86700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4bc102000 0x7fc4bc19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:38.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.702+0000 7fc4c0d86700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4bc102000 0x7fc4bc19ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42068/0 (socket says 172.21.15.150:42068) 2024-05-10T04:03:38.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.702+0000 7fc4c0d86700 1 -- 172.21.15.150:0/2007138213 learned_addr learned my addr 172.21.15.150:0/2007138213 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:38.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.702+0000 7fc4c1587700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4bc101110 0x7fc4bc100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:38.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.702+0000 7fc4c1d88700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc102950 0x7fc4bc19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:38.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.703+0000 7fc4c0d86700 1 -- 172.21.15.150:0/2007138213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4bc101110 msgr2=0x7fc4bc100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:38.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.703+0000 7fc4c0d86700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4bc101110 0x7fc4bc100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.703+0000 7fc4c0d86700 1 -- 172.21.15.150:0/2007138213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc102950 msgr2=0x7fc4bc19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:38.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.703+0000 7fc4c0d86700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc102950 0x7fc4bc19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.703+0000 7fc4c0d86700 1 -- 172.21.15.150:0/2007138213 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac0095f0 con 0x7fc4bc102000 2024-05-10T04:03:38.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.703+0000 7fc4c0d86700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4bc102000 0x7fc4bc19ce70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800e9a0 tx=0x7fc4b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:38.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.703+0000 7fc4b27fc700 1 -- 172.21.15.150:0/2007138213 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8009790 con 0x7fc4bc102000 2024-05-10T04:03:38.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.703+0000 7fc4b27fc700 1 -- 172.21.15.150:0/2007138213 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4b8010430 con 0x7fc4bc102000 2024-05-10T04:03:38.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.703+0000 7fc4b27fc700 1 -- 172.21.15.150:0/2007138213 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8018680 con 0x7fc4bc102000 2024-05-10T04:03:38.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.703+0000 7fc4c37eb700 1 -- 172.21.15.150:0/2007138213 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc1a01f0 con 0x7fc4bc102000 2024-05-10T04:03:38.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.703+0000 7fc4c37eb700 1 -- 172.21.15.150:0/2007138213 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4bc1a0700 con 0x7fc4bc102000 2024-05-10T04:03:38.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.704+0000 7fc4c37eb700 1 -- 172.21.15.150:0/2007138213 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4bc1080f0 con 0x7fc4bc102000 2024-05-10T04:03:38.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.706+0000 7fc4b27fc700 1 -- 172.21.15.150:0/2007138213 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc4b80105a0 con 0x7fc4bc102000 2024-05-10T04:03:38.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.707+0000 7fc4b27fc700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc4a80a12e0 0x7fc4a80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:38.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.707+0000 7fc4c1587700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc4a80a12e0 0x7fc4a80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:38.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.707+0000 7fc4b27fc700 1 -- 172.21.15.150:0/2007138213 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc4b8014070 con 0x7fc4bc102000 2024-05-10T04:03:38.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.707+0000 7fc4c1587700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc4a80a12e0 0x7fc4a80a37f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00b2e0 tx=0x7fc4ac01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:38.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.711+0000 7fc4b27fc700 1 -- 172.21.15.150:0/2007138213 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc4b8096460 con 0x7fc4bc102000 2024-05-10T04:03:38.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:38 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3954220802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:03:38.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:38 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3690292807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:38.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:38 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2938504059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:38.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:38 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3954220802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:03:38.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:38 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3690292807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:38.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:38 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2938504059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:38.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.910+0000 7fc4c37eb700 1 -- 172.21.15.150:0/2007138213 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc4bc10c720 con 0x7fc4bc102000 2024-05-10T04:03:38.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.911+0000 7fc4b27fc700 1 -- 172.21.15.150:0/2007138213 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc4b80260a0 con 0x7fc4bc102000 2024-05-10T04:03:38.913 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:03:38.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.913+0000 7fc4c37eb700 1 -- 172.21.15.150:0/2007138213 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc4a80a12e0 msgr2=0x7fc4a80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:38.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.913+0000 7fc4c37eb700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc4a80a12e0 0x7fc4a80a37f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00b2e0 tx=0x7fc4ac01a040 comp rx=0 tx=0).stop 2024-05-10T04:03:38.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.913+0000 7fc4c37eb700 1 -- 172.21.15.150:0/2007138213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4bc102000 msgr2=0x7fc4bc19ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:38.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.913+0000 7fc4c37eb700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4bc102000 0x7fc4bc19ce70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800e9a0 tx=0x7fc4b800ecb0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.913+0000 7fc4c37eb700 1 -- 172.21.15.150:0/2007138213 shutdown_connections 2024-05-10T04:03:38.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.913+0000 7fc4c37eb700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc4a80a12e0 0x7fc4a80a37f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.913+0000 7fc4c37eb700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc102950 0x7fc4bc19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.913+0000 7fc4c37eb700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4bc101110 0x7fc4bc100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.913+0000 7fc4c37eb700 1 --2- 172.21.15.150:0/2007138213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4bc102000 0x7fc4bc19ce70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:38.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.913+0000 7fc4c37eb700 1 -- 172.21.15.150:0/2007138213 >> 172.21.15.150:0/2007138213 conn(0x7fc4bc0fb8f0 msgr2=0x7fc4bc105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:38.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.913+0000 7fc4c37eb700 1 -- 172.21.15.150:0/2007138213 shutdown_connections 2024-05-10T04:03:38.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:38.913+0000 7fc4c37eb700 1 -- 172.21.15.150:0/2007138213 wait complete. 2024-05-10T04:03:38.928 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:03:39.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.061+0000 7f867ffa5700 1 -- 172.21.15.150:0/4273328014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8678100280 msgr2=0x7f8678100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:39.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.061+0000 7f867ffa5700 1 --2- 172.21.15.150:0/4273328014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8678100280 0x7f8678100690 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8674009990 tx=0x7f8674009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.061+0000 7f867ffa5700 1 -- 172.21.15.150:0/4273328014 shutdown_connections 2024-05-10T04:03:39.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.061+0000 7f867ffa5700 1 --2- 172.21.15.150:0/4273328014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8678101d40 0x7f86781085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.061+0000 7f867ffa5700 1 --2- 172.21.15.150:0/4273328014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8678100280 0x7f8678100690 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.061+0000 7f867ffa5700 1 --2- 172.21.15.150:0/4273328014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86781013a0 0x7f86781017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.061+0000 7f867ffa5700 1 -- 172.21.15.150:0/4273328014 >> 172.21.15.150:0/4273328014 conn(0x7f86780fb8b0 msgr2=0x7f86780fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:39.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.061+0000 7f867ffa5700 1 -- 172.21.15.150:0/4273328014 shutdown_connections 2024-05-10T04:03:39.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.061+0000 7f867ffa5700 1 -- 172.21.15.150:0/4273328014 wait complete. 2024-05-10T04:03:39.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867ffa5700 1 Processor -- start 2024-05-10T04:03:39.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867ffa5700 1 -- start start 2024-05-10T04:03:39.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867ffa5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8678100280 0x7f86780fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:39.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867ffa5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86781013a0 0x7f867819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:39.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867ffa5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8678101d40 0x7f867819f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:39.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867ffa5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867819f9e0 con 0x7f8678100280 2024-05-10T04:03:39.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867ffa5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867819fb50 con 0x7f86781013a0 2024-05-10T04:03:39.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867ffa5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867819fd10 con 0x7f8678101d40 2024-05-10T04:03:39.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867dd41700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8678100280 0x7f86780fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:39.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867dd41700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8678100280 0x7f86780fd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42092/0 (socket says 172.21.15.150:42092) 2024-05-10T04:03:39.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867dd41700 1 -- 172.21.15.150:0/1281436780 learned_addr learned my addr 172.21.15.150:0/1281436780 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:39.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867e542700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8678101d40 0x7f867819f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:39.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867d540700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86781013a0 0x7f867819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:39.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867dd41700 1 -- 172.21.15.150:0/1281436780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8678101d40 msgr2=0x7f867819f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:39.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867dd41700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8678101d40 0x7f867819f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867dd41700 1 -- 172.21.15.150:0/1281436780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86781013a0 msgr2=0x7f867819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:39.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867dd41700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86781013a0 0x7f867819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867dd41700 1 -- 172.21.15.150:0/1281436780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86740095f0 con 0x7f8678100280 2024-05-10T04:03:39.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.062+0000 7f867dd41700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8678100280 0x7f86780fd6d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8674006010 tx=0x7f867400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:39.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.063+0000 7f866effd700 1 -- 172.21.15.150:0/1281436780 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867401d070 con 0x7f8678100280 2024-05-10T04:03:39.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.063+0000 7f867ffa5700 1 -- 172.21.15.150:0/1281436780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867819ff90 con 0x7f8678100280 2024-05-10T04:03:39.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.063+0000 7f866effd700 1 -- 172.21.15.150:0/1281436780 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8674003960 con 0x7f8678100280 2024-05-10T04:03:39.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.063+0000 7f866effd700 1 -- 172.21.15.150:0/1281436780 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86740054f0 con 0x7f8678100280 2024-05-10T04:03:39.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.063+0000 7f867ffa5700 1 -- 172.21.15.150:0/1281436780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86781a0350 con 0x7f8678100280 2024-05-10T04:03:39.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.064+0000 7f867ffa5700 1 -- 172.21.15.150:0/1281436780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8678107ec0 con 0x7f8678100280 2024-05-10T04:03:39.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.065+0000 7f866effd700 1 -- 172.21.15.150:0/1281436780 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f867400fb40 con 0x7f8678100280 2024-05-10T04:03:39.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.066+0000 7f866effd700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f86640a1260 0x7f86640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:39.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.066+0000 7f867d540700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f86640a1260 0x7f86640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:39.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.066+0000 7f866effd700 1 -- 172.21.15.150:0/1281436780 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f86740ce2c0 con 0x7f8678100280 2024-05-10T04:03:39.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.066+0000 7f867d540700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f86640a1260 0x7f86640a3770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8668005950 tx=0x7f86680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:39.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.069+0000 7f866effd700 1 -- 172.21.15.150:0/1281436780 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8674097710 con 0x7f8678100280 2024-05-10T04:03:39.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.269+0000 7f867ffa5700 1 -- 172.21.15.150:0/1281436780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f867810bb90 con 0x7f8678100280 2024-05-10T04:03:39.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.270+0000 7f866effd700 1 -- 172.21.15.150:0/1281436780 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f867409ad30 con 0x7f8678100280 2024-05-10T04:03:39.272 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:03:39.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.272+0000 7f867ffa5700 1 -- 172.21.15.150:0/1281436780 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f86640a1260 msgr2=0x7f86640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:39.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.272+0000 7f867ffa5700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f86640a1260 0x7f86640a3770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8668005950 tx=0x7f86680058e0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.272+0000 7f867ffa5700 1 -- 172.21.15.150:0/1281436780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8678100280 msgr2=0x7f86780fd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:39.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.272+0000 7f867ffa5700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8678100280 0x7f86780fd6d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8674006010 tx=0x7f867400f690 comp rx=0 tx=0).stop 2024-05-10T04:03:39.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.272+0000 7f867ffa5700 1 -- 172.21.15.150:0/1281436780 shutdown_connections 2024-05-10T04:03:39.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.272+0000 7f867ffa5700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f86640a1260 0x7f86640a3770 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.272+0000 7f867ffa5700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8678101d40 0x7f867819f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.272+0000 7f867ffa5700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8678100280 0x7f86780fd6d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.272+0000 7f867ffa5700 1 --2- 172.21.15.150:0/1281436780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86781013a0 0x7f867819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.272+0000 7f867ffa5700 1 -- 172.21.15.150:0/1281436780 >> 172.21.15.150:0/1281436780 conn(0x7f86780fb8b0 msgr2=0x7f8678105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:39.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.272+0000 7f867ffa5700 1 -- 172.21.15.150:0/1281436780 shutdown_connections 2024-05-10T04:03:39.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.272+0000 7f867ffa5700 1 -- 172.21.15.150:0/1281436780 wait complete. 2024-05-10T04:03:39.287 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:03:39.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.423+0000 7f38e56d4700 1 -- 172.21.15.150:0/767386030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e0101a30 msgr2=0x7f38e0101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:39.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.423+0000 7f38e56d4700 1 --2- 172.21.15.150:0/767386030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e0101a30 0x7f38e0101f00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009990 tx=0x7f38d0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.423+0000 7f38e56d4700 1 -- 172.21.15.150:0/767386030 shutdown_connections 2024-05-10T04:03:39.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.423+0000 7f38e56d4700 1 --2- 172.21.15.150:0/767386030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0102590 0x7f38e0108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.423+0000 7f38e56d4700 1 --2- 172.21.15.150:0/767386030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38e0101030 0x7f38e0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.423+0000 7f38e56d4700 1 --2- 172.21.15.150:0/767386030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e0101a30 0x7f38e0101f00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.423+0000 7f38e56d4700 1 -- 172.21.15.150:0/767386030 >> 172.21.15.150:0/767386030 conn(0x7f38e00fb8b0 msgr2=0x7f38e00fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:39.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.423+0000 7f38e56d4700 1 -- 172.21.15.150:0/767386030 shutdown_connections 2024-05-10T04:03:39.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.423+0000 7f38e56d4700 1 -- 172.21.15.150:0/767386030 wait complete. 2024-05-10T04:03:39.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.424+0000 7f38e56d4700 1 Processor -- start 2024-05-10T04:03:39.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.424+0000 7f38e56d4700 1 -- start start 2024-05-10T04:03:39.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.424+0000 7f38e56d4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e0101030 0x7f38e0100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:39.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.424+0000 7f38e56d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0101a30 0x7f38e019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:39.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.424+0000 7f38e56d4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38e0102590 0x7f38e019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:39.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.424+0000 7f38e56d4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e019fa10 con 0x7f38e0101030 2024-05-10T04:03:39.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.424+0000 7f38e56d4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e019fb80 con 0x7f38e0102590 2024-05-10T04:03:39.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.424+0000 7f38e56d4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e019fd40 con 0x7f38e0101a30 2024-05-10T04:03:39.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.424+0000 7f38deffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e0101030 0x7f38e0100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:39.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.424+0000 7f38deffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e0101030 0x7f38e0100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52488/0 (socket says 172.21.15.150:52488) 2024-05-10T04:03:39.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.424+0000 7f38deffd700 1 -- 172.21.15.150:0/1434714127 learned_addr learned my addr 172.21.15.150:0/1434714127 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:39.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38df7fe700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38e0102590 0x7f38e019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:39.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38de7fc700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0101a30 0x7f38e019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:39.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38deffd700 1 -- 172.21.15.150:0/1434714127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0101a30 msgr2=0x7f38e019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:39.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38deffd700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0101a30 0x7f38e019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38deffd700 1 -- 172.21.15.150:0/1434714127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38e0102590 msgr2=0x7f38e019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:39.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38deffd700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38e0102590 0x7f38e019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38deffd700 1 -- 172.21.15.150:0/1434714127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d00095f0 con 0x7f38e0101030 2024-05-10T04:03:39.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38de7fc700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0101a30 0x7f38e019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:39.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38deffd700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e0101030 0x7f38e0100710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f38d400d950 tx=0x7f38d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:39.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38c7fff700 1 -- 172.21.15.150:0/1434714127 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4004220 con 0x7f38e0101030 2024-05-10T04:03:39.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38c7fff700 1 -- 172.21.15.150:0/1434714127 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38d4004380 con 0x7f38e0101030 2024-05-10T04:03:39.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38e56d4700 1 -- 172.21.15.150:0/1434714127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e019ffc0 con 0x7f38e0101030 2024-05-10T04:03:39.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38c7fff700 1 -- 172.21.15.150:0/1434714127 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4003bf0 con 0x7f38e0101030 2024-05-10T04:03:39.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.425+0000 7f38e56d4700 1 -- 172.21.15.150:0/1434714127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e01a0440 con 0x7f38e0101030 2024-05-10T04:03:39.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.426+0000 7f38e56d4700 1 -- 172.21.15.150:0/1434714127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e0107e90 con 0x7f38e0101030 2024-05-10T04:03:39.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.428+0000 7f38c7fff700 1 -- 172.21.15.150:0/1434714127 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f38d4003d50 con 0x7f38e0101030 2024-05-10T04:03:39.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.428+0000 7f38c7fff700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38c80a12e0 0x7f38c80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:39.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.429+0000 7f38c7fff700 1 -- 172.21.15.150:0/1434714127 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f38d40ccff0 con 0x7f38e0101030 2024-05-10T04:03:39.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.429+0000 7f38de7fc700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38c80a12e0 0x7f38c80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:39.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.429+0000 7f38de7fc700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38c80a12e0 0x7f38c80a37f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f38d000b5c0 tx=0x7f38d001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:39.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.432+0000 7f38c7fff700 1 -- 172.21.15.150:0/1434714127 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f38d4096330 con 0x7f38e0101030 2024-05-10T04:03:39.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.630+0000 7f38e56d4700 1 -- 172.21.15.150:0/1434714127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f38e010c300 con 0x7f38e0101030 2024-05-10T04:03:39.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.630+0000 7f38c7fff700 1 -- 172.21.15.150:0/1434714127 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f38d4017070 con 0x7f38e0101030 2024-05-10T04:03:39.632 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:03:39.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.632+0000 7f38e56d4700 1 -- 172.21.15.150:0/1434714127 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38c80a12e0 msgr2=0x7f38c80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:39.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.632+0000 7f38e56d4700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38c80a12e0 0x7f38c80a37f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f38d000b5c0 tx=0x7f38d001a040 comp rx=0 tx=0).stop 2024-05-10T04:03:39.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.632+0000 7f38e56d4700 1 -- 172.21.15.150:0/1434714127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e0101030 msgr2=0x7f38e0100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:39.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.632+0000 7f38e56d4700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e0101030 0x7f38e0100710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f38d400d950 tx=0x7f38d400dc60 comp rx=0 tx=0).stop 2024-05-10T04:03:39.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.633+0000 7f38e56d4700 1 -- 172.21.15.150:0/1434714127 shutdown_connections 2024-05-10T04:03:39.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.633+0000 7f38e56d4700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38c80a12e0 0x7f38c80a37f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.633+0000 7f38e56d4700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38e0102590 0x7f38e019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.633+0000 7f38e56d4700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e0101030 0x7f38e0100710 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.633+0000 7f38e56d4700 1 --2- 172.21.15.150:0/1434714127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0101a30 0x7f38e019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.633+0000 7f38e56d4700 1 -- 172.21.15.150:0/1434714127 >> 172.21.15.150:0/1434714127 conn(0x7f38e00fb8b0 msgr2=0x7f38e0104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:39.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.633+0000 7f38e56d4700 1 -- 172.21.15.150:0/1434714127 shutdown_connections 2024-05-10T04:03:39.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.633+0000 7f38e56d4700 1 -- 172.21.15.150:0/1434714127 wait complete. 2024-05-10T04:03:39.647 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:39 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1905303821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:03:39.647 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:39 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2007138213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:03:39.647 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:39 smithi150 ceph-mon[42477]: pgmap v262: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 23 B/s, 1 keys/s, 1 objects/s recovering 2024-05-10T04:03:39.647 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:39 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1281436780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:03:39.648 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:03:39.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:39 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1905303821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:03:39.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:39 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2007138213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:03:39.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:39 smithi181 ceph-mon[46602]: pgmap v262: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 23 B/s, 1 keys/s, 1 objects/s recovering 2024-05-10T04:03:39.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:39 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1281436780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:03:39.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:39 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1905303821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:03:39.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:39 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2007138213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:03:39.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:39 smithi161 ceph-mon[46918]: pgmap v262: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 23 B/s, 1 keys/s, 1 objects/s recovering 2024-05-10T04:03:39.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:39 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1281436780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:03:39.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.779+0000 7fd68c090700 1 -- 172.21.15.150:0/796587226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd684101bd0 msgr2=0x7fd684101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:39.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.779+0000 7fd68c090700 1 --2- 172.21.15.150:0/796587226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd684101bd0 0x7fd684101fe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd680009990 tx=0x7fd680009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.779+0000 7fd68c090700 1 -- 172.21.15.150:0/796587226 shutdown_connections 2024-05-10T04:03:39.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.779+0000 7fd68c090700 1 --2- 172.21.15.150:0/796587226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd684103690 0x7fd684109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.779+0000 7fd68c090700 1 --2- 172.21.15.150:0/796587226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd684101bd0 0x7fd684101fe0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.779+0000 7fd68c090700 1 --2- 172.21.15.150:0/796587226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd684102cf0 0x7fd684103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.779+0000 7fd68c090700 1 -- 172.21.15.150:0/796587226 >> 172.21.15.150:0/796587226 conn(0x7fd6840fd1e0 msgr2=0x7fd6840ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:39.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.779+0000 7fd68c090700 1 -- 172.21.15.150:0/796587226 shutdown_connections 2024-05-10T04:03:39.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.779+0000 7fd68c090700 1 -- 172.21.15.150:0/796587226 wait complete. 2024-05-10T04:03:39.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68c090700 1 Processor -- start 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68c090700 1 -- start start 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68c090700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd684101bd0 0x7fd684198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68c090700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd684102cf0 0x7fd6841985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68c090700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd684103690 0x7fd68419f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68c090700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68419fb40 con 0x7fd684103690 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68c090700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68419fcb0 con 0x7fd684102cf0 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68c090700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68419fe70 con 0x7fd684101bd0 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68a62d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd684103690 0x7fd68419f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68a62d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd684103690 0x7fd68419f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52510/0 (socket says 172.21.15.150:52510) 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68a62d700 1 -- 172.21.15.150:0/2200195397 learned_addr learned my addr 172.21.15.150:0/2200195397 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd689e2c700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd684101bd0 0x7fd684198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68962b700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd684102cf0 0x7fd6841985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68a62d700 1 -- 172.21.15.150:0/2200195397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd684101bd0 msgr2=0x7fd684198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:39.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68a62d700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd684101bd0 0x7fd684198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68a62d700 1 -- 172.21.15.150:0/2200195397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd684102cf0 msgr2=0x7fd6841985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:39.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68a62d700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd684102cf0 0x7fd6841985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:39.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68a62d700 1 -- 172.21.15.150:0/2200195397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6800095f0 con 0x7fd684103690 2024-05-10T04:03:39.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68962b700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd684102cf0 0x7fd6841985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:39.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.780+0000 7fd68a62d700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd684103690 0x7fd68419f530 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00ef30 tx=0x7fd67c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:39.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.781+0000 7fd67affd700 1 -- 172.21.15.150:0/2200195397 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c018070 con 0x7fd684103690 2024-05-10T04:03:39.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.781+0000 7fd67affd700 1 -- 172.21.15.150:0/2200195397 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd67c00f040 con 0x7fd684103690 2024-05-10T04:03:39.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.781+0000 7fd68c090700 1 -- 172.21.15.150:0/2200195397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6841a00f0 con 0x7fd684103690 2024-05-10T04:03:39.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.781+0000 7fd67affd700 1 -- 172.21.15.150:0/2200195397 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c003dc0 con 0x7fd684103690 2024-05-10T04:03:39.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.781+0000 7fd68c090700 1 -- 172.21.15.150:0/2200195397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6841a05b0 con 0x7fd684103690 2024-05-10T04:03:39.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.782+0000 7fd68c090700 1 -- 172.21.15.150:0/2200195397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd684108de0 con 0x7fd684103690 2024-05-10T04:03:39.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.783+0000 7fd67affd700 1 -- 172.21.15.150:0/2200195397 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd67c00caa0 con 0x7fd684103690 2024-05-10T04:03:39.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.783+0000 7fd67affd700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd6700a11b0 0x7fd6700a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:39.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.784+0000 7fd689e2c700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd6700a11b0 0x7fd6700a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:39.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.784+0000 7fd67affd700 1 -- 172.21.15.150:0/2200195397 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd67c0cd360 con 0x7fd684103690 2024-05-10T04:03:39.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.784+0000 7fd689e2c700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd6700a11b0 0x7fd6700a36c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd68000b7b0 tx=0x7fd68001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:39.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:39.787+0000 7fd67affd700 1 -- 172.21.15.150:0/2200195397 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd67c0966b0 con 0x7fd684103690 2024-05-10T04:03:40.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.031+0000 7fd68c090700 1 -- 172.21.15.150:0/2200195397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6840678b0 con 0x7fd684103690 2024-05-10T04:03:40.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.032+0000 7fd67affd700 1 -- 172.21.15.150:0/2200195397 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10886 (secure 0 0 0) 0x7fd67c099cd0 con 0x7fd684103690 2024-05-10T04:03:40.034 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:40.034 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:02:57.183845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:03:40.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.034+0000 7fd68c090700 1 -- 172.21.15.150:0/2200195397 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd6700a11b0 msgr2=0x7fd6700a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.034+0000 7fd68c090700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd6700a11b0 0x7fd6700a36c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd68000b7b0 tx=0x7fd68001a040 comp rx=0 tx=0).stop 2024-05-10T04:03:40.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.034+0000 7fd68c090700 1 -- 172.21.15.150:0/2200195397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd684103690 msgr2=0x7fd68419f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.034+0000 7fd68c090700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd684103690 0x7fd68419f530 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00ef30 tx=0x7fd67c00c450 comp rx=0 tx=0).stop 2024-05-10T04:03:40.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.034+0000 7fd68c090700 1 -- 172.21.15.150:0/2200195397 shutdown_connections 2024-05-10T04:03:40.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.034+0000 7fd68c090700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd6700a11b0 0x7fd6700a36c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.034+0000 7fd68c090700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd684103690 0x7fd68419f530 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.034+0000 7fd68c090700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd684101bd0 0x7fd684198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.034+0000 7fd68c090700 1 --2- 172.21.15.150:0/2200195397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd684102cf0 0x7fd6841985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.034+0000 7fd68c090700 1 -- 172.21.15.150:0/2200195397 >> 172.21.15.150:0/2200195397 conn(0x7fd6840fd1e0 msgr2=0x7fd684105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:40.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.034+0000 7fd68c090700 1 -- 172.21.15.150:0/2200195397 shutdown_connections 2024-05-10T04:03:40.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.034+0000 7fd68c090700 1 -- 172.21.15.150:0/2200195397 wait complete. 2024-05-10T04:03:40.037 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 24 2024-05-10T04:03:40.050 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:03:40.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.180+0000 7ff7a09f0700 1 -- 172.21.15.150:0/859076320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c102630 msgr2=0x7ff79c108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.180+0000 7ff7a09f0700 1 --2- 172.21.15.150:0/859076320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c102630 0x7ff79c108ee0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff79400b0a0 tx=0x7ff79400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.180+0000 7ff7a09f0700 1 -- 172.21.15.150:0/859076320 shutdown_connections 2024-05-10T04:03:40.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.180+0000 7ff7a09f0700 1 --2- 172.21.15.150:0/859076320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c102630 0x7ff79c108ee0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.180+0000 7ff7a09f0700 1 --2- 172.21.15.150:0/859076320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff79c100b70 0x7ff79c100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.180+0000 7ff7a09f0700 1 --2- 172.21.15.150:0/859076320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c101c90 0x7ff79c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.180+0000 7ff7a09f0700 1 -- 172.21.15.150:0/859076320 >> 172.21.15.150:0/859076320 conn(0x7ff79c0fc180 msgr2=0x7ff79c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:40.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.180+0000 7ff7a09f0700 1 -- 172.21.15.150:0/859076320 shutdown_connections 2024-05-10T04:03:40.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.180+0000 7ff7a09f0700 1 -- 172.21.15.150:0/859076320 wait complete. 2024-05-10T04:03:40.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.181+0000 7ff7a09f0700 1 Processor -- start 2024-05-10T04:03:40.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.181+0000 7ff7a09f0700 1 -- start start 2024-05-10T04:03:40.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.181+0000 7ff7a09f0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c100b70 0x7ff79c196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.181+0000 7ff7a09f0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff79c101c90 0x7ff79c1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.181+0000 7ff7a09f0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c102630 0x7ff79c19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.181+0000 7ff7a09f0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c19ea40 con 0x7ff79c100b70 2024-05-10T04:03:40.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.181+0000 7ff7a09f0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c19eb80 con 0x7ff79c102630 2024-05-10T04:03:40.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.181+0000 7ff7a09f0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c19ed10 con 0x7ff79c101c90 2024-05-10T04:03:40.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.181+0000 7ff79a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c100b70 0x7ff79c196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.181+0000 7ff79a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c100b70 0x7ff79c196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52530/0 (socket says 172.21.15.150:52530) 2024-05-10T04:03:40.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.181+0000 7ff79a59c700 1 -- 172.21.15.150:0/1735420504 learned_addr learned my addr 172.21.15.150:0/1735420504 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:40.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff799d9b700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff79c101c90 0x7ff79c1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff79a59c700 1 -- 172.21.15.150:0/1735420504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff79c101c90 msgr2=0x7ff79c1974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff79a59c700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff79c101c90 0x7ff79c1974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff79a59c700 1 -- 172.21.15.150:0/1735420504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c102630 msgr2=0x7ff79c19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff79ad9d700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c102630 0x7ff79c19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff79a59c700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c102630 0x7ff79c19e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff79a59c700 1 -- 172.21.15.150:0/1735420504 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff794009d00 con 0x7ff79c100b70 2024-05-10T04:03:40.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff799d9b700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff79c101c90 0x7ff79c1974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:40.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff79a59c700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c100b70 0x7ff79c196fa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff79000c9a0 tx=0x7ff79000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:40.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff78b7fe700 1 -- 172.21.15.150:0/1735420504 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790007ac0 con 0x7ff79c100b70 2024-05-10T04:03:40.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff78b7fe700 1 -- 172.21.15.150:0/1735420504 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff790007c20 con 0x7ff79c100b70 2024-05-10T04:03:40.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff7a09f0700 1 -- 172.21.15.150:0/1735420504 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79c19eff0 con 0x7ff79c100b70 2024-05-10T04:03:40.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff78b7fe700 1 -- 172.21.15.150:0/1735420504 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790018660 con 0x7ff79c100b70 2024-05-10T04:03:40.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.182+0000 7ff7a09f0700 1 -- 172.21.15.150:0/1735420504 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79c19f470 con 0x7ff79c100b70 2024-05-10T04:03:40.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.183+0000 7ff7a09f0700 1 -- 172.21.15.150:0/1735420504 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79c107d80 con 0x7ff79c100b70 2024-05-10T04:03:40.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.184+0000 7ff78b7fe700 1 -- 172.21.15.150:0/1735420504 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff790004020 con 0x7ff79c100b70 2024-05-10T04:03:40.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.185+0000 7ff78b7fe700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff7840a1230 0x7ff7840a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.185+0000 7ff78b7fe700 1 -- 172.21.15.150:0/1735420504 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff7900cce20 con 0x7ff79c100b70 2024-05-10T04:03:40.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.185+0000 7ff799d9b700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff7840a1230 0x7ff7840a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.186+0000 7ff799d9b700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff7840a1230 0x7ff7840a3740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff78c005fd0 tx=0x7ff78c005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:40.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.189+0000 7ff78b7fe700 1 -- 172.21.15.150:0/1735420504 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff790096250 con 0x7ff79c100b70 2024-05-10T04:03:40.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.386+0000 7ff7a09f0700 1 -- 172.21.15.150:0/1735420504 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff79c10c310 con 0x7ff79c100b70 2024-05-10T04:03:40.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.387+0000 7ff78b7fe700 1 -- 172.21.15.150:0/1735420504 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24554 (secure 0 0 0) 0x7ff790099870 con 0x7ff79c100b70 2024-05-10T04:03:40.390 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:40.390 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":87,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:03:30.185138+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:03:40.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.390+0000 7ff7a09f0700 1 -- 172.21.15.150:0/1735420504 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff7840a1230 msgr2=0x7ff7840a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.390+0000 7ff7a09f0700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff7840a1230 0x7ff7840a3740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff78c005fd0 tx=0x7ff78c005f00 comp rx=0 tx=0).stop 2024-05-10T04:03:40.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.390+0000 7ff7a09f0700 1 -- 172.21.15.150:0/1735420504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c100b70 msgr2=0x7ff79c196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.390+0000 7ff7a09f0700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c100b70 0x7ff79c196fa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff79000c9a0 tx=0x7ff79000cd60 comp rx=0 tx=0).stop 2024-05-10T04:03:40.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.390+0000 7ff7a09f0700 1 -- 172.21.15.150:0/1735420504 shutdown_connections 2024-05-10T04:03:40.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.390+0000 7ff7a09f0700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff7840a1230 0x7ff7840a3740 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.390+0000 7ff7a09f0700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c102630 0x7ff79c19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.390+0000 7ff7a09f0700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c100b70 0x7ff79c196fa0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.390+0000 7ff7a09f0700 1 --2- 172.21.15.150:0/1735420504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff79c101c90 0x7ff79c1974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.390+0000 7ff7a09f0700 1 -- 172.21.15.150:0/1735420504 >> 172.21.15.150:0/1735420504 conn(0x7ff79c0fc180 msgr2=0x7ff79c104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:40.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.390+0000 7ff7a09f0700 1 -- 172.21.15.150:0/1735420504 shutdown_connections 2024-05-10T04:03:40.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.390+0000 7ff7a09f0700 1 -- 172.21.15.150:0/1735420504 wait complete. 2024-05-10T04:03:40.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:03:40.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.537+0000 7f80bcac1700 1 -- 172.21.15.150:0/3966828267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80b8102580 msgr2=0x7f80b8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.537+0000 7f80bcac1700 1 --2- 172.21.15.150:0/3966828267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80b8102580 0x7f80b8108e30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f80b000b0a0 tx=0x7f80b000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.537+0000 7f80bcac1700 1 -- 172.21.15.150:0/3966828267 shutdown_connections 2024-05-10T04:03:40.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.537+0000 7f80bcac1700 1 --2- 172.21.15.150:0/3966828267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80b8102580 0x7f80b8108e30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.537+0000 7f80bcac1700 1 --2- 172.21.15.150:0/3966828267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b8100ac0 0x7f80b8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.537+0000 7f80bcac1700 1 --2- 172.21.15.150:0/3966828267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80b8101be0 0x7f80b8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.537+0000 7f80bcac1700 1 -- 172.21.15.150:0/3966828267 >> 172.21.15.150:0/3966828267 conn(0x7f80b80fc0d0 msgr2=0x7f80b80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:40.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.537+0000 7f80bcac1700 1 -- 172.21.15.150:0/3966828267 shutdown_connections 2024-05-10T04:03:40.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.537+0000 7f80bcac1700 1 -- 172.21.15.150:0/3966828267 wait complete. 2024-05-10T04:03:40.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.538+0000 7f80bcac1700 1 Processor -- start 2024-05-10T04:03:40.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.538+0000 7f80bcac1700 1 -- start start 2024-05-10T04:03:40.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.538+0000 7f80bcac1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b8100ac0 0x7f80b8068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.538+0000 7f80bcac1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80b8101be0 0x7f80b819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.538+0000 7f80bcac1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80b8102580 0x7f80b819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.538+0000 7f80bcac1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819fb30 con 0x7f80b8101be0 2024-05-10T04:03:40.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.538+0000 7f80bcac1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819fca0 con 0x7f80b8100ac0 2024-05-10T04:03:40.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.538+0000 7f80bcac1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819fe60 con 0x7f80b8102580 2024-05-10T04:03:40.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.538+0000 7f80b5d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80b8101be0 0x7f80b819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.538+0000 7f80b5d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80b8101be0 0x7f80b819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52544/0 (socket says 172.21.15.150:52544) 2024-05-10T04:03:40.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.538+0000 7f80b5d9b700 1 -- 172.21.15.150:0/2657839981 learned_addr learned my addr 172.21.15.150:0/2657839981 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:40.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.538+0000 7f80b659c700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b8100ac0 0x7f80b8068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.539+0000 7f80b6d9d700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80b8102580 0x7f80b819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.539+0000 7f80b5d9b700 1 -- 172.21.15.150:0/2657839981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80b8102580 msgr2=0x7f80b819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.539+0000 7f80b5d9b700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80b8102580 0x7f80b819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.539+0000 7f80b5d9b700 1 -- 172.21.15.150:0/2657839981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b8100ac0 msgr2=0x7f80b8068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.539+0000 7f80b5d9b700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b8100ac0 0x7f80b8068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.539+0000 7f80b5d9b700 1 -- 172.21.15.150:0/2657839981 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b0009d00 con 0x7f80b8101be0 2024-05-10T04:03:40.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.539+0000 7f80b6d9d700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80b8102580 0x7f80b819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:40.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.539+0000 7f80b5d9b700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80b8101be0 0x7f80b819ce20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00d980 tx=0x7f80ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:40.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.539+0000 7f80a77fe700 1 -- 172.21.15.150:0/2657839981 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac004220 con 0x7f80b8101be0 2024-05-10T04:03:40.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.539+0000 7f80a77fe700 1 -- 172.21.15.150:0/2657839981 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80ac007640 con 0x7f80b8101be0 2024-05-10T04:03:40.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.539+0000 7f80bcac1700 1 -- 172.21.15.150:0/2657839981 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b81a0140 con 0x7f80b8101be0 2024-05-10T04:03:40.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.539+0000 7f80a77fe700 1 -- 172.21.15.150:0/2657839981 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac005610 con 0x7f80b8101be0 2024-05-10T04:03:40.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.539+0000 7f80bcac1700 1 -- 172.21.15.150:0/2657839981 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b81a0650 con 0x7f80b8101be0 2024-05-10T04:03:40.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.540+0000 7f80bcac1700 1 -- 172.21.15.150:0/2657839981 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b8107cd0 con 0x7f80b8101be0 2024-05-10T04:03:40.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.542+0000 7f80a77fe700 1 -- 172.21.15.150:0/2657839981 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f80ac004750 con 0x7f80b8101be0 2024-05-10T04:03:40.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.542+0000 7f80a77fe700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f80a00a11b0 0x7f80a00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.542+0000 7f80b659c700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f80a00a11b0 0x7f80a00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.542+0000 7f80a77fe700 1 -- 172.21.15.150:0/2657839981 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f80ac0cd0d0 con 0x7f80b8101be0 2024-05-10T04:03:40.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.543+0000 7f80b659c700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f80a00a11b0 0x7f80a00a36c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f80b8101ae0 tx=0x7f80a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:40.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.546+0000 7f80a77fe700 1 -- 172.21.15.150:0/2657839981 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f80ac0963f0 con 0x7f80b8101be0 2024-05-10T04:03:40.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:40 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1434714127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:03:40.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:40 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2200195397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:40.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:40 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1735420504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:40.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:40 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1434714127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:03:40.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:40 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2200195397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:40.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:40 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1735420504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:40.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:40 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1434714127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:03:40.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:40 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2200195397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:40.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:40 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1735420504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:40.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.786+0000 7f80bcac1700 1 -- 172.21.15.150:0/2657839981 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80b80694d0 con 0x7f80b8101be0 2024-05-10T04:03:40.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.787+0000 7f80a77fe700 1 -- 172.21.15.150:0/2657839981 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10886 (secure 0 0 0) 0x7f80ac016090 con 0x7f80b8101be0 2024-05-10T04:03:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:40.790 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:02:57.183845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:03:40.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.789+0000 7f80bcac1700 1 -- 172.21.15.150:0/2657839981 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f80a00a11b0 msgr2=0x7f80a00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.789+0000 7f80bcac1700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f80a00a11b0 0x7f80a00a36c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f80b8101ae0 tx=0x7f80a8009260 comp rx=0 tx=0).stop 2024-05-10T04:03:40.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.789+0000 7f80bcac1700 1 -- 172.21.15.150:0/2657839981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80b8101be0 msgr2=0x7f80b819ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.789+0000 7f80bcac1700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80b8101be0 0x7f80b819ce20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00d980 tx=0x7f80ac00dc90 comp rx=0 tx=0).stop 2024-05-10T04:03:40.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.789+0000 7f80bcac1700 1 -- 172.21.15.150:0/2657839981 shutdown_connections 2024-05-10T04:03:40.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.789+0000 7f80bcac1700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f80a00a11b0 0x7f80a00a36c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.789+0000 7f80bcac1700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80b8102580 0x7f80b819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.789+0000 7f80bcac1700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b8100ac0 0x7f80b8068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.789+0000 7f80bcac1700 1 --2- 172.21.15.150:0/2657839981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80b8101be0 0x7f80b819ce20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.789+0000 7f80bcac1700 1 -- 172.21.15.150:0/2657839981 >> 172.21.15.150:0/2657839981 conn(0x7f80b80fc0d0 msgr2=0x7f80b8104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:40.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.790+0000 7f80bcac1700 1 -- 172.21.15.150:0/2657839981 shutdown_connections 2024-05-10T04:03:40.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.790+0000 7f80bcac1700 1 -- 172.21.15.150:0/2657839981 wait complete. 2024-05-10T04:03:40.793 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 24 2024-05-10T04:03:40.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-10T04:02:31.605102+0000', 'modified': '2024-05-10T04:02:57.183845+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24557, 'mds_1': 24556, 'mds_2': 24535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24484': {'gid': 24484, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6833/2987418700', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6832', 'nonce': 2987418700}, {'type': 'v1', 'addr': '172.21.15.181:6833', 'nonce': 2987418700}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24503': {'gid': 24503, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/1403383813', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 1403383813}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 1403383813}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6835/501599730', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 501599730}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 501599730}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.181:6837/1798187480', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 1798187480}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1798187480}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24556': {'gid': 24556, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.181:6839/1415332890', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1415332890}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1415332890}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6839/267491715', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 267491715}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 267491715}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T04:03:40.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-10T04:03:40.806 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T04:03:40.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.937+0000 7f159875c700 1 -- 172.21.15.150:0/2656111274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1590069090 msgr2=0x7f1590104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.937+0000 7f159875c700 1 --2- 172.21.15.150:0/2656111274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1590069090 0x7f1590104720 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1580009990 tx=0x7f1580009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.937+0000 7f159875c700 1 -- 172.21.15.150:0/2656111274 shutdown_connections 2024-05-10T04:03:40.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.937+0000 7f159875c700 1 --2- 172.21.15.150:0/2656111274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1590107880 0x7f1590109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.937+0000 7f159875c700 1 --2- 172.21.15.150:0/2656111274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1590069090 0x7f1590104720 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.937+0000 7f159875c700 1 --2- 172.21.15.150:0/2656111274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1590104d10 0x7f15901071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.937+0000 7f159875c700 1 -- 172.21.15.150:0/2656111274 >> 172.21.15.150:0/2656111274 conn(0x7f15900fc0d0 msgr2=0x7f15900fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.937+0000 7f159875c700 1 -- 172.21.15.150:0/2656111274 shutdown_connections 2024-05-10T04:03:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.937+0000 7f159875c700 1 -- 172.21.15.150:0/2656111274 wait complete. 2024-05-10T04:03:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f159875c700 1 Processor -- start 2024-05-10T04:03:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f159875c700 1 -- start start 2024-05-10T04:03:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f159875c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1590069090 0x7f1590101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f159875c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1590104d10 0x7f1590100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f159875c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1590107880 0x7f1590100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f159875c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15901011c0 con 0x7f1590104d10 2024-05-10T04:03:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f159875c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1590101330 con 0x7f1590107880 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f159875c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15901014f0 con 0x7f1590069090 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f1595cf7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1590104d10 0x7f1590100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f1595cf7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1590104d10 0x7f1590100620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52550/0 (socket says 172.21.15.150:52550) 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f1595cf7700 1 -- 172.21.15.150:0/1524314434 learned_addr learned my addr 172.21.15.150:0/1524314434 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f15964f8700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1590069090 0x7f1590101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f1596cf9700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1590107880 0x7f1590100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f1595cf7700 1 -- 172.21.15.150:0/1524314434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1590069090 msgr2=0x7f1590101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f1595cf7700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1590069090 0x7f1590101f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f1595cf7700 1 -- 172.21.15.150:0/1524314434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1590107880 msgr2=0x7f1590100b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f1595cf7700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1590107880 0x7f1590100b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.938+0000 7f1595cf7700 1 -- 172.21.15.150:0/1524314434 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15800095f0 con 0x7f1590104d10 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.939+0000 7f1595cf7700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1590104d10 0x7f1590100620 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c9b0 tx=0x7f158c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.939+0000 7f15877fe700 1 -- 172.21.15.150:0/1524314434 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c007b00 con 0x7f1590104d10 2024-05-10T04:03:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.939+0000 7f159875c700 1 -- 172.21.15.150:0/1524314434 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15901a2180 con 0x7f1590104d10 2024-05-10T04:03:40.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.939+0000 7f15877fe700 1 -- 172.21.15.150:0/1524314434 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f158c007c60 con 0x7f1590104d10 2024-05-10T04:03:40.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.939+0000 7f15877fe700 1 -- 172.21.15.150:0/1524314434 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c018660 con 0x7f1590104d10 2024-05-10T04:03:40.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.939+0000 7f159875c700 1 -- 172.21.15.150:0/1524314434 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15901a2690 con 0x7f1590104d10 2024-05-10T04:03:40.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.941+0000 7f15877fe700 1 -- 172.21.15.150:0/1524314434 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f158c018880 con 0x7f1590104d10 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.942+0000 7f15877fe700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f157c0a1260 0x7f157c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.942+0000 7f15877fe700 1 -- 172.21.15.150:0/1524314434 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f158c0cce70 con 0x7f1590104d10 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.942+0000 7f15964f8700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f157c0a1260 0x7f157c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.942+0000 7f15964f8700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f157c0a1260 0x7f157c0a3770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1580003730 tx=0x7f158001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.942+0000 7f15a8aa0b80 1 Processor -- start 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f15a8aa0b80 1 -- start start 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f15a8aa0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8a650 0x55c0b8d8aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f15a8aa0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 0x55c0b8d8b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f15a8aa0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d92770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f15a8aa0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0b8cb1100 con 0x55c0b8d8b520 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f15a8aa0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0b8caeac0 con 0x55c0b8d8be70 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f15a8aa0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0b8caa3b0 con 0x55c0b8d8a650 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f1595cf7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 0x55c0b8d8b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f1595cf7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 0x55c0b8d8b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52564/0 (socket says 172.21.15.150:52564) 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f1595cf7700 1 -- 172.21.15.150:0/1799924010 learned_addr learned my addr 172.21.15.150:0/1799924010 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f15964f8700 1 --2- 172.21.15.150:0/1799924010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8a650 0x55c0b8d8aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f1596cf9700 1 --2- 172.21.15.150:0/1799924010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d92770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f1595cf7700 1 -- 172.21.15.150:0/1799924010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8a650 msgr2=0x55c0b8d8aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f1595cf7700 1 --2- 172.21.15.150:0/1799924010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8a650 0x55c0b8d8aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f1595cf7700 1 -- 172.21.15.150:0/1799924010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 msgr2=0x55c0b8d92770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f1595cf7700 1 --2- 172.21.15.150:0/1799924010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d92770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f1595cf7700 1 -- 172.21.15.150:0/1799924010 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0b8b2b360 con 0x55c0b8d8b520 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.943+0000 7f1596cf9700 1 --2- 172.21.15.150:0/1799924010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d92770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.944+0000 7f1595cf7700 1 --2- 172.21.15.150:0/1799924010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 0x55c0b8d8b930 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f158c099de0 tx=0x7f158c007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41481ff3c7342072 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.944+0000 7f1584ff9700 1 -- 172.21.15.150:0/1799924010 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c0b1070 con 0x55c0b8d8b520 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.944+0000 7f1584ff9700 1 -- 172.21.15.150:0/1799924010 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f158c0a22d0 con 0x55c0b8d8b520 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.944+0000 7f1584ff9700 1 -- 172.21.15.150:0/1799924010 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c09a310 con 0x55c0b8d8b520 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/1799924010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 msgr2=0x55c0b8d8b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/1799924010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 0x55c0b8d8b930 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f158c099de0 tx=0x7f158c007f00 comp rx=0 tx=0).stop 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/1799924010 shutdown_connections 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/1799924010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d92770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/1799924010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8a650 0x55c0b8d8aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/1799924010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 0x55c0b8d8b930 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/1799924010 >> 172.21.15.150:0/1799924010 conn(0x55c0b8ceca10 msgr2=0x55c0b8d84f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/1799924010 shutdown_connections 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/1799924010 wait complete. 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 -- start start 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 Processor -- start 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8a650 0x55c0b8d94730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 0x55c0b8d94c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d9bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0b8c46510 con 0x55c0b8d8b520 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0b8c461b0 con 0x55c0b8d8be70 2024-05-10T04:03:40.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0b8b2d650 con 0x55c0b8d8a650 2024-05-10T04:03:40.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.945+0000 7f15a8aa0b80 10 client.0 _ll_register_callbacks cb 0x55c0b8c4fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:03:40.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1595cf7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 0x55c0b8d94c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1595cf7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 0x55c0b8d94c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52572/0 (socket says 172.21.15.150:52572) 2024-05-10T04:03:40.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1595cf7700 1 -- 172.21.15.150:0/4081920437 learned_addr learned my addr 172.21.15.150:0/4081920437 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:40.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f15964f8700 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8a650 0x55c0b8d94730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1596cf9700 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d9bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1595cf7700 1 -- 172.21.15.150:0/4081920437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8a650 msgr2=0x55c0b8d94730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:40.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1595cf7700 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8a650 0x55c0b8d94730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1595cf7700 1 -- 172.21.15.150:0/4081920437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 msgr2=0x55c0b8d9bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:40.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1595cf7700 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d9bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1595cf7700 1 -- 172.21.15.150:0/4081920437 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15901a2690 con 0x55c0b8d8b520 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f15964f8700 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8a650 0x55c0b8d94730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1596cf9700 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d9bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1595cf7700 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 0x55c0b8d94c70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f158c099b50 tx=0x7f158c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39c4cb9a67483ab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1579ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1579ffb700 1 -- 172.21.15.150:0/4081920437 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c0b1030 con 0x55c0b8d8b520 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1579ffb700 1 -- 172.21.15.150:0/4081920437 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f158c0b2070 con 0x55c0b8d8b520 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/4081920437 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0b8c03220 con 0x55c0b8d8b520 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1579ffb700 1 -- 172.21.15.150:0/4081920437 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f158c09a0b0 con 0x55c0b8d8b520 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f15a8aa0b80 10 client.15258 fetch_fsmap learned FSMap version 24 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/4081920437 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c0b8ad7b10 con 0x55c0b8d8b520 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.946+0000 7f1579ffb700 1 -- 172.21.15.150:0/4081920437 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c0b8c03220 con 0x55c0b8d8b520 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.947+0000 7f1579ffb700 1 -- 172.21.15.150:0/4081920437 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f158c0add20 con 0x55c0b8d8b520 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.947+0000 7f15a8aa0b80 10 client.15258 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.947+0000 7f15a8aa0b80 10 client.15258 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T04:03:40.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.947+0000 7f15a8aa0b80 20 client.15258 populate_metadata read hostname 'smithi150' 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.947+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55c0b8d9fd40 0x55c0b8da4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.947+0000 7f15a8aa0b80 4 client.15258 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.947+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/4081920437 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0b8d59730 con 0x55c0b8d9fd40 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.947+0000 7f15964f8700 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55c0b8d9fd40 0x55c0b8da4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.948+0000 7f1579ffb700 10 client.15258 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.948+0000 7f15964f8700 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55c0b8d9fd40 0x55c0b8da4480 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=590fb157464a6bbf server_cookie=b1779f600324f4fa in_seq=0 out_seq=0 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.948+0000 7f1579ffb700 1 -- 172.21.15.150:0/4081920437 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c0b8d59730 con 0x55c0b8d9fd40 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.948+0000 7f1579ffb700 10 client.15258 handle_command_reply: tid=0 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.948+0000 7f15a8aa0b80 1 client.15258 shutdown 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.949+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/4081920437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 msgr2=0x55c0b8d94c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.949+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 0x55c0b8d94c70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f158c099b50 tx=0x7f158c000bc0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.949+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/4081920437 shutdown_connections 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.949+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d9bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.949+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55c0b8d9fd40 0x55c0b8da4480 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.949+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8a650 0x55c0b8d94730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.949+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/4081920437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8b520 0x55c0b8d94c70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.949+0000 7f1579ffb700 0 client.15258 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:03:40.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.949+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/4081920437 >> 172.21.15.150:0/4081920437 conn(0x55c0b8ceca10 msgr2=0x55c0b8d84f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:40.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.949+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/4081920437 shutdown_connections 2024-05-10T04:03:40.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.949+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/4081920437 wait complete. 2024-05-10T04:03:40.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.949+0000 7f15a8aa0b80 20 client.15258 trim_cache size 0 max 16384 2024-05-10T04:03:40.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.967+0000 7f15a8aa0b80 1 Processor -- start 2024-05-10T04:03:40.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.967+0000 7f15a8aa0b80 1 -- start start 2024-05-10T04:03:40.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.967+0000 7f15a8aa0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8a650 0x55c0b8d93970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.967+0000 7f15a8aa0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8b520 0x55c0b8d99b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.967+0000 7f15a8aa0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d9a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.967+0000 7f15a8aa0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0b8c03220 con 0x55c0b8d8a650 2024-05-10T04:03:40.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.967+0000 7f15a8aa0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0b8c682b0 con 0x55c0b8d8be70 2024-05-10T04:03:40.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.967+0000 7f15a8aa0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0b8ca7010 con 0x55c0b8d8b520 2024-05-10T04:03:40.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.967+0000 7f15964f8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8a650 0x55c0b8d93970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.967+0000 7f15964f8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8a650 0x55c0b8d93970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52578/0 (socket says 172.21.15.150:52578) 2024-05-10T04:03:40.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.967+0000 7f15964f8700 1 -- 172.21.15.150:0/1995083720 learned_addr learned my addr 172.21.15.150:0/1995083720 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:40.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.967+0000 7f1595cf7700 1 --2- 172.21.15.150:0/1995083720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8b520 0x55c0b8d99b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.967+0000 7f1596cf9700 1 --2- 172.21.15.150:0/1995083720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d9a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.968+0000 7f15964f8700 1 -- 172.21.15.150:0/1995083720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8b520 msgr2=0x55c0b8d99b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:40.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.968+0000 7f15964f8700 1 --2- 172.21.15.150:0/1995083720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8b520 0x55c0b8d99b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.968+0000 7f15964f8700 1 -- 172.21.15.150:0/1995083720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 msgr2=0x55c0b8d9a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:40.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.968+0000 7f15964f8700 1 --2- 172.21.15.150:0/1995083720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d9a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.968+0000 7f15964f8700 1 -- 172.21.15.150:0/1995083720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0b8c70510 con 0x55c0b8d8a650 2024-05-10T04:03:40.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.968+0000 7f15964f8700 1 --2- 172.21.15.150:0/1995083720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8a650 0x55c0b8d93970 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1580005b40 tx=0x7f158001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e1f8aa7f92e2f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:40.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.968+0000 7f15857fa700 1 -- 172.21.15.150:0/1995083720 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580029040 con 0x55c0b8d8a650 2024-05-10T04:03:40.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.968+0000 7f15857fa700 1 -- 172.21.15.150:0/1995083720 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f158002a040 con 0x55c0b8d8a650 2024-05-10T04:03:40.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.968+0000 7f15857fa700 1 -- 172.21.15.150:0/1995083720 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158001d040 con 0x55c0b8d8a650 2024-05-10T04:03:40.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/1995083720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8a650 msgr2=0x55c0b8d93970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/1995083720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8a650 0x55c0b8d93970 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1580005b40 tx=0x7f158001e000 comp rx=0 tx=0).stop 2024-05-10T04:03:40.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/1995083720 shutdown_connections 2024-05-10T04:03:40.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/1995083720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8be70 0x55c0b8d9a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/1995083720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8a650 0x55c0b8d93970 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/1995083720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8b520 0x55c0b8d99b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/1995083720 >> 172.21.15.150:0/1995083720 conn(0x55c0b8ceca10 msgr2=0x55c0b8d8e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:40.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/1995083720 shutdown_connections 2024-05-10T04:03:40.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/1995083720 wait complete. 2024-05-10T04:03:40.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 -- start start 2024-05-10T04:03:40.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 Processor -- start 2024-05-10T04:03:40.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8a650 0x55c0b8d999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8b520 0x55c0b8d99f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8be70 0x55c0b8d980b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0b8d98720 con 0x55c0b8d8be70 2024-05-10T04:03:40.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0b8d98890 con 0x55c0b8d8a650 2024-05-10T04:03:40.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0b8d98a50 con 0x55c0b8d8b520 2024-05-10T04:03:40.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.969+0000 7f15a8aa0b80 10 client.0 _ll_register_callbacks cb 0x55c0b8c4fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:03:40.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1596cf9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8be70 0x55c0b8d980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1596cf9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8be70 0x55c0b8d980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52584/0 (socket says 172.21.15.150:52584) 2024-05-10T04:03:40.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1596cf9700 1 -- 172.21.15.150:0/3843672521 learned_addr learned my addr 172.21.15.150:0/3843672521 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:40.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1595cf7700 1 --2- 172.21.15.150:0/3843672521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8b520 0x55c0b8d99f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f15964f8700 1 --2- 172.21.15.150:0/3843672521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8a650 0x55c0b8d999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1596cf9700 1 -- 172.21.15.150:0/3843672521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8b520 msgr2=0x55c0b8d99f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:40.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1596cf9700 1 --2- 172.21.15.150:0/3843672521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8b520 0x55c0b8d99f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1596cf9700 1 -- 172.21.15.150:0/3843672521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8a650 msgr2=0x55c0b8d999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:40.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1596cf9700 1 --2- 172.21.15.150:0/3843672521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8a650 0x55c0b8d999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1596cf9700 1 -- 172.21.15.150:0/3843672521 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159010a9b0 con 0x55c0b8d8be70 2024-05-10T04:03:40.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1596cf9700 1 --2- 172.21.15.150:0/3843672521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8be70 0x55c0b8d980b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f15880078c0 tx=0x7f1588007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65406f1dbd7de721 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:40.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1579ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:03:40.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1579ffb700 1 -- 172.21.15.150:0/3843672521 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800e070 con 0x55c0b8d8be70 2024-05-10T04:03:40.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1579ffb700 1 -- 172.21.15.150:0/3843672521 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1588004bb0 con 0x55c0b8d8be70 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/3843672521 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f158c0a22d0 con 0x55c0b8d8be70 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f1579ffb700 1 -- 172.21.15.150:0/3843672521 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1588004d70 con 0x55c0b8d8be70 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f15a8aa0b80 10 client.15264 fetch_fsmap learned FSMap version 24 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.970+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/3843672521 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c0b8b2fc80 con 0x55c0b8d8be70 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.971+0000 7f1579ffb700 1 -- 172.21.15.150:0/3843672521 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f158c0a22d0 con 0x55c0b8d8be70 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.971+0000 7f1579ffb700 1 -- 172.21.15.150:0/3843672521 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f15880187f0 con 0x55c0b8d8be70 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.971+0000 7f15a8aa0b80 10 client.15264 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.971+0000 7f15a8aa0b80 10 client.15264 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.971+0000 7f15a8aa0b80 20 client.15264 populate_metadata read hostname 'smithi150' 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.971+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/3843672521 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55c0b8df8d50 0x55c0b8df72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.971+0000 7f15a8aa0b80 4 client.15264 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.971+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/3843672521 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c0b8ca2e70 con 0x55c0b8df8d50 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.971+0000 7f15964f8700 1 --2- 172.21.15.150:0/3843672521 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55c0b8df8d50 0x55c0b8df72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:40.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f1579ffb700 10 client.15264 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:03:40.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f15964f8700 1 --2- 172.21.15.150:0/3843672521 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55c0b8df8d50 0x55c0b8df72b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47685d470be79b5e server_cookie=411abfb381d6d61 in_seq=0 out_seq=0 2024-05-10T04:03:40.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f1579ffb700 1 -- 172.21.15.150:0/3843672521 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c0b8ca2e70 con 0x55c0b8df8d50 2024-05-10T04:03:40.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f1579ffb700 10 client.15264 handle_command_reply: tid=0 2024-05-10T04:03:40.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f15a8aa0b80 1 client.15264 shutdown 2024-05-10T04:03:40.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/3843672521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8be70 msgr2=0x55c0b8d980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/3843672521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8be70 0x55c0b8d980b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f15880078c0 tx=0x7f1588007bd0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/3843672521 shutdown_connections 2024-05-10T04:03:40.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/3843672521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c0b8d8be70 0x55c0b8d980b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/3843672521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0b8d8a650 0x55c0b8d999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/3843672521 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55c0b8df8d50 0x55c0b8df72b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f15a8aa0b80 1 --2- 172.21.15.150:0/3843672521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0b8d8b520 0x55c0b8d99f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f1579ffb700 0 client.15264 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:03:40.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/3843672521 >> 172.21.15.150:0/3843672521 conn(0x55c0b8ceca10 msgr2=0x55c0b8da84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:40.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/3843672521 shutdown_connections 2024-05-10T04:03:40.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f15a8aa0b80 1 -- 172.21.15.150:0/3843672521 wait complete. 2024-05-10T04:03:40.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.972+0000 7f15a8aa0b80 20 client.15264 trim_cache size 0 max 16384 2024-05-10T04:03:40.983 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:03:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "325b4c82-0e81-11ef-bc98-c7b262605968", 2024-05-10T04:03:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 2, 2024-05-10T04:03:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24535, 2024-05-10T04:03:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:03:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:03:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:03:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 44.801792112000001, 2024-05-10T04:03:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 24, 2024-05-10T04:03:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 87, 2024-05-10T04:03:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 85, 2024-05-10T04:03:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 83.759632793999998 2024-05-10T04:03:40.984 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:03:40.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.973+0000 7f159875c700 1 -- 172.21.15.150:0/1524314434 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f157c0a1260 msgr2=0x7f157c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.973+0000 7f159875c700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f157c0a1260 0x7f157c0a3770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1580003730 tx=0x7f158001a040 comp rx=0 tx=0).stop 2024-05-10T04:03:40.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.973+0000 7f159875c700 1 -- 172.21.15.150:0/1524314434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1590104d10 msgr2=0x7f1590100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:40.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.973+0000 7f159875c700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1590104d10 0x7f1590100620 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c9b0 tx=0x7f158c00cd70 comp rx=0 tx=0).stop 2024-05-10T04:03:40.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.973+0000 7f159875c700 1 -- 172.21.15.150:0/1524314434 shutdown_connections 2024-05-10T04:03:40.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.973+0000 7f159875c700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f157c0a1260 0x7f157c0a3770 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.973+0000 7f159875c700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1590107880 0x7f1590100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.973+0000 7f159875c700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1590069090 0x7f1590101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.973+0000 7f159875c700 1 --2- 172.21.15.150:0/1524314434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1590104d10 0x7f1590100620 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:40.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.973+0000 7f159875c700 1 -- 172.21.15.150:0/1524314434 >> 172.21.15.150:0/1524314434 conn(0x7f15900fc0d0 msgr2=0x7f1590102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:40.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.973+0000 7f159875c700 1 -- 172.21.15.150:0/1524314434 shutdown_connections 2024-05-10T04:03:40.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:40.973+0000 7f159875c700 1 -- 172.21.15.150:0/1524314434 wait complete. 2024-05-10T04:03:40.988 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-10T04:03:41.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.120+0000 7f7706fd0700 1 -- 172.21.15.150:0/2309611502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7700101be0 msgr2=0x7f7700101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.120+0000 7f7706fd0700 1 --2- 172.21.15.150:0/2309611502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7700101be0 0x7f7700101ff0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009990 tx=0x7f76ec009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.120+0000 7f7706fd0700 1 -- 172.21.15.150:0/2309611502 shutdown_connections 2024-05-10T04:03:41.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.120+0000 7f7706fd0700 1 --2- 172.21.15.150:0/2309611502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7700102580 0x7f7700108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.120+0000 7f7706fd0700 1 --2- 172.21.15.150:0/2309611502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7700100ac0 0x7f7700100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.120+0000 7f7706fd0700 1 --2- 172.21.15.150:0/2309611502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7700101be0 0x7f7700101ff0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.120+0000 7f7706fd0700 1 -- 172.21.15.150:0/2309611502 >> 172.21.15.150:0/2309611502 conn(0x7f77000fc0d0 msgr2=0x7f77000fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:41.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.120+0000 7f7706fd0700 1 -- 172.21.15.150:0/2309611502 shutdown_connections 2024-05-10T04:03:41.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.120+0000 7f7706fd0700 1 -- 172.21.15.150:0/2309611502 wait complete. 2024-05-10T04:03:41.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.120+0000 7f7706fd0700 1 Processor -- start 2024-05-10T04:03:41.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7706fd0700 1 -- start start 2024-05-10T04:03:41.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7706fd0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7700100ac0 0x7f7700197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7706fd0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7700101be0 0x7f7700198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7706fd0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7700102580 0x7f770019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7706fd0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770019f960 con 0x7f7700100ac0 2024-05-10T04:03:41.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7706fd0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770019faa0 con 0x7f7700101be0 2024-05-10T04:03:41.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7706fd0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770019fc30 con 0x7f7700102580 2024-05-10T04:03:41.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7704d6c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7700100ac0 0x7f7700197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7704d6c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7700100ac0 0x7f7700197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52602/0 (socket says 172.21.15.150:52602) 2024-05-10T04:03:41.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7704d6c700 1 -- 172.21.15.150:0/1198320372 learned_addr learned my addr 172.21.15.150:0/1198320372 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:41.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f770556d700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7700102580 0x7f770019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f76fffff700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7700101be0 0x7f7700198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7704d6c700 1 -- 172.21.15.150:0/1198320372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7700102580 msgr2=0x7f770019f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7704d6c700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7700102580 0x7f770019f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7704d6c700 1 -- 172.21.15.150:0/1198320372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7700101be0 msgr2=0x7f7700198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7704d6c700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7700101be0 0x7f7700198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7704d6c700 1 -- 172.21.15.150:0/1198320372 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec0095f0 con 0x7f7700100ac0 2024-05-10T04:03:41.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f76fffff700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7700101be0 0x7f7700198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:41.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f7704d6c700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7700100ac0 0x7f7700197ec0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f76f400c9a0 tx=0x7f76f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:41.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.121+0000 7f76fdffb700 1 -- 172.21.15.150:0/1198320372 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4007ac0 con 0x7f7700100ac0 2024-05-10T04:03:41.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.122+0000 7f76fdffb700 1 -- 172.21.15.150:0/1198320372 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76f4007c20 con 0x7f7700100ac0 2024-05-10T04:03:41.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.122+0000 7f7706fd0700 1 -- 172.21.15.150:0/1198320372 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770019ff10 con 0x7f7700100ac0 2024-05-10T04:03:41.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.122+0000 7f76fdffb700 1 -- 172.21.15.150:0/1198320372 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4018660 con 0x7f7700100ac0 2024-05-10T04:03:41.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.122+0000 7f7706fd0700 1 -- 172.21.15.150:0/1198320372 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77001a0390 con 0x7f7700100ac0 2024-05-10T04:03:41.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.125+0000 7f76fdffb700 1 -- 172.21.15.150:0/1198320372 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f76f400f450 con 0x7f7700100ac0 2024-05-10T04:03:41.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.125+0000 7f76fdffb700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f76f00a12e0 0x7f76f00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.125+0000 7f76fdffb700 1 -- 172.21.15.150:0/1198320372 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f76f40cccc0 con 0x7f7700100ac0 2024-05-10T04:03:41.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.125+0000 7f76fffff700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f76f00a12e0 0x7f76f00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.126+0000 7f76fffff700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f76f00a12e0 0x7f76f00a37f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00b2e0 tx=0x7f76ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:41.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.126+0000 7f7717314b80 1 Processor -- start 2024-05-10T04:03:41.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.126+0000 7f7717314b80 1 -- start start 2024-05-10T04:03:41.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.126+0000 7f7717314b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfc650 0x560bbddfcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.126+0000 7f7717314b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 0x560bbddfd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.126+0000 7f7717314b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde04770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.126+0000 7f7717314b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbdd23100 con 0x560bbddfde70 2024-05-10T04:03:41.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.126+0000 7f7717314b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbdd20ac0 con 0x560bbddfd520 2024-05-10T04:03:41.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.126+0000 7f7717314b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbdd1c3b0 con 0x560bbddfc650 2024-05-10T04:03:41.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.126+0000 7f770556d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde04770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.126+0000 7f770556d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde04770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52618/0 (socket says 172.21.15.150:52618) 2024-05-10T04:03:41.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.126+0000 7f770556d700 1 -- 172.21.15.150:0/612360096 learned_addr learned my addr 172.21.15.150:0/612360096 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:41.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.126+0000 7f76fffff700 1 --2- 172.21.15.150:0/612360096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 0x560bbddfd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.127+0000 7f7704d6c700 1 --2- 172.21.15.150:0/612360096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfc650 0x560bbddfcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.127+0000 7f770556d700 1 -- 172.21.15.150:0/612360096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfc650 msgr2=0x560bbddfcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.127+0000 7f770556d700 1 --2- 172.21.15.150:0/612360096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfc650 0x560bbddfcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.127+0000 7f770556d700 1 -- 172.21.15.150:0/612360096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 msgr2=0x560bbddfd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.127+0000 7f770556d700 1 --2- 172.21.15.150:0/612360096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 0x560bbddfd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.127+0000 7f770556d700 1 -- 172.21.15.150:0/612360096 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bbdb9d360 con 0x560bbddfde70 2024-05-10T04:03:41.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.127+0000 7f770556d700 1 --2- 172.21.15.150:0/612360096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde04770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f76f80115d0 tx=0x7f76f8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f4124fc5ef591f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:41.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.127+0000 7f76eaffd700 1 -- 172.21.15.150:0/612360096 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f800e040 con 0x560bbddfde70 2024-05-10T04:03:41.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.127+0000 7f76eaffd700 1 -- 172.21.15.150:0/612360096 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76f800f040 con 0x560bbddfde70 2024-05-10T04:03:41.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.127+0000 7f76eaffd700 1 -- 172.21.15.150:0/612360096 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8004690 con 0x560bbddfde70 2024-05-10T04:03:41.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.128+0000 7f7717314b80 1 -- 172.21.15.150:0/612360096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 msgr2=0x560bbde04770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.128+0000 7f7717314b80 1 --2- 172.21.15.150:0/612360096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde04770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f76f80115d0 tx=0x7f76f8009920 comp rx=0 tx=0).stop 2024-05-10T04:03:41.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.128+0000 7f7717314b80 1 -- 172.21.15.150:0/612360096 shutdown_connections 2024-05-10T04:03:41.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.128+0000 7f7717314b80 1 --2- 172.21.15.150:0/612360096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde04770 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.128+0000 7f7717314b80 1 --2- 172.21.15.150:0/612360096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfc650 0x560bbddfcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.128+0000 7f7717314b80 1 --2- 172.21.15.150:0/612360096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 0x560bbddfd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.128+0000 7f7717314b80 1 -- 172.21.15.150:0/612360096 >> 172.21.15.150:0/612360096 conn(0x560bbdd5ea10 msgr2=0x560bbddf6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:41.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.128+0000 7f7717314b80 1 -- 172.21.15.150:0/612360096 shutdown_connections 2024-05-10T04:03:41.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.128+0000 7f7717314b80 1 -- 172.21.15.150:0/612360096 wait complete. 2024-05-10T04:03:41.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.128+0000 7f7717314b80 1 -- start start 2024-05-10T04:03:41.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.128+0000 7f7717314b80 1 Processor -- start 2024-05-10T04:03:41.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f7717314b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfc650 0x560bbde06640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f7717314b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfd520 0x560bbde06b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f7717314b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde0dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f7717314b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbdcb8510 con 0x560bbddfde70 2024-05-10T04:03:41.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f7717314b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbdcb81b0 con 0x560bbddfc650 2024-05-10T04:03:41.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f7717314b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbdb9f650 con 0x560bbddfd520 2024-05-10T04:03:41.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f7717314b80 10 client.0 _ll_register_callbacks cb 0x560bbdcc1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:03:41.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f770556d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde0dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f770556d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde0dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52632/0 (socket says 172.21.15.150:52632) 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f770556d700 1 -- 172.21.15.150:0/3944956629 learned_addr learned my addr 172.21.15.150:0/3944956629 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f76fffff700 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfd520 0x560bbde06b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f7704d6c700 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfc650 0x560bbde06640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f770556d700 1 -- 172.21.15.150:0/3944956629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfd520 msgr2=0x560bbde06b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f770556d700 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfd520 0x560bbde06b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f770556d700 1 -- 172.21.15.150:0/3944956629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfc650 msgr2=0x560bbde06640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f770556d700 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfc650 0x560bbde06640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f770556d700 1 -- 172.21.15.150:0/3944956629 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f80112b0 con 0x560bbddfde70 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f7704d6c700 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfc650 0x560bbde06640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f770556d700 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde0dae0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f76f8011f20 tx=0x7f76f8011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67d7b9f768c2994 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.129+0000 7f76dffff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f76dffff700 1 -- 172.21.15.150:0/3944956629 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f800e030 con 0x560bbddfde70 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f76dffff700 1 -- 172.21.15.150:0/3944956629 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f76f8004da0 con 0x560bbddfde70 2024-05-10T04:03:41.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f7717314b80 1 -- 172.21.15.150:0/3944956629 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560bbdc75220 con 0x560bbddfde70 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f76dffff700 1 -- 172.21.15.150:0/3944956629 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76f8003b00 con 0x560bbddfde70 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f7717314b80 10 client.15276 fetch_fsmap learned FSMap version 24 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f7717314b80 1 -- 172.21.15.150:0/3944956629 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560bbdb49b10 con 0x560bbddfde70 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f76dffff700 1 -- 172.21.15.150:0/3944956629 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560bbdc75220 con 0x560bbddfde70 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f76dffff700 1 -- 172.21.15.150:0/3944956629 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f76f8024020 con 0x560bbddfde70 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f7717314b80 10 client.15276 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f7717314b80 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f7717314b80 20 client.15276 populate_metadata read hostname 'smithi150' 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f7717314b80 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560bbde11bf0 0x560bbde16920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f7717314b80 4 client.15276 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f7717314b80 1 -- 172.21.15.150:0/3944956629 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560bbddcb730 con 0x560bbde11bf0 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.130+0000 7f7704d6c700 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560bbde11bf0 0x560bbde16920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.131+0000 7f76dffff700 10 client.15276 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:03:41.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.131+0000 7f7704d6c700 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560bbde11bf0 0x560bbde16920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b600faac1c914199 server_cookie=52fb88b4b3ab3d64 in_seq=0 out_seq=0 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f76dffff700 1 -- 172.21.15.150:0/3944956629 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560bbddcb730 con 0x560bbde11bf0 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f76dffff700 10 client.15276 handle_command_reply: tid=0 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f7717314b80 1 client.15276 shutdown 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f7717314b80 1 -- 172.21.15.150:0/3944956629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 msgr2=0x560bbde0dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f7717314b80 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde0dae0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f76f8011f20 tx=0x7f76f8011f50 comp rx=0 tx=0).stop 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f7717314b80 1 -- 172.21.15.150:0/3944956629 shutdown_connections 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f7717314b80 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde0dae0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f7717314b80 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560bbde11bf0 0x560bbde16920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f7717314b80 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfc650 0x560bbde06640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f7717314b80 1 --2- 172.21.15.150:0/3944956629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfd520 0x560bbde06b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f76dffff700 0 client.15276 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f7717314b80 1 -- 172.21.15.150:0/3944956629 >> 172.21.15.150:0/3944956629 conn(0x560bbdd5ea10 msgr2=0x560bbddf6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f7717314b80 1 -- 172.21.15.150:0/3944956629 shutdown_connections 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f7717314b80 1 -- 172.21.15.150:0/3944956629 wait complete. 2024-05-10T04:03:41.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.132+0000 7f7717314b80 20 client.15276 trim_cache size 0 max 16384 2024-05-10T04:03:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7717314b80 1 Processor -- start 2024-05-10T04:03:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7717314b80 1 -- start start 2024-05-10T04:03:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7717314b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfc650 0x560bbde0db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7717314b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 0x560bbde0e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7717314b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfde70 0x560bbddf7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7717314b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbdd15150 con 0x560bbddfc650 2024-05-10T04:03:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7717314b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbdd19010 con 0x560bbddfd520 2024-05-10T04:03:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7717314b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbddf3ce0 con 0x560bbddfde70 2024-05-10T04:03:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7704d6c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfc650 0x560bbde0db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7704d6c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfc650 0x560bbde0db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52636/0 (socket says 172.21.15.150:52636) 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7704d6c700 1 -- 172.21.15.150:0/836055375 learned_addr learned my addr 172.21.15.150:0/836055375 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f770556d700 1 --2- 172.21.15.150:0/836055375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfde70 0x560bbddf7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7704d6c700 1 -- 172.21.15.150:0/836055375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfde70 msgr2=0x560bbddf7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f76fffff700 1 --2- 172.21.15.150:0/836055375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 0x560bbde0e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7704d6c700 1 --2- 172.21.15.150:0/836055375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfde70 0x560bbddf7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7704d6c700 1 -- 172.21.15.150:0/836055375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 msgr2=0x560bbde0e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7704d6c700 1 --2- 172.21.15.150:0/836055375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 0x560bbde0e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7704d6c700 1 -- 172.21.15.150:0/836055375 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bbdce2510 con 0x560bbddfc650 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.150+0000 7f7704d6c700 1 --2- 172.21.15.150:0/836055375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfc650 0x560bbde0db00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f76f400a6b0 tx=0x7f76f400fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c73efd5ed7bf19 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.151+0000 7f76eb7fe700 1 -- 172.21.15.150:0/836055375 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40b6070 con 0x560bbddfc650 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.151+0000 7f76eb7fe700 1 -- 172.21.15.150:0/836055375 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76f409dad0 con 0x560bbddfc650 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.151+0000 7f76eb7fe700 1 -- 172.21.15.150:0/836055375 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4098140 con 0x560bbddfc650 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.151+0000 7f7717314b80 1 -- 172.21.15.150:0/836055375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfc650 msgr2=0x560bbde0db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.151+0000 7f7717314b80 1 --2- 172.21.15.150:0/836055375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfc650 0x560bbde0db00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f76f400a6b0 tx=0x7f76f400fbc0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.151+0000 7f7717314b80 1 -- 172.21.15.150:0/836055375 shutdown_connections 2024-05-10T04:03:41.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.151+0000 7f7717314b80 1 --2- 172.21.15.150:0/836055375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfde70 0x560bbddf7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.151+0000 7f7717314b80 1 --2- 172.21.15.150:0/836055375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfc650 0x560bbde0db00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.151+0000 7f7717314b80 1 --2- 172.21.15.150:0/836055375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 0x560bbde0e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.151+0000 7f7717314b80 1 -- 172.21.15.150:0/836055375 >> 172.21.15.150:0/836055375 conn(0x560bbdd5ea10 msgr2=0x560bbddf6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:41.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.151+0000 7f7717314b80 1 -- 172.21.15.150:0/836055375 shutdown_connections 2024-05-10T04:03:41.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.151+0000 7f7717314b80 1 -- 172.21.15.150:0/836055375 wait complete. 2024-05-10T04:03:41.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.152+0000 7f7717314b80 1 -- start start 2024-05-10T04:03:41.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.152+0000 7f7717314b80 1 Processor -- start 2024-05-10T04:03:41.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.152+0000 7f7717314b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfc650 0x560bbde64850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.152+0000 7f7717314b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 0x560bbde64d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.152+0000 7f7717314b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde6b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.152+0000 7f7717314b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbde6be20 con 0x560bbddfde70 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.152+0000 7f7717314b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbde6bf60 con 0x560bbddfd520 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.152+0000 7f7717314b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbde6c0f0 con 0x560bbddfc650 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.152+0000 7f7717314b80 10 client.0 _ll_register_callbacks cb 0x560bbdcc1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.152+0000 7f770556d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde6b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.152+0000 7f770556d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde6b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52642/0 (socket says 172.21.15.150:52642) 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.152+0000 7f770556d700 1 -- 172.21.15.150:0/379879430 learned_addr learned my addr 172.21.15.150:0/379879430 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.152+0000 7f7704d6c700 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfc650 0x560bbde64850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f76fffff700 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 0x560bbde64d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f770556d700 1 -- 172.21.15.150:0/379879430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfc650 msgr2=0x560bbde64850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f770556d700 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfc650 0x560bbde64850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f770556d700 1 -- 172.21.15.150:0/379879430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 msgr2=0x560bbde64d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f770556d700 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 0x560bbde64d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f770556d700 1 -- 172.21.15.150:0/379879430 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77001a0390 con 0x560bbddfde70 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f76fffff700 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 0x560bbde64d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:41.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f770556d700 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde6b8e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f76f80095a0 tx=0x7f76f8007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9529c1207be7d971 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f76dffff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f76dffff700 1 -- 172.21.15.150:0/379879430 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f800e070 con 0x560bbddfde70 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f76dffff700 1 -- 172.21.15.150:0/379879430 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f76f8004fb0 con 0x560bbddfde70 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f7717314b80 1 -- 172.21.15.150:0/379879430 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76f800f040 con 0x560bbddfde70 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f76dffff700 1 -- 172.21.15.150:0/379879430 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76f801d810 con 0x560bbddfde70 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f7717314b80 10 client.15282 fetch_fsmap learned FSMap version 24 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f7717314b80 1 -- 172.21.15.150:0/379879430 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560bbdba1c80 con 0x560bbddfde70 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.153+0000 7f76dffff700 1 -- 172.21.15.150:0/379879430 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76f800f040 con 0x560bbddfde70 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.154+0000 7f76dffff700 1 -- 172.21.15.150:0/379879430 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f76f8024070 con 0x560bbddfde70 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.154+0000 7f7717314b80 10 client.15282 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.154+0000 7f7717314b80 10 client.15282 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.154+0000 7f7717314b80 20 client.15282 populate_metadata read hostname 'smithi150' 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.154+0000 7f7717314b80 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560bbde6ed90 0x560bbde6d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.154+0000 7f7717314b80 4 client.15282 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.154+0000 7f7717314b80 1 -- 172.21.15.150:0/379879430 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560bbdd14e70 con 0x560bbde6ed90 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.154+0000 7f7704d6c700 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560bbde6ed90 0x560bbde6d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.154+0000 7f76dffff700 10 client.15282 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.154+0000 7f7704d6c700 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560bbde6ed90 0x560bbde6d350 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8bdcb0ab5c352f15 server_cookie=403abae59e9ebf52 in_seq=0 out_seq=0 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.154+0000 7f76dffff700 1 -- 172.21.15.150:0/379879430 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x560bbdd14e70 con 0x560bbde6ed90 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.154+0000 7f76dffff700 10 client.15282 handle_command_reply: tid=0 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.154+0000 7f7717314b80 1 client.15282 shutdown 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7717314b80 1 -- 172.21.15.150:0/379879430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 msgr2=0x560bbde6b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7717314b80 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde6b8e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f76f80095a0 tx=0x7f76f8007eb0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7717314b80 1 -- 172.21.15.150:0/379879430 shutdown_connections 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7717314b80 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bbddfde70 0x560bbde6b8e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7717314b80 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bbddfc650 0x560bbde64850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7717314b80 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560bbde6ed90 0x560bbde6d350 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7717314b80 1 --2- 172.21.15.150:0/379879430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bbddfd520 0x560bbde64d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f76dffff700 0 client.15282 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:03:41.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7717314b80 1 -- 172.21.15.150:0/379879430 >> 172.21.15.150:0/379879430 conn(0x560bbdd5ea10 msgr2=0x560bbde04890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:41.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7717314b80 1 -- 172.21.15.150:0/379879430 shutdown_connections 2024-05-10T04:03:41.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7717314b80 1 -- 172.21.15.150:0/379879430 wait complete. 2024-05-10T04:03:41.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7717314b80 20 client.15282 trim_cache size 0 max 16384 2024-05-10T04:03:41.164 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:03:41.164 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "325b4c82-0e81-11ef-bc98-c7b262605968", 2024-05-10T04:03:41.164 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 1, 2024-05-10T04:03:41.164 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24556, 2024-05-10T04:03:41.164 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:03:41.164 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:03:41.164 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:03:41.164 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 45.991393709, 2024-05-10T04:03:41.165 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 24, 2024-05-10T04:03:41.165 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 87, 2024-05-10T04:03:41.165 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 85, 2024-05-10T04:03:41.165 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 80.335918726000003 2024-05-10T04:03:41.165 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:03:41.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7706fd0700 1 -- 172.21.15.150:0/1198320372 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f76f00a12e0 msgr2=0x7f76f00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7706fd0700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f76f00a12e0 0x7f76f00a37f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00b2e0 tx=0x7f76ec005fd0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7706fd0700 1 -- 172.21.15.150:0/1198320372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7700100ac0 msgr2=0x7f7700197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7706fd0700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7700100ac0 0x7f7700197ec0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f76f400c9a0 tx=0x7f76f400ccb0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7706fd0700 1 -- 172.21.15.150:0/1198320372 shutdown_connections 2024-05-10T04:03:41.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7706fd0700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f76f00a12e0 0x7f76f00a37f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7706fd0700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7700102580 0x7f770019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7706fd0700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7700100ac0 0x7f7700197ec0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7706fd0700 1 --2- 172.21.15.150:0/1198320372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7700101be0 0x7f7700198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.155+0000 7f7706fd0700 1 -- 172.21.15.150:0/1198320372 >> 172.21.15.150:0/1198320372 conn(0x7f77000fc0d0 msgr2=0x7f77000fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:41.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.156+0000 7f7706fd0700 1 -- 172.21.15.150:0/1198320372 shutdown_connections 2024-05-10T04:03:41.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.156+0000 7f7706fd0700 1 -- 172.21.15.150:0/1198320372 wait complete. 2024-05-10T04:03:41.171 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-10T04:03:41.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.302+0000 7fdde18ad700 1 -- 172.21.15.150:0/3731246789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc100ac0 msgr2=0x7fdddc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.302+0000 7fdde18ad700 1 --2- 172.21.15.150:0/3731246789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc100ac0 0x7fdddc100ed0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009990 tx=0x7fddd0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.302+0000 7fdde18ad700 1 -- 172.21.15.150:0/3731246789 shutdown_connections 2024-05-10T04:03:41.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.302+0000 7fdde18ad700 1 --2- 172.21.15.150:0/3731246789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdddc102580 0x7fdddc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.302+0000 7fdde18ad700 1 --2- 172.21.15.150:0/3731246789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc100ac0 0x7fdddc100ed0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.302+0000 7fdde18ad700 1 --2- 172.21.15.150:0/3731246789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdddc101be0 0x7fdddc101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.302+0000 7fdde18ad700 1 -- 172.21.15.150:0/3731246789 >> 172.21.15.150:0/3731246789 conn(0x7fdddc0fc0d0 msgr2=0x7fdddc0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:41.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.302+0000 7fdde18ad700 1 -- 172.21.15.150:0/3731246789 shutdown_connections 2024-05-10T04:03:41.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.302+0000 7fdde18ad700 1 -- 172.21.15.150:0/3731246789 wait complete. 2024-05-10T04:03:41.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.303+0000 7fdde18ad700 1 Processor -- start 2024-05-10T04:03:41.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.303+0000 7fdde18ad700 1 -- start start 2024-05-10T04:03:41.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.303+0000 7fdde18ad700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc100ac0 0x7fdddc197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.303+0000 7fdde18ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdddc101be0 0x7fdddc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.303+0000 7fdde18ad700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdddc102580 0x7fdddc19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.303+0000 7fdde18ad700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc19f9a0 con 0x7fdddc100ac0 2024-05-10T04:03:41.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.303+0000 7fdde18ad700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc19fb10 con 0x7fdddc102580 2024-05-10T04:03:41.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.303+0000 7fdde18ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc19fcd0 con 0x7fdddc101be0 2024-05-10T04:03:41.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.303+0000 7fdddaffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc100ac0 0x7fdddc197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.303+0000 7fdddaffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc100ac0 0x7fdddc197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52672/0 (socket says 172.21.15.150:52672) 2024-05-10T04:03:41.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.303+0000 7fdddaffd700 1 -- 172.21.15.150:0/4133287965 learned_addr learned my addr 172.21.15.150:0/4133287965 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.303+0000 7fdddb7fe700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdddc102580 0x7fdddc19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.303+0000 7fddda7fc700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdddc101be0 0x7fdddc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.304+0000 7fdddaffd700 1 -- 172.21.15.150:0/4133287965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdddc101be0 msgr2=0x7fdddc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.304+0000 7fdddaffd700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdddc101be0 0x7fdddc198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.304+0000 7fdddaffd700 1 -- 172.21.15.150:0/4133287965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdddc102580 msgr2=0x7fdddc19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.304+0000 7fdddaffd700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdddc102580 0x7fdddc19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.304+0000 7fdddaffd700 1 -- 172.21.15.150:0/4133287965 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd00095f0 con 0x7fdddc100ac0 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.304+0000 7fddda7fc700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdddc101be0 0x7fdddc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.304+0000 7fdddaffd700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc100ac0 0x7fdddc197ec0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fddd00098e0 tx=0x7fddd0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.304+0000 7fdde08ab700 1 -- 172.21.15.150:0/4133287965 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd001d070 con 0x7fdddc100ac0 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.304+0000 7fdde08ab700 1 -- 172.21.15.150:0/4133287965 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddd0021410 con 0x7fdddc100ac0 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.304+0000 7fdde18ad700 1 -- 172.21.15.150:0/4133287965 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc19ff50 con 0x7fdddc100ac0 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.304+0000 7fdde08ab700 1 -- 172.21.15.150:0/4133287965 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0017650 con 0x7fdddc100ac0 2024-05-10T04:03:41.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.304+0000 7fdde18ad700 1 -- 172.21.15.150:0/4133287965 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc1a02c0 con 0x7fdddc100ac0 2024-05-10T04:03:41.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.306+0000 7fdde08ab700 1 -- 172.21.15.150:0/4133287965 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fddd0021580 con 0x7fdddc100ac0 2024-05-10T04:03:41.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.307+0000 7fdde08ab700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fddc40a1260 0x7fddc40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.307+0000 7fdde08ab700 1 -- 172.21.15.150:0/4133287965 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fddd00ce310 con 0x7fdddc100ac0 2024-05-10T04:03:41.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.307+0000 7fddda7fc700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fddc40a1260 0x7fddc40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.307+0000 7fddda7fc700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fddc40a1260 0x7fddc40a3770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdddc198f60 tx=0x7fddcc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:41.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.308+0000 7fddf1bf1b80 1 Processor -- start 2024-05-10T04:03:41.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.308+0000 7fddf1bf1b80 1 -- start start 2024-05-10T04:03:41.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.308+0000 7fddf1bf1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 0x563c06269aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.308+0000 7fddf1bf1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c0626a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.308+0000 7fddf1bf1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626ae70 0x563c06271770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.308+0000 7fddf1bf1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c06190100 con 0x563c0626a520 2024-05-10T04:03:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.308+0000 7fddf1bf1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c0618dac0 con 0x563c06269650 2024-05-10T04:03:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.308+0000 7fddf1bf1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c061893b0 con 0x563c0626ae70 2024-05-10T04:03:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.308+0000 7fddda7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c0626a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.308+0000 7fddda7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c0626a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52686/0 (socket says 172.21.15.150:52686) 2024-05-10T04:03:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.308+0000 7fddda7fc700 1 -- 172.21.15.150:0/38121748 learned_addr learned my addr 172.21.15.150:0/38121748 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.308+0000 7fdddaffd700 1 --2- 172.21.15.150:0/38121748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 0x563c06269aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.309+0000 7fdddb7fe700 1 --2- 172.21.15.150:0/38121748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626ae70 0x563c06271770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.309+0000 7fddda7fc700 1 -- 172.21.15.150:0/38121748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626ae70 msgr2=0x563c06271770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.309+0000 7fddda7fc700 1 --2- 172.21.15.150:0/38121748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626ae70 0x563c06271770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.309+0000 7fddda7fc700 1 -- 172.21.15.150:0/38121748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 msgr2=0x563c06269aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.309+0000 7fddda7fc700 1 --2- 172.21.15.150:0/38121748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 0x563c06269aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.309+0000 7fddda7fc700 1 -- 172.21.15.150:0/38121748 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c0600a360 con 0x563c0626a520 2024-05-10T04:03:41.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.309+0000 7fdddaffd700 1 --2- 172.21.15.150:0/38121748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 0x563c06269aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:03:41.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.309+0000 7fdddb7fe700 1 --2- 172.21.15.150:0/38121748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626ae70 0x563c06271770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:41.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.309+0000 7fddda7fc700 1 --2- 172.21.15.150:0/38121748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c0626a930 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fddcc011ed0 tx=0x7fddcc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29f51e428f922e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:41.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.309+0000 7fddc1ffb700 1 -- 172.21.15.150:0/38121748 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc012dc0 con 0x563c0626a520 2024-05-10T04:03:41.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.309+0000 7fddc1ffb700 1 -- 172.21.15.150:0/38121748 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddcc016070 con 0x563c0626a520 2024-05-10T04:03:41.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.309+0000 7fddc1ffb700 1 -- 172.21.15.150:0/38121748 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc01d610 con 0x563c0626a520 2024-05-10T04:03:41.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.310+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/38121748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 msgr2=0x563c0626a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.310+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/38121748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c0626a930 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fddcc011ed0 tx=0x7fddcc014000 comp rx=0 tx=0).stop 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.310+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/38121748 shutdown_connections 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.310+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/38121748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626ae70 0x563c06271770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.310+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/38121748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 0x563c06269aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.310+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/38121748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c0626a930 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.310+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/38121748 >> 172.21.15.150:0/38121748 conn(0x563c061cba10 msgr2=0x563c06263f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.310+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/38121748 shutdown_connections 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.310+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/38121748 wait complete. 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.310+0000 7fddf1bf1b80 1 -- start start 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddf1bf1b80 1 Processor -- start 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddf1bf1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c06269650 0x563c06273610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddf1bf1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c06273b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddf1bf1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c0626ae70 0x563c0627aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddf1bf1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c06125510 con 0x563c0626a520 2024-05-10T04:03:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddf1bf1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c061251b0 con 0x563c0626ae70 2024-05-10T04:03:41.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddf1bf1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c0600c650 con 0x563c06269650 2024-05-10T04:03:41.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddf1bf1b80 10 client.0 _ll_register_callbacks cb 0x563c0612eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:03:41.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddda7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c06273b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddda7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c06273b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52690/0 (socket says 172.21.15.150:52690) 2024-05-10T04:03:41.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddda7fc700 1 -- 172.21.15.150:0/3376655513 learned_addr learned my addr 172.21.15.150:0/3376655513 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:41.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fdddb7fe700 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c0626ae70 0x563c0627aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fdddaffd700 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c06269650 0x563c06273610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddda7fc700 1 -- 172.21.15.150:0/3376655513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c06269650 msgr2=0x563c06273610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddda7fc700 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c06269650 0x563c06273610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddda7fc700 1 -- 172.21.15.150:0/3376655513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c0626ae70 msgr2=0x563c0627aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddda7fc700 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c0626ae70 0x563c0627aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddda7fc700 1 -- 172.21.15.150:0/3376655513 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc1a02c0 con 0x563c0626a520 2024-05-10T04:03:41.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fdddaffd700 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c06269650 0x563c06273610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:41.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddda7fc700 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c06273b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fddcc01f680 tx=0x7fddcc01f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d1ad7edc658fac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:41.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.311+0000 7fddbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:03:41.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddbaffd700 1 -- 172.21.15.150:0/3376655513 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc02d070 con 0x563c0626a520 2024-05-10T04:03:41.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddbaffd700 1 -- 172.21.15.150:0/3376655513 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fddcc018070 con 0x563c0626a520 2024-05-10T04:03:41.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/3376655513 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c060e2220 con 0x563c0626a520 2024-05-10T04:03:41.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddbaffd700 1 -- 172.21.15.150:0/3376655513 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddcc009a00 con 0x563c0626a520 2024-05-10T04:03:41.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddf1bf1b80 10 client.15294 fetch_fsmap learned FSMap version 24 2024-05-10T04:03:41.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/3376655513 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fddcc011c60 con 0x563c0626a520 2024-05-10T04:03:41.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddbaffd700 1 -- 172.21.15.150:0/3376655513 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c060e2220 con 0x563c0626a520 2024-05-10T04:03:41.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddbaffd700 1 -- 172.21.15.150:0/3376655513 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fddcc004bf0 con 0x563c0626a520 2024-05-10T04:03:41.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddf1bf1b80 10 client.15294 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T04:03:41.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddf1bf1b80 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T04:03:41.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddf1bf1b80 20 client.15294 populate_metadata read hostname 'smithi150' 2024-05-10T04:03:41.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563c0627eb60 0x563c06283870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddf1bf1b80 4 client.15294 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:03:41.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.312+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/3376655513 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c06238730 con 0x563c0627eb60 2024-05-10T04:03:41.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.313+0000 7fdddaffd700 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563c0627eb60 0x563c06283870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.313+0000 7fdddaffd700 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563c0627eb60 0x563c06283870 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa92cf39ee7d50ec server_cookie=89f1e092a5d83ae3 in_seq=0 out_seq=0 2024-05-10T04:03:41.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.313+0000 7fddbaffd700 10 client.15294 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:03:41.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddbaffd700 1 -- 172.21.15.150:0/3376655513 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563c06238730 con 0x563c0627eb60 2024-05-10T04:03:41.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddbaffd700 10 client.15294 handle_command_reply: tid=0 2024-05-10T04:03:41.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddf1bf1b80 1 client.15294 shutdown 2024-05-10T04:03:41.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/3376655513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 msgr2=0x563c06273b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c06273b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fddcc01f680 tx=0x7fddcc01f6b0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/3376655513 shutdown_connections 2024-05-10T04:03:41.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563c0627eb60 0x563c06283870 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c0626ae70 0x563c0627aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c06269650 0x563c06273610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/3376655513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c06273b50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/3376655513 >> 172.21.15.150:0/3376655513 conn(0x563c061cba10 msgr2=0x563c06263f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:41.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddbaffd700 0 client.15294 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:03:41.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/3376655513 shutdown_connections 2024-05-10T04:03:41.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/3376655513 wait complete. 2024-05-10T04:03:41.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.314+0000 7fddf1bf1b80 20 client.15294 trim_cache size 0 max 16384 2024-05-10T04:03:41.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.332+0000 7fddf1bf1b80 1 Processor -- start 2024-05-10T04:03:41.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.332+0000 7fddf1bf1b80 1 -- start start 2024-05-10T04:03:41.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.332+0000 7fddf1bf1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 0x563c06272010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.332+0000 7fddf1bf1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626a520 0x563c06278a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.332+0000 7fddf1bf1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626ae70 0x563c06278fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.332+0000 7fddf1bf1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c061285a0 con 0x563c0626ae70 2024-05-10T04:03:41.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.332+0000 7fddf1bf1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c0625e770 con 0x563c06269650 2024-05-10T04:03:41.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.332+0000 7fddf1bf1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c0626ec90 con 0x563c0626a520 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.332+0000 7fdddb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626ae70 0x563c06278fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.332+0000 7fdddb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626ae70 0x563c06278fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52704/0 (socket says 172.21.15.150:52704) 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.332+0000 7fdddb7fe700 1 -- 172.21.15.150:0/3334533052 learned_addr learned my addr 172.21.15.150:0/3334533052 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.332+0000 7fdddaffd700 1 --2- 172.21.15.150:0/3334533052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 0x563c06272010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.332+0000 7fddda7fc700 1 --2- 172.21.15.150:0/3334533052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626a520 0x563c06278a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.333+0000 7fdddb7fe700 1 -- 172.21.15.150:0/3334533052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626a520 msgr2=0x563c06278a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.333+0000 7fdddb7fe700 1 --2- 172.21.15.150:0/3334533052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626a520 0x563c06278a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.333+0000 7fdddb7fe700 1 -- 172.21.15.150:0/3334533052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 msgr2=0x563c06272010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.333+0000 7fdddb7fe700 1 --2- 172.21.15.150:0/3334533052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 0x563c06272010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.333+0000 7fdddb7fe700 1 -- 172.21.15.150:0/3334533052 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c05fb6b10 con 0x563c0626ae70 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.333+0000 7fdddb7fe700 1 --2- 172.21.15.150:0/3334533052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626ae70 0x563c06278fa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009f50 tx=0x7fddd4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1650ed27e0c4f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.333+0000 7fddc27fc700 1 -- 172.21.15.150:0/3334533052 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4017410 con 0x563c0626ae70 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.333+0000 7fddc27fc700 1 -- 172.21.15.150:0/3334533052 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddd400f040 con 0x563c0626ae70 2024-05-10T04:03:41.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.333+0000 7fddc27fc700 1 -- 172.21.15.150:0/3334533052 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd401f8c0 con 0x563c0626ae70 2024-05-10T04:03:41.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.334+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/3334533052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626ae70 msgr2=0x563c06278fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.334+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/3334533052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626ae70 0x563c06278fa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009f50 tx=0x7fddd4009f80 comp rx=0 tx=0).stop 2024-05-10T04:03:41.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.334+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/3334533052 shutdown_connections 2024-05-10T04:03:41.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.334+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/3334533052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626ae70 0x563c06278fa0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.334+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/3334533052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 0x563c06272010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.334+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/3334533052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626a520 0x563c06278a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.334+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/3334533052 >> 172.21.15.150:0/3334533052 conn(0x563c061cba10 msgr2=0x563c06271900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:41.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.334+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/3334533052 shutdown_connections 2024-05-10T04:03:41.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.334+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/3334533052 wait complete. 2024-05-10T04:03:41.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.334+0000 7fddf1bf1b80 1 -- start start 2024-05-10T04:03:41.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.334+0000 7fddf1bf1b80 1 Processor -- start 2024-05-10T04:03:41.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddf1bf1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 0x563c06278db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddf1bf1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c06277420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddf1bf1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626ae70 0x563c06277960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddf1bf1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c06277f40 con 0x563c0626a520 2024-05-10T04:03:41.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddf1bf1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c062780b0 con 0x563c06269650 2024-05-10T04:03:41.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddf1bf1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c06278270 con 0x563c0626ae70 2024-05-10T04:03:41.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddf1bf1b80 10 client.0 _ll_register_callbacks cb 0x563c0612eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:03:41.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddda7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c06277420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddda7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c06277420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52720/0 (socket says 172.21.15.150:52720) 2024-05-10T04:03:41.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddda7fc700 1 -- 172.21.15.150:0/223394568 learned_addr learned my addr 172.21.15.150:0/223394568 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:41.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fdddaffd700 1 --2- 172.21.15.150:0/223394568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 0x563c06278db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fdddb7fe700 1 --2- 172.21.15.150:0/223394568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626ae70 0x563c06277960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddda7fc700 1 -- 172.21.15.150:0/223394568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626ae70 msgr2=0x563c06277960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddda7fc700 1 --2- 172.21.15.150:0/223394568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626ae70 0x563c06277960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddda7fc700 1 -- 172.21.15.150:0/223394568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 msgr2=0x563c06278db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:41.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddda7fc700 1 --2- 172.21.15.150:0/223394568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 0x563c06278db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddda7fc700 1 -- 172.21.15.150:0/223394568 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd4009c30 con 0x563c0626a520 2024-05-10T04:03:41.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddda7fc700 1 --2- 172.21.15.150:0/223394568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c06277420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fddcc005010 tx=0x7fddcc009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89e6262ea23e42e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:41.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:03:41.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddbaffd700 1 -- 172.21.15.150:0/223394568 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc02d040 con 0x563c0626a520 2024-05-10T04:03:41.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddbaffd700 1 -- 172.21.15.150:0/223394568 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fddcc018070 con 0x563c0626a520 2024-05-10T04:03:41.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/223394568 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddcc016070 con 0x563c0626a520 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddbaffd700 1 -- 172.21.15.150:0/223394568 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddcc0275e0 con 0x563c0626a520 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddf1bf1b80 10 client.15300 fetch_fsmap learned FSMap version 24 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.335+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/223394568 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563c0614f510 con 0x563c0626a520 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.336+0000 7fddbaffd700 1 -- 172.21.15.150:0/223394568 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddcc016070 con 0x563c0626a520 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.336+0000 7fddbaffd700 1 -- 172.21.15.150:0/223394568 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fddcc005250 con 0x563c0626a520 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.336+0000 7fddf1bf1b80 10 client.15300 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.336+0000 7fddf1bf1b80 10 client.15300 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.336+0000 7fddf1bf1b80 20 client.15300 populate_metadata read hostname 'smithi150' 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.336+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/223394568 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563c062d80d0 0x563c062d6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.336+0000 7fddf1bf1b80 4 client.15300 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.336+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/223394568 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563c06181e70 con 0x563c062d80d0 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.336+0000 7fdddaffd700 1 --2- 172.21.15.150:0/223394568 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563c062d80d0 0x563c062d6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.336+0000 7fddbaffd700 10 client.15300 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:03:41.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.336+0000 7fdddaffd700 1 --2- 172.21.15.150:0/223394568 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563c062d80d0 0x563c062d6630 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=583154263c5367b1 server_cookie=646181aa698b2859 in_seq=0 out_seq=0 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddbaffd700 1 -- 172.21.15.150:0/223394568 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563c06181e70 con 0x563c062d80d0 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddbaffd700 10 client.15300 handle_command_reply: tid=0 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddf1bf1b80 1 client.15300 shutdown 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/223394568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 msgr2=0x563c06277420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/223394568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c06277420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fddcc005010 tx=0x7fddcc009b00 comp rx=0 tx=0).stop 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/223394568 shutdown_connections 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/223394568 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563c062d80d0 0x563c062d6630 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/223394568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c0626ae70 0x563c06277960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/223394568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c06269650 0x563c06278db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddf1bf1b80 1 --2- 172.21.15.150:0/223394568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c0626a520 0x563c06277420 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddbaffd700 0 client.15300 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/223394568 >> 172.21.15.150:0/223394568 conn(0x563c061cba10 msgr2=0x563c06287890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/223394568 shutdown_connections 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddf1bf1b80 1 -- 172.21.15.150:0/223394568 wait complete. 2024-05-10T04:03:41.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.337+0000 7fddf1bf1b80 20 client.15300 trim_cache size 0 max 16384 2024-05-10T04:03:41.346 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:03:41.346 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "325b4c82-0e81-11ef-bc98-c7b262605968", 2024-05-10T04:03:41.346 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 0, 2024-05-10T04:03:41.346 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24557, 2024-05-10T04:03:41.346 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:03:41.346 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:03:41.346 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:03:41.346 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 69.729822490000004, 2024-05-10T04:03:41.346 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 24, 2024-05-10T04:03:41.346 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 87, 2024-05-10T04:03:41.346 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 85, 2024-05-10T04:03:41.347 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 75.254090656000002 2024-05-10T04:03:41.347 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:03:41.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.338+0000 7fdde18ad700 1 -- 172.21.15.150:0/4133287965 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fddc40a1260 msgr2=0x7fddc40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.338+0000 7fdde18ad700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fddc40a1260 0x7fddc40a3770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdddc198f60 tx=0x7fddcc009260 comp rx=0 tx=0).stop 2024-05-10T04:03:41.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.338+0000 7fdde18ad700 1 -- 172.21.15.150:0/4133287965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc100ac0 msgr2=0x7fdddc197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:41.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.338+0000 7fdde18ad700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc100ac0 0x7fdddc197ec0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fddd00098e0 tx=0x7fddd0009580 comp rx=0 tx=0).stop 2024-05-10T04:03:41.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.338+0000 7fdde18ad700 1 -- 172.21.15.150:0/4133287965 shutdown_connections 2024-05-10T04:03:41.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.338+0000 7fdde18ad700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fddc40a1260 0x7fddc40a3770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.338+0000 7fdde18ad700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdddc102580 0x7fdddc19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.338+0000 7fdde18ad700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdddc100ac0 0x7fdddc197ec0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.338+0000 7fdde18ad700 1 --2- 172.21.15.150:0/4133287965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdddc101be0 0x7fdddc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:41.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.338+0000 7fdde18ad700 1 -- 172.21.15.150:0/4133287965 >> 172.21.15.150:0/4133287965 conn(0x7fdddc0fc0d0 msgr2=0x7fdddc0fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:41.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.338+0000 7fdde18ad700 1 -- 172.21.15.150:0/4133287965 shutdown_connections 2024-05-10T04:03:41.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:41.338+0000 7fdde18ad700 1 -- 172.21.15.150:0/4133287965 wait complete. 2024-05-10T04:03:41.353 INFO:tasks.cephadm:Setup complete, yielding 2024-05-10T04:03:41.353 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-10T04:03:41.363 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:03:41.364 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-10T04:03:41.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:41 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2657839981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:41.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:41 smithi150 ceph-mon[42477]: pgmap v263: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 20 B/s, 0 keys/s, 1 objects/s recovering 2024-05-10T04:03:41.586 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:41.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:41 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2657839981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:41.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:41 smithi181 ceph-mon[46602]: pgmap v263: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 20 B/s, 0 keys/s, 1 objects/s recovering 2024-05-10T04:03:41.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:41 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2657839981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:41.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:41 smithi161 ceph-mon[46918]: pgmap v263: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 20 B/s, 0 keys/s, 1 objects/s recovering 2024-05-10T04:03:42.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.531+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2716377573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e240fffd0 msgr2=0x7f0e241003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:42.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.531+0000 7f0e2bd13700 1 --2- 172.21.15.150:0/2716377573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e240fffd0 0x7f0e241003e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009990 tx=0x7f0e20009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:42.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.531+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2716377573 shutdown_connections 2024-05-10T04:03:42.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.531+0000 7f0e2bd13700 1 --2- 172.21.15.150:0/2716377573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e24107ab0 0x7f0e24109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:42.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.531+0000 7f0e2bd13700 1 --2- 172.21.15.150:0/2716377573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e240fffd0 0x7f0e241003e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:42.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.531+0000 7f0e2bd13700 1 --2- 172.21.15.150:0/2716377573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e24105150 0x7f0e24107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:42.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.531+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2716377573 >> 172.21.15.150:0/2716377573 conn(0x7f0e240fb950 msgr2=0x7f0e240fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:42.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.531+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2716377573 shutdown_connections 2024-05-10T04:03:42.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.531+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2716377573 wait complete. 2024-05-10T04:03:42.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e2bd13700 1 Processor -- start 2024-05-10T04:03:42.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e2bd13700 1 -- start start 2024-05-10T04:03:42.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e2bd13700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e240fffd0 0x7f0e24102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e2bd13700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e24105150 0x7f0e24100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e2bd13700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e24107ab0 0x7f0e241015d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e2bd13700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24101c70 con 0x7f0e240fffd0 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e2bd13700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24101de0 con 0x7f0e24105150 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e2bd13700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e241a20d0 con 0x7f0e24107ab0 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e29aaf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e240fffd0 0x7f0e24102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e29aaf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e240fffd0 0x7f0e24102aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52744/0 (socket says 172.21.15.150:52744) 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e29aaf700 1 -- 172.21.15.150:0/2063281117 learned_addr learned my addr 172.21.15.150:0/2063281117 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e292ae700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e24105150 0x7f0e24100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e2a2b0700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e24107ab0 0x7f0e241015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e29aaf700 1 -- 172.21.15.150:0/2063281117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e24107ab0 msgr2=0x7f0e241015d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e29aaf700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e24107ab0 0x7f0e241015d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e29aaf700 1 -- 172.21.15.150:0/2063281117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e24105150 msgr2=0x7f0e24100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e29aaf700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e24105150 0x7f0e24100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:42.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e29aaf700 1 -- 172.21.15.150:0/2063281117 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e200095f0 con 0x7f0e240fffd0 2024-05-10T04:03:42.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e292ae700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e24105150 0x7f0e24100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:42.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e2a2b0700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e24107ab0 0x7f0e241015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:42.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.532+0000 7f0e29aaf700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e240fffd0 0x7f0e24102aa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0e20006010 tx=0x7f0e2000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:42.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.533+0000 7f0e1affd700 1 -- 172.21.15.150:0/2063281117 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2001d070 con 0x7f0e240fffd0 2024-05-10T04:03:42.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.533+0000 7f0e1affd700 1 -- 172.21.15.150:0/2063281117 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e2000fc30 con 0x7f0e240fffd0 2024-05-10T04:03:42.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.533+0000 7f0e1affd700 1 -- 172.21.15.150:0/2063281117 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20017650 con 0x7f0e240fffd0 2024-05-10T04:03:42.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.533+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2063281117 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e241a2350 con 0x7f0e240fffd0 2024-05-10T04:03:42.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.533+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2063281117 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e241a26b0 con 0x7f0e240fffd0 2024-05-10T04:03:42.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.534+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2063281117 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e240fd500 con 0x7f0e240fffd0 2024-05-10T04:03:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.535+0000 7f0e1affd700 1 -- 172.21.15.150:0/2063281117 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0e200050c0 con 0x7f0e240fffd0 2024-05-10T04:03:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.536+0000 7f0e1affd700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0e100a11b0 0x7f0e100a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.536+0000 7f0e292ae700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0e100a11b0 0x7f0e100a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.536+0000 7f0e1affd700 1 -- 172.21.15.150:0/2063281117 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0e200ce680 con 0x7f0e240fffd0 2024-05-10T04:03:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.537+0000 7f0e292ae700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0e100a11b0 0x7f0e100a36c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0e14005950 tx=0x7f0e14005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:42.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.540+0000 7f0e1affd700 1 -- 172.21.15.150:0/2063281117 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0e2009b4c0 con 0x7f0e240fffd0 2024-05-10T04:03:42.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.749+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2063281117 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e2410d6d0 con 0x7f0e100a11b0 2024-05-10T04:03:42.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.750+0000 7f0e1affd700 1 -- 172.21.15.150:0/2063281117 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f0e2410d6d0 con 0x7f0e100a11b0 2024-05-10T04:03:42.752 INFO:teuthology.orchestra.run.smithi150.stdout:Backend: cephadm 2024-05-10T04:03:42.752 INFO:teuthology.orchestra.run.smithi150.stdout:Available: Yes 2024-05-10T04:03:42.752 INFO:teuthology.orchestra.run.smithi150.stdout:Paused: No 2024-05-10T04:03:42.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.752+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2063281117 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0e100a11b0 msgr2=0x7f0e100a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:42.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.752+0000 7f0e2bd13700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0e100a11b0 0x7f0e100a36c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0e14005950 tx=0x7f0e14005f50 comp rx=0 tx=0).stop 2024-05-10T04:03:42.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.752+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2063281117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e240fffd0 msgr2=0x7f0e24102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:42.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.752+0000 7f0e2bd13700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e240fffd0 0x7f0e24102aa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0e20006010 tx=0x7f0e2000f740 comp rx=0 tx=0).stop 2024-05-10T04:03:42.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.752+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2063281117 shutdown_connections 2024-05-10T04:03:42.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.752+0000 7f0e2bd13700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0e100a11b0 0x7f0e100a36c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:42.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.752+0000 7f0e2bd13700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e24107ab0 0x7f0e241015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:42.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.752+0000 7f0e2bd13700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e240fffd0 0x7f0e24102aa0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:42.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.752+0000 7f0e2bd13700 1 --2- 172.21.15.150:0/2063281117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e24105150 0x7f0e24100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:42.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.752+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2063281117 >> 172.21.15.150:0/2063281117 conn(0x7f0e240fb950 msgr2=0x7f0e24106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:42.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.752+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2063281117 shutdown_connections 2024-05-10T04:03:42.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:42.752+0000 7f0e2bd13700 1 -- 172.21.15.150:0/2063281117 wait complete. 2024-05-10T04:03:43.150 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-10T04:03:43.377 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:44.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.339+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1196040747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47e4101ce0 msgr2=0x7f47e41020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:44.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.339+0000 7f47ea7cc700 1 --2- 172.21.15.150:0/1196040747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47e4101ce0 0x7f47e41020f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009990 tx=0x7f47d8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:44.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.339+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1196040747 shutdown_connections 2024-05-10T04:03:44.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.339+0000 7f47ea7cc700 1 --2- 172.21.15.150:0/1196040747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47e4102680 0x7f47e4108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:44.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.339+0000 7f47ea7cc700 1 --2- 172.21.15.150:0/1196040747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e4100bb0 0x7f47e4100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:44.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.339+0000 7f47ea7cc700 1 --2- 172.21.15.150:0/1196040747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47e4101ce0 0x7f47e41020f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:44.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.339+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1196040747 >> 172.21.15.150:0/1196040747 conn(0x7f47e40fc1d0 msgr2=0x7f47e40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:44.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.339+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1196040747 shutdown_connections 2024-05-10T04:03:44.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.339+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1196040747 wait complete. 2024-05-10T04:03:44.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.340+0000 7f47ea7cc700 1 Processor -- start 2024-05-10T04:03:44.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.340+0000 7f47ea7cc700 1 -- start start 2024-05-10T04:03:44.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.340+0000 7f47ea7cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47e4100bb0 0x7f47e41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:44.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47ea7cc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e4101ce0 0x7f47e4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:44.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47ea7cc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47e4102680 0x7f47e4199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:44.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47ea7cc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4199ad0 con 0x7f47e4102680 2024-05-10T04:03:44.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47ea7cc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4199c10 con 0x7f47e4101ce0 2024-05-10T04:03:44.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47ea7cc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4199dd0 con 0x7f47e4100bb0 2024-05-10T04:03:44.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e8d69700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47e4102680 0x7f47e4199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:44.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e8d69700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47e4102680 0x7f47e4199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52748/0 (socket says 172.21.15.150:52748) 2024-05-10T04:03:44.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e8d69700 1 -- 172.21.15.150:0/1331587321 learned_addr learned my addr 172.21.15.150:0/1331587321 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:44.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e37fe700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e4101ce0 0x7f47e4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:44.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e3fff700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47e4100bb0 0x7f47e41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:44.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e8d69700 1 -- 172.21.15.150:0/1331587321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47e4100bb0 msgr2=0x7f47e41981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:44.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e8d69700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47e4100bb0 0x7f47e41981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:44.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e8d69700 1 -- 172.21.15.150:0/1331587321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e4101ce0 msgr2=0x7f47e4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:44.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e8d69700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e4101ce0 0x7f47e4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:44.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e8d69700 1 -- 172.21.15.150:0/1331587321 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d80095f0 con 0x7f47e4102680 2024-05-10T04:03:44.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e8d69700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47e4102680 0x7f47e4199470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f47dc007ee0 tx=0x7f47dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:44.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e17fa700 1 -- 172.21.15.150:0/1331587321 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc017070 con 0x7f47e4102680 2024-05-10T04:03:44.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1331587321 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e419a0b0 con 0x7f47e4102680 2024-05-10T04:03:44.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e17fa700 1 -- 172.21.15.150:0/1331587321 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47dc00f040 con 0x7f47e4102680 2024-05-10T04:03:44.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47e17fa700 1 -- 172.21.15.150:0/1331587321 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc005740 con 0x7f47e4102680 2024-05-10T04:03:44.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.341+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1331587321 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e41a0890 con 0x7f47e4102680 2024-05-10T04:03:44.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.342+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1331587321 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e40ffeb0 con 0x7f47e4102680 2024-05-10T04:03:44.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.344+0000 7f47e17fa700 1 -- 172.21.15.150:0/1331587321 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f47dc00dc60 con 0x7f47e4102680 2024-05-10T04:03:44.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.344+0000 7f47e17fa700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f47cc0a11b0 0x7f47cc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:44.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.344+0000 7f47e3fff700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f47cc0a11b0 0x7f47cc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:44.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.344+0000 7f47e17fa700 1 -- 172.21.15.150:0/1331587321 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f47dc0ccc40 con 0x7f47e4102680 2024-05-10T04:03:44.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.345+0000 7f47e3fff700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f47cc0a11b0 0x7f47cc0a36c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f47d4005fd0 tx=0x7f47d4005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:44.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.348+0000 7f47e17fa700 1 -- 172.21.15.150:0/1331587321 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f47dc095f60 con 0x7f47e4102680 2024-05-10T04:03:44.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:44 smithi150 ceph-mon[42477]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:44.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:44 smithi150 ceph-mon[42477]: pgmap v264: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 19 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T04:03:44.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:44 smithi181 ceph-mon[46602]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:44.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:44 smithi181 ceph-mon[46602]: pgmap v264: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 19 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T04:03:44.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:44 smithi161 ceph-mon[46918]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:44.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:44 smithi161 ceph-mon[46918]: pgmap v264: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 19 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T04:03:44.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.550+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1331587321 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f47e40fe310 con 0x7f47cc0a11b0 2024-05-10T04:03:44.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.563+0000 7f47e17fa700 1 -- 172.21.15.150:0/1331587321 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f47e40fe310 con 0x7f47cc0a11b0 2024-05-10T04:03:44.566 INFO:teuthology.orchestra.run.smithi150.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-10T04:03:44.566 INFO:teuthology.orchestra.run.smithi150.stdout:mds.a smithi150 running (80s) 13s ago 80s 85.0M - 18.2.2-1330-g50174015 0e8066d08b37 bff7fcf467e7 2024-05-10T04:03:44.566 INFO:teuthology.orchestra.run.smithi150.stdout:mds.b smithi161 running (78s) 14s ago 78s 100M - 18.2.2-1330-g50174015 0e8066d08b37 d5fddd4ab17d 2024-05-10T04:03:44.566 INFO:teuthology.orchestra.run.smithi150.stdout:mds.c smithi181 running (87s) 14s ago 87s 86.1M - 18.2.2-1330-g50174015 0e8066d08b37 80bce2168786 2024-05-10T04:03:44.566 INFO:teuthology.orchestra.run.smithi150.stdout:mds.d smithi150 running (92s) 13s ago 92s 82.5M - 18.2.2-1330-g50174015 0e8066d08b37 4bc2c32833e0 2024-05-10T04:03:44.566 INFO:teuthology.orchestra.run.smithi150.stdout:mds.e smithi161 running (91s) 14s ago 90s 84.2M - 18.2.2-1330-g50174015 0e8066d08b37 53cb2b8009a2 2024-05-10T04:03:44.566 INFO:teuthology.orchestra.run.smithi150.stdout:mds.f smithi181 running (89s) 14s ago 89s 88.9M - 18.2.2-1330-g50174015 0e8066d08b37 07ad2d26b3d6 2024-05-10T04:03:44.566 INFO:teuthology.orchestra.run.smithi150.stdout:mds.g smithi150 running (82s) 13s ago 82s 85.2M - 18.2.2-1330-g50174015 0e8066d08b37 9d9b74151083 2024-05-10T04:03:44.566 INFO:teuthology.orchestra.run.smithi150.stdout:mds.h smithi161 running (85s) 14s ago 85s 82.7M - 18.2.2-1330-g50174015 0e8066d08b37 2b7f5cfd783d 2024-05-10T04:03:44.566 INFO:teuthology.orchestra.run.smithi150.stdout:mds.i smithi181 running (84s) 14s ago 83s 86.2M - 18.2.2-1330-g50174015 0e8066d08b37 38f035aad37d 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:mds.j smithi150 running (94s) 13s ago 94s 85.0M - 18.2.2-1330-g50174015 0e8066d08b37 5167a669b4ce 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:mds.k smithi161 running (96s) 14s ago 96s 82.3M - 18.2.2-1330-g50174015 0e8066d08b37 ba0cc00b4d61 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:mds.l smithi181 running (98s) 14s ago 97s 85.1M - 18.2.2-1330-g50174015 0e8066d08b37 20b6e8af45d2 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:mgr.x smithi150 *:9283,8765 running (7m) 13s ago 6m 599M - 18.2.2-1330-g50174015 0e8066d08b37 0eaac6f82a17 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:mgr.y smithi161 *:8443,8765 running (5m) 14s ago 5m 527M - 18.2.2-1330-g50174015 0e8066d08b37 668c0c0c30f0 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:mgr.z smithi181 *:8443,8765 running (5m) 14s ago 5m 526M - 18.2.2-1330-g50174015 0e8066d08b37 c64f7be61e40 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:mon.a smithi150 running (7m) 13s ago 7m 159M 2048M 18.2.2-1330-g50174015 0e8066d08b37 5de85c52a53e 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:mon.b smithi161 running (5m) 14s ago 5m 130M 2048M 18.2.2-1330-g50174015 0e8066d08b37 b2f4e05ddd6e 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:mon.c smithi181 running (5m) 14s ago 5m 128M 2048M 18.2.2-1330-g50174015 0e8066d08b37 7cbe92669baf 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0 smithi150 running (5m) 13s ago 5m 136M 4278M 18.2.2-1330-g50174015 0e8066d08b37 8e3afbbde5cd 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1 smithi161 running (4m) 14s ago 4m 215M 4278M 18.2.2-1330-g50174015 0e8066d08b37 33ce65b1d6eb 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2 smithi181 running (4m) 14s ago 4m 209M 4278M 18.2.2-1330-g50174015 0e8066d08b37 94d6b820cb37 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3 smithi150 running (4m) 13s ago 4m 132M 4278M 18.2.2-1330-g50174015 0e8066d08b37 f53c9e5f5455 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4 smithi161 running (3m) 14s ago 3m 129M 4278M 18.2.2-1330-g50174015 0e8066d08b37 34785f3af5e5 2024-05-10T04:03:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5 smithi181 running (3m) 14s ago 3m 207M 4278M 18.2.2-1330-g50174015 0e8066d08b37 d79f4032612e 2024-05-10T04:03:44.568 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6 smithi150 running (3m) 13s ago 3m 198M 4278M 18.2.2-1330-g50174015 0e8066d08b37 34faebd541eb 2024-05-10T04:03:44.568 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7 smithi161 running (3m) 14s ago 3m 201M 4278M 18.2.2-1330-g50174015 0e8066d08b37 24a8166d8b5a 2024-05-10T04:03:44.568 INFO:teuthology.orchestra.run.smithi150.stdout:osd.8 smithi181 running (2m) 14s ago 2m 199M 4278M 18.2.2-1330-g50174015 0e8066d08b37 659dcdfa686f 2024-05-10T04:03:44.568 INFO:teuthology.orchestra.run.smithi150.stdout:osd.9 smithi150 running (2m) 13s ago 2m 196M 4278M 18.2.2-1330-g50174015 0e8066d08b37 b9a3a58e5746 2024-05-10T04:03:44.568 INFO:teuthology.orchestra.run.smithi150.stdout:osd.10 smithi161 running (2m) 14s ago 2m 198M 4278M 18.2.2-1330-g50174015 0e8066d08b37 c6148f255d60 2024-05-10T04:03:44.568 INFO:teuthology.orchestra.run.smithi150.stdout:osd.11 smithi181 running (115s) 14s ago 114s 192M 4278M 18.2.2-1330-g50174015 0e8066d08b37 0c679bc8b67e 2024-05-10T04:03:44.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.565+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1331587321 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f47cc0a11b0 msgr2=0x7f47cc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:44.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.565+0000 7f47ea7cc700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f47cc0a11b0 0x7f47cc0a36c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f47d4005fd0 tx=0x7f47d4005dd0 comp rx=0 tx=0).stop 2024-05-10T04:03:44.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.565+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1331587321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47e4102680 msgr2=0x7f47e4199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:44.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.565+0000 7f47ea7cc700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47e4102680 0x7f47e4199470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f47dc007ee0 tx=0x7f47dc00d3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:44.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.566+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1331587321 shutdown_connections 2024-05-10T04:03:44.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.566+0000 7f47ea7cc700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f47cc0a11b0 0x7f47cc0a36c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:44.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.566+0000 7f47ea7cc700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47e4102680 0x7f47e4199470 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:44.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.566+0000 7f47ea7cc700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47e4100bb0 0x7f47e41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:44.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.566+0000 7f47ea7cc700 1 --2- 172.21.15.150:0/1331587321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e4101ce0 0x7f47e4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:44.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.566+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1331587321 >> 172.21.15.150:0/1331587321 conn(0x7f47e40fc1d0 msgr2=0x7f47e40fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:44.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.566+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1331587321 shutdown_connections 2024-05-10T04:03:44.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:44.566+0000 7f47ea7cc700 1 -- 172.21.15.150:0/1331587321 wait complete. 2024-05-10T04:03:44.950 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-10T04:03:45.174 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:46.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.115+0000 7fd7b28dd700 1 -- 172.21.15.150:0/3033266408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7ac100b80 msgr2=0x7fd7ac100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:46.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.115+0000 7fd7b28dd700 1 --2- 172.21.15.150:0/3033266408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7ac100b80 0x7fd7ac100f90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009990 tx=0x7fd79c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:46.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.115+0000 7fd7b28dd700 1 -- 172.21.15.150:0/3033266408 shutdown_connections 2024-05-10T04:03:46.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.115+0000 7fd7b28dd700 1 --2- 172.21.15.150:0/3033266408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7ac102650 0x7fd7ac108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:46.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.115+0000 7fd7b28dd700 1 --2- 172.21.15.150:0/3033266408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7ac100b80 0x7fd7ac100f90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:46.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.115+0000 7fd7b28dd700 1 --2- 172.21.15.150:0/3033266408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7ac101cb0 0x7fd7ac1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:46.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.115+0000 7fd7b28dd700 1 -- 172.21.15.150:0/3033266408 >> 172.21.15.150:0/3033266408 conn(0x7fd7ac0fc1c0 msgr2=0x7fd7ac0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:46.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.115+0000 7fd7b28dd700 1 -- 172.21.15.150:0/3033266408 shutdown_connections 2024-05-10T04:03:46.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.115+0000 7fd7b28dd700 1 -- 172.21.15.150:0/3033266408 wait complete. 2024-05-10T04:03:46.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.115+0000 7fd7b28dd700 1 Processor -- start 2024-05-10T04:03:46.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7b28dd700 1 -- start start 2024-05-10T04:03:46.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7b28dd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7ac100b80 0x7fd7ac19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:46.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7b28dd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7ac101cb0 0x7fd7ac19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:46.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7b28dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7ac102650 0x7fd7ac19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:46.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7b28dd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac069c70 con 0x7fd7ac101cb0 2024-05-10T04:03:46.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7b28dd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac19fd60 con 0x7fd7ac100b80 2024-05-10T04:03:46.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7b28dd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac19fed0 con 0x7fd7ac102650 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7ab7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7ac101cb0 0x7fd7ac19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7ab7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7ac101cb0 0x7fd7ac19d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52766/0 (socket says 172.21.15.150:52766) 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7ab7fe700 1 -- 172.21.15.150:0/447863236 learned_addr learned my addr 172.21.15.150:0/447863236 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7b0e7a700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7ac102650 0x7fd7ac19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7abfff700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7ac100b80 0x7fd7ac19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7ab7fe700 1 -- 172.21.15.150:0/447863236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7ac102650 msgr2=0x7fd7ac19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7ab7fe700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7ac102650 0x7fd7ac19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7ab7fe700 1 -- 172.21.15.150:0/447863236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7ac100b80 msgr2=0x7fd7ac19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7ab7fe700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7ac100b80 0x7fd7ac19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.116+0000 7fd7ab7fe700 1 -- 172.21.15.150:0/447863236 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c0095f0 con 0x7fd7ac101cb0 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.117+0000 7fd7ab7fe700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7ac101cb0 0x7fd7ac19d2d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000c9b0 tx=0x7fd7a000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.117+0000 7fd7a97fa700 1 -- 172.21.15.150:0/447863236 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0007b00 con 0x7fd7ac101cb0 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.117+0000 7fd7a97fa700 1 -- 172.21.15.150:0/447863236 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7a0007c60 con 0x7fd7ac101cb0 2024-05-10T04:03:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.117+0000 7fd7b28dd700 1 -- 172.21.15.150:0/447863236 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ac1a01b0 con 0x7fd7ac101cb0 2024-05-10T04:03:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.117+0000 7fd7a97fa700 1 -- 172.21.15.150:0/447863236 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0018660 con 0x7fd7ac101cb0 2024-05-10T04:03:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.117+0000 7fd7b28dd700 1 -- 172.21.15.150:0/447863236 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac1a0700 con 0x7fd7ac101cb0 2024-05-10T04:03:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.118+0000 7fd7b28dd700 1 -- 172.21.15.150:0/447863236 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ac107da0 con 0x7fd7ac101cb0 2024-05-10T04:03:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.119+0000 7fd7a97fa700 1 -- 172.21.15.150:0/447863236 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd7a000f450 con 0x7fd7ac101cb0 2024-05-10T04:03:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.119+0000 7fd7a97fa700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd7940a1230 0x7fd7940a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.120+0000 7fd7a97fa700 1 -- 172.21.15.150:0/447863236 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd7a00ccfd0 con 0x7fd7ac101cb0 2024-05-10T04:03:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.120+0000 7fd7abfff700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd7940a1230 0x7fd7940a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.120+0000 7fd7abfff700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd7940a1230 0x7fd7940a3740 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0098e0 tx=0x7fd79c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.123+0000 7fd7a97fa700 1 -- 172.21.15.150:0/447863236 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd7a0096400 con 0x7fd7ac101cb0 2024-05-10T04:03:46.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:46 smithi150 ceph-mon[42477]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:46.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:46 smithi150 ceph-mon[42477]: pgmap v265: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 19 B/s, 0 objects/s recovering 2024-05-10T04:03:46.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.328+0000 7fd7b28dd700 1 -- 172.21.15.150:0/447863236 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7ac10c4e0 con 0x7fd7940a1230 2024-05-10T04:03:46.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.333+0000 7fd7a97fa700 1 -- 172.21.15.150:0/447863236 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7fd7ac10c4e0 con 0x7fd7940a1230 2024-05-10T04:03:46.336 INFO:teuthology.orchestra.run.smithi150.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-10T04:03:46.336 INFO:teuthology.orchestra.run.smithi150.stdout:mds.all 12/12 15s ago 101s smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2024-05-10T04:03:46.336 INFO:teuthology.orchestra.run.smithi150.stdout:mgr 3/3 15s ago 5m smithi150=x;smithi161=y;smithi181=z;count:3 2024-05-10T04:03:46.336 INFO:teuthology.orchestra.run.smithi150.stdout:mon 3/3 15s ago 5m smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c;count:3 2024-05-10T04:03:46.336 INFO:teuthology.orchestra.run.smithi150.stdout:osd 12 15s ago - 2024-05-10T04:03:46.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.336+0000 7fd7b28dd700 1 -- 172.21.15.150:0/447863236 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd7940a1230 msgr2=0x7fd7940a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:46.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.336+0000 7fd7b28dd700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd7940a1230 0x7fd7940a3740 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0098e0 tx=0x7fd79c01a040 comp rx=0 tx=0).stop 2024-05-10T04:03:46.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.336+0000 7fd7b28dd700 1 -- 172.21.15.150:0/447863236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7ac101cb0 msgr2=0x7fd7ac19d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:46.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.336+0000 7fd7b28dd700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7ac101cb0 0x7fd7ac19d2d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000c9b0 tx=0x7fd7a000ccc0 comp rx=0 tx=0).stop 2024-05-10T04:03:46.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.336+0000 7fd7b28dd700 1 -- 172.21.15.150:0/447863236 shutdown_connections 2024-05-10T04:03:46.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.336+0000 7fd7b28dd700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd7940a1230 0x7fd7940a3740 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:46.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.336+0000 7fd7b28dd700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7ac102650 0x7fd7ac19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:46.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.336+0000 7fd7b28dd700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7ac100b80 0x7fd7ac19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:46.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.336+0000 7fd7b28dd700 1 --2- 172.21.15.150:0/447863236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7ac101cb0 0x7fd7ac19d2d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:46.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.336+0000 7fd7b28dd700 1 -- 172.21.15.150:0/447863236 >> 172.21.15.150:0/447863236 conn(0x7fd7ac0fc1c0 msgr2=0x7fd7ac104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:46.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.336+0000 7fd7b28dd700 1 -- 172.21.15.150:0/447863236 shutdown_connections 2024-05-10T04:03:46.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:46.336+0000 7fd7b28dd700 1 -- 172.21.15.150:0/447863236 wait complete. 2024-05-10T04:03:46.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:46 smithi181 ceph-mon[46602]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:46.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:46 smithi181 ceph-mon[46602]: pgmap v265: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 19 B/s, 0 objects/s recovering 2024-05-10T04:03:46.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:46 smithi161 ceph-mon[46918]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:46.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:46 smithi161 ceph-mon[46918]: pgmap v265: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 19 B/s, 0 objects/s recovering 2024-05-10T04:03:46.709 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-10T04:03:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:47.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.882+0000 7fb2da8d8700 1 -- 172.21.15.150:0/3774207786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4101cb0 msgr2=0x7fb2d41020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:47.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.882+0000 7fb2da8d8700 1 --2- 172.21.15.150:0/3774207786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4101cb0 0x7fb2d41020c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009990 tx=0x7fb2c4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:47.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.882+0000 7fb2da8d8700 1 -- 172.21.15.150:0/3774207786 shutdown_connections 2024-05-10T04:03:47.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.882+0000 7fb2da8d8700 1 --2- 172.21.15.150:0/3774207786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d4102650 0x7fb2d4108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:47.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.882+0000 7fb2da8d8700 1 --2- 172.21.15.150:0/3774207786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2d4100b80 0x7fb2d4100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:47.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.882+0000 7fb2da8d8700 1 --2- 172.21.15.150:0/3774207786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4101cb0 0x7fb2d41020c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:47.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.882+0000 7fb2da8d8700 1 -- 172.21.15.150:0/3774207786 >> 172.21.15.150:0/3774207786 conn(0x7fb2d40fc1c0 msgr2=0x7fb2d40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:47.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.882+0000 7fb2da8d8700 1 -- 172.21.15.150:0/3774207786 shutdown_connections 2024-05-10T04:03:47.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.882+0000 7fb2da8d8700 1 -- 172.21.15.150:0/3774207786 wait complete. 2024-05-10T04:03:47.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.883+0000 7fb2da8d8700 1 Processor -- start 2024-05-10T04:03:47.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.883+0000 7fb2da8d8700 1 -- start start 2024-05-10T04:03:47.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.883+0000 7fb2da8d8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d4100b80 0x7fb2d4198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:47.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.883+0000 7fb2da8d8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2d4101cb0 0x7fb2d4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:47.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.883+0000 7fb2da8d8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4102650 0x7fb2d4199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:47.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.883+0000 7fb2da8d8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4199a80 con 0x7fb2d4102650 2024-05-10T04:03:47.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.883+0000 7fb2da8d8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4199bc0 con 0x7fb2d4101cb0 2024-05-10T04:03:47.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.883+0000 7fb2da8d8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4199d80 con 0x7fb2d4100b80 2024-05-10T04:03:47.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d8e75700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4102650 0x7fb2d4199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:47.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d8e75700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4102650 0x7fb2d4199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52772/0 (socket says 172.21.15.150:52772) 2024-05-10T04:03:47.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d8e75700 1 -- 172.21.15.150:0/91147305 learned_addr learned my addr 172.21.15.150:0/91147305 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:47.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d37fe700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2d4101cb0 0x7fb2d4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:47.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d3fff700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d4100b80 0x7fb2d4198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:47.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d8e75700 1 -- 172.21.15.150:0/91147305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d4100b80 msgr2=0x7fb2d4198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:47.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d8e75700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d4100b80 0x7fb2d4198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:47.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d8e75700 1 -- 172.21.15.150:0/91147305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2d4101cb0 msgr2=0x7fb2d4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:47.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d8e75700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2d4101cb0 0x7fb2d4198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:47.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d8e75700 1 -- 172.21.15.150:0/91147305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c40095f0 con 0x7fb2d4102650 2024-05-10T04:03:47.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d8e75700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4102650 0x7fb2d4199420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc007ee0 tx=0x7fb2cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:47.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d17fa700 1 -- 172.21.15.150:0/91147305 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc00ddd0 con 0x7fb2d4102650 2024-05-10T04:03:47.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d17fa700 1 -- 172.21.15.150:0/91147305 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2cc00f040 con 0x7fb2d4102650 2024-05-10T04:03:47.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2d17fa700 1 -- 172.21.15.150:0/91147305 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc0048e0 con 0x7fb2d4102650 2024-05-10T04:03:47.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2da8d8700 1 -- 172.21.15.150:0/91147305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d419a060 con 0x7fb2d4102650 2024-05-10T04:03:47.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.884+0000 7fb2da8d8700 1 -- 172.21.15.150:0/91147305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d41a0840 con 0x7fb2d4102650 2024-05-10T04:03:47.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.885+0000 7fb2da8d8700 1 -- 172.21.15.150:0/91147305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d40ffe80 con 0x7fb2d4102650 2024-05-10T04:03:47.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.887+0000 7fb2d17fa700 1 -- 172.21.15.150:0/91147305 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb2cc003a00 con 0x7fb2d4102650 2024-05-10T04:03:47.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.887+0000 7fb2d17fa700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb2c00a11b0 0x7fb2c00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:47.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.888+0000 7fb2d3fff700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb2c00a11b0 0x7fb2c00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:47.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.888+0000 7fb2d17fa700 1 -- 172.21.15.150:0/91147305 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb2cc0ccee0 con 0x7fb2d4102650 2024-05-10T04:03:47.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.888+0000 7fb2d3fff700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb2c00a11b0 0x7fb2c00a36c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc005950 tx=0x7fb2bc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:47.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:47.891+0000 7fb2d17fa700 1 -- 172.21.15.150:0/91147305 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb2cc096200 con 0x7fb2d4102650 2024-05-10T04:03:48.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.091+0000 7fb2da8d8700 1 -- 172.21.15.150:0/91147305 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2d40fe370 con 0x7fb2c00a11b0 2024-05-10T04:03:48.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.092+0000 7fb2d17fa700 1 -- 172.21.15.150:0/91147305 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fb2d40fe370 con 0x7fb2c00a11b0 2024-05-10T04:03:48.095 INFO:teuthology.orchestra.run.smithi150.stdout:HOST ADDR LABELS STATUS 2024-05-10T04:03:48.095 INFO:teuthology.orchestra.run.smithi150.stdout:smithi150 172.21.15.150 2024-05-10T04:03:48.095 INFO:teuthology.orchestra.run.smithi150.stdout:smithi161 172.21.15.161 2024-05-10T04:03:48.095 INFO:teuthology.orchestra.run.smithi150.stdout:smithi181 172.21.15.181 2024-05-10T04:03:48.095 INFO:teuthology.orchestra.run.smithi150.stdout:3 hosts in cluster 2024-05-10T04:03:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.094+0000 7fb2da8d8700 1 -- 172.21.15.150:0/91147305 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb2c00a11b0 msgr2=0x7fb2c00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.094+0000 7fb2da8d8700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb2c00a11b0 0x7fb2c00a36c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc005950 tx=0x7fb2bc0058e0 comp rx=0 tx=0).stop 2024-05-10T04:03:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.094+0000 7fb2da8d8700 1 -- 172.21.15.150:0/91147305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4102650 msgr2=0x7fb2d4199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.094+0000 7fb2da8d8700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4102650 0x7fb2d4199420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc007ee0 tx=0x7fb2cc00d3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.095+0000 7fb2da8d8700 1 -- 172.21.15.150:0/91147305 shutdown_connections 2024-05-10T04:03:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.095+0000 7fb2da8d8700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb2c00a11b0 0x7fb2c00a36c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.095+0000 7fb2da8d8700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4102650 0x7fb2d4199420 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.095+0000 7fb2da8d8700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d4100b80 0x7fb2d4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.095+0000 7fb2da8d8700 1 --2- 172.21.15.150:0/91147305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2d4101cb0 0x7fb2d4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.095+0000 7fb2da8d8700 1 -- 172.21.15.150:0/91147305 >> 172.21.15.150:0/91147305 conn(0x7fb2d40fc1c0 msgr2=0x7fb2d40fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.095+0000 7fb2da8d8700 1 -- 172.21.15.150:0/91147305 shutdown_connections 2024-05-10T04:03:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:48.095+0000 7fb2da8d8700 1 -- 172.21.15.150:0/91147305 wait complete. 2024-05-10T04:03:48.381 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:48 smithi150 ceph-mon[42477]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:48.382 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:48 smithi150 ceph-mon[42477]: pgmap v266: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:48.483 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-10T04:03:48.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:48 smithi181 ceph-mon[46602]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:48.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:48 smithi181 ceph-mon[46602]: pgmap v266: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:48.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:48 smithi161 ceph-mon[46918]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:48.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:48 smithi161 ceph-mon[46918]: pgmap v266: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:48.705 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:49.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:49 smithi150 ceph-mon[42477]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:49.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:49 smithi181 ceph-mon[46602]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:49.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:49 smithi161 ceph-mon[46918]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:49.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.732+0000 7f664defe700 1 -- 172.21.15.150:0/3699419085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6648100b80 msgr2=0x7f6648100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:49.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.732+0000 7f664defe700 1 --2- 172.21.15.150:0/3699419085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6648100b80 0x7f6648100f90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6638009990 tx=0x7f6638009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:49.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.732+0000 7f664defe700 1 -- 172.21.15.150:0/3699419085 shutdown_connections 2024-05-10T04:03:49.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.732+0000 7f664defe700 1 --2- 172.21.15.150:0/3699419085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6648102650 0x7f6648108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:49.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.732+0000 7f664defe700 1 --2- 172.21.15.150:0/3699419085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6648100b80 0x7f6648100f90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:49.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.732+0000 7f664defe700 1 --2- 172.21.15.150:0/3699419085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6648101cb0 0x7f66481020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:49.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.732+0000 7f664defe700 1 -- 172.21.15.150:0/3699419085 >> 172.21.15.150:0/3699419085 conn(0x7f66480fc1c0 msgr2=0x7f66480fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:49.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.732+0000 7f664defe700 1 -- 172.21.15.150:0/3699419085 shutdown_connections 2024-05-10T04:03:49.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.732+0000 7f664defe700 1 -- 172.21.15.150:0/3699419085 wait complete. 2024-05-10T04:03:49.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.733+0000 7f664defe700 1 Processor -- start 2024-05-10T04:03:49.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.733+0000 7f664defe700 1 -- start start 2024-05-10T04:03:49.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.733+0000 7f664defe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6648100b80 0x7f66481970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:49.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.733+0000 7f664defe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6648101cb0 0x7f6648197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:49.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.733+0000 7f664defe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6648102650 0x7f66481983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:49.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.733+0000 7f664defe700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6648198a30 con 0x7f6648100b80 2024-05-10T04:03:49.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.733+0000 7f664defe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6648198b70 con 0x7f6648101cb0 2024-05-10T04:03:49.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.733+0000 7f664defe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6648198d30 con 0x7f6648102650 2024-05-10T04:03:49.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f66477fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6648100b80 0x7f66481970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:49.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f66477fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6648100b80 0x7f66481970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38470/0 (socket says 172.21.15.150:38470) 2024-05-10T04:03:49.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f66477fe700 1 -- 172.21.15.150:0/1312192906 learned_addr learned my addr 172.21.15.150:0/1312192906 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:49.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f6647fff700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6648102650 0x7f66481983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:49.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f6646ffd700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6648101cb0 0x7f6648197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:49.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f66477fe700 1 -- 172.21.15.150:0/1312192906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6648102650 msgr2=0x7f66481983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:49.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f66477fe700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6648102650 0x7f66481983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:49.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f66477fe700 1 -- 172.21.15.150:0/1312192906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6648101cb0 msgr2=0x7f6648197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:49.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f66477fe700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6648101cb0 0x7f6648197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:49.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f66477fe700 1 -- 172.21.15.150:0/1312192906 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66380095f0 con 0x7f6648100b80 2024-05-10T04:03:49.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f66477fe700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6648100b80 0x7f66481970e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6638005b40 tx=0x7f663800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:49.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f6644ff9700 1 -- 172.21.15.150:0/1312192906 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663801d070 con 0x7f6648100b80 2024-05-10T04:03:49.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f664defe700 1 -- 172.21.15.150:0/1312192906 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6648198fb0 con 0x7f6648100b80 2024-05-10T04:03:49.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f6644ff9700 1 -- 172.21.15.150:0/1312192906 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66380054f0 con 0x7f6648100b80 2024-05-10T04:03:49.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f6644ff9700 1 -- 172.21.15.150:0/1312192906 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638004a10 con 0x7f6648100b80 2024-05-10T04:03:49.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.734+0000 7f664defe700 1 -- 172.21.15.150:0/1312192906 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664819f7f0 con 0x7f6648100b80 2024-05-10T04:03:49.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.735+0000 7f664defe700 1 -- 172.21.15.150:0/1312192906 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66480ffe80 con 0x7f6648100b80 2024-05-10T04:03:49.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.736+0000 7f6644ff9700 1 -- 172.21.15.150:0/1312192906 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6638004b70 con 0x7f6648100b80 2024-05-10T04:03:49.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.737+0000 7f6644ff9700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f66300a11b0 0x7f66300a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:49.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.737+0000 7f6644ff9700 1 -- 172.21.15.150:0/1312192906 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f66380ce010 con 0x7f6648100b80 2024-05-10T04:03:49.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.737+0000 7f6646ffd700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f66300a11b0 0x7f66300a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:49.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.737+0000 7f6646ffd700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f66300a11b0 0x7f66300a36c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f663c005950 tx=0x7f663c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:49.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.741+0000 7f6644ff9700 1 -- 172.21.15.150:0/1312192906 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f66380974e0 con 0x7f6648100b80 2024-05-10T04:03:49.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.942+0000 7f664defe700 1 -- 172.21.15.150:0/1312192906 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6648068d20 con 0x7f66300a11b0 2024-05-10T04:03:49.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.944+0000 7f6644ff9700 1 -- 172.21.15.150:0/1312192906 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f6648068d20 con 0x7f66300a11b0 2024-05-10T04:03:49.946 INFO:teuthology.orchestra.run.smithi150.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-10T04:03:49.946 INFO:teuthology.orchestra.run.smithi150.stdout:smithi150 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GX400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-10T04:03:49.946 INFO:teuthology.orchestra.run.smithi150.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-10T04:03:49.946 INFO:teuthology.orchestra.run.smithi150.stdout:smithi181 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000E400BGN 372G No 115s ago Has a FileSystem, LVM detected 2024-05-10T04:03:49.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.946+0000 7f664defe700 1 -- 172.21.15.150:0/1312192906 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f66300a11b0 msgr2=0x7f66300a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:49.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.946+0000 7f664defe700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f66300a11b0 0x7f66300a36c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f663c005950 tx=0x7f663c0058e0 comp rx=0 tx=0).stop 2024-05-10T04:03:49.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.946+0000 7f664defe700 1 -- 172.21.15.150:0/1312192906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6648100b80 msgr2=0x7f66481970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:49.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.946+0000 7f664defe700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6648100b80 0x7f66481970e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6638005b40 tx=0x7f663800f710 comp rx=0 tx=0).stop 2024-05-10T04:03:49.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.946+0000 7f664defe700 1 -- 172.21.15.150:0/1312192906 shutdown_connections 2024-05-10T04:03:49.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.946+0000 7f664defe700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f66300a11b0 0x7f66300a36c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:49.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.946+0000 7f664defe700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6648102650 0x7f66481983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:49.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.946+0000 7f664defe700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6648100b80 0x7f66481970e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:49.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.946+0000 7f664defe700 1 --2- 172.21.15.150:0/1312192906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6648101cb0 0x7f6648197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:49.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.946+0000 7f664defe700 1 -- 172.21.15.150:0/1312192906 >> 172.21.15.150:0/1312192906 conn(0x7f66480fc1c0 msgr2=0x7f6648104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:49.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.946+0000 7f664defe700 1 -- 172.21.15.150:0/1312192906 shutdown_connections 2024-05-10T04:03:49.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:49.946+0000 7f664defe700 1 -- 172.21.15.150:0/1312192906 wait complete. 2024-05-10T04:03:50.344 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:50 smithi150 ceph-mon[42477]: pgmap v267: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:50.345 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-10T04:03:50.354 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:03:50.354 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-10T04:03:50.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:50 smithi181 ceph-mon[46602]: pgmap v267: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:50.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:50 smithi161 ceph-mon[46918]: pgmap v267: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:50.578 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:51.438 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:51 smithi150 ceph-mon[42477]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:51.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:51 smithi181 ceph-mon[46602]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:51.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:51 smithi161 ceph-mon[46918]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:03:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.614+0000 7f84c9969700 1 -- 172.21.15.150:0/3918129987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84c4101cb0 msgr2=0x7f84c41020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:51.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.614+0000 7f84c9969700 1 --2- 172.21.15.150:0/3918129987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84c4101cb0 0x7f84c41020c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009990 tx=0x7f84b4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:51.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.614+0000 7f84c9969700 1 -- 172.21.15.150:0/3918129987 shutdown_connections 2024-05-10T04:03:51.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.614+0000 7f84c9969700 1 --2- 172.21.15.150:0/3918129987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4102650 0x7f84c4108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:51.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.614+0000 7f84c9969700 1 --2- 172.21.15.150:0/3918129987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c4100b80 0x7f84c4100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:51.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.614+0000 7f84c9969700 1 --2- 172.21.15.150:0/3918129987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84c4101cb0 0x7f84c41020c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:51.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.614+0000 7f84c9969700 1 -- 172.21.15.150:0/3918129987 >> 172.21.15.150:0/3918129987 conn(0x7f84c40fc1c0 msgr2=0x7f84c40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:51.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.614+0000 7f84c9969700 1 -- 172.21.15.150:0/3918129987 shutdown_connections 2024-05-10T04:03:51.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.614+0000 7f84c9969700 1 -- 172.21.15.150:0/3918129987 wait complete. 2024-05-10T04:03:51.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.615+0000 7f84c9969700 1 Processor -- start 2024-05-10T04:03:51.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.615+0000 7f84c9969700 1 -- start start 2024-05-10T04:03:51.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.615+0000 7f84c9969700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4100b80 0x7f84c419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:51.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.615+0000 7f84c9969700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84c4101cb0 0x7f84c419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.615+0000 7f84c9969700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c4102650 0x7f84c419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.615+0000 7f84c9969700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c419fe00 con 0x7f84c4101cb0 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.615+0000 7f84c9969700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c419ff40 con 0x7f84c4102650 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.615+0000 7f84c9969700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c41a0080 con 0x7f84c4100b80 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.615+0000 7f84c27fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84c4101cb0 0x7f84c419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.615+0000 7f84c27fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84c4101cb0 0x7f84c419d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38490/0 (socket says 172.21.15.150:38490) 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.615+0000 7f84c27fc700 1 -- 172.21.15.150:0/2867708307 learned_addr learned my addr 172.21.15.150:0/2867708307 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.615+0000 7f84c2ffd700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4100b80 0x7f84c419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c37fe700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c4102650 0x7f84c419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c27fc700 1 -- 172.21.15.150:0/2867708307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4100b80 msgr2=0x7f84c419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c27fc700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4100b80 0x7f84c419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c27fc700 1 -- 172.21.15.150:0/2867708307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c4102650 msgr2=0x7f84c419f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c27fc700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c4102650 0x7f84c419f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c27fc700 1 -- 172.21.15.150:0/2867708307 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b40095f0 con 0x7f84c4101cb0 2024-05-10T04:03:51.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c2ffd700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4100b80 0x7f84c419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:51.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c37fe700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c4102650 0x7f84c419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:51.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c27fc700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84c4101cb0 0x7f84c419d370 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f84b4005740 tx=0x7f84b4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:51.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c8967700 1 -- 172.21.15.150:0/2867708307 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b401d070 con 0x7f84c4101cb0 2024-05-10T04:03:51.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c8967700 1 -- 172.21.15.150:0/2867708307 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84b400fc00 con 0x7f84c4101cb0 2024-05-10T04:03:51.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c9969700 1 -- 172.21.15.150:0/2867708307 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c41a0280 con 0x7f84c4101cb0 2024-05-10T04:03:51.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c8967700 1 -- 172.21.15.150:0/2867708307 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4017650 con 0x7f84c4101cb0 2024-05-10T04:03:51.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.616+0000 7f84c9969700 1 -- 172.21.15.150:0/2867708307 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c41a0690 con 0x7f84c4101cb0 2024-05-10T04:03:51.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.617+0000 7f84c9969700 1 -- 172.21.15.150:0/2867708307 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c4107da0 con 0x7f84c4101cb0 2024-05-10T04:03:51.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.618+0000 7f84c8967700 1 -- 172.21.15.150:0/2867708307 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f84b4021410 con 0x7f84c4101cb0 2024-05-10T04:03:51.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.619+0000 7f84c8967700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f84ac0a11b0 0x7f84ac0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:51.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.619+0000 7f84c2ffd700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f84ac0a11b0 0x7f84ac0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:51.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.619+0000 7f84c8967700 1 -- 172.21.15.150:0/2867708307 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f84b40ce290 con 0x7f84c4101cb0 2024-05-10T04:03:51.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.620+0000 7f84c2ffd700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f84ac0a11b0 0x7f84ac0a36c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f84c40696a0 tx=0x7f84b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.624+0000 7f84c8967700 1 -- 172.21.15.150:0/2867708307 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f84b40975e0 con 0x7f84c4101cb0 2024-05-10T04:03:51.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.862+0000 7f84c9969700 1 -- 172.21.15.150:0/2867708307 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f84c40ffe80 con 0x7f84c4101cb0 2024-05-10T04:03:51.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.863+0000 7f84c8967700 1 -- 172.21.15.150:0/2867708307 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+2911 (secure 0 0 0) 0x7f84b4026b70 con 0x7f84c4101cb0 2024-05-10T04:03:51.865 INFO:teuthology.orchestra.run.smithi150.stdout:e24 2024-05-10T04:03:51.865 INFO:teuthology.orchestra.run.smithi150.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T04:03:51.865 INFO:teuthology.orchestra.run.smithi150.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T04:03:51.866 INFO:teuthology.orchestra.run.smithi150.stdout:legacy client fscid: 1 2024-05-10T04:03:51.866 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:51.866 INFO:teuthology.orchestra.run.smithi150.stdout:Filesystem 'cephfs' (1) 2024-05-10T04:03:51.866 INFO:teuthology.orchestra.run.smithi150.stdout:fs_name cephfs 2024-05-10T04:03:51.866 INFO:teuthology.orchestra.run.smithi150.stdout:epoch 24 2024-05-10T04:03:51.866 INFO:teuthology.orchestra.run.smithi150.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-10T04:03:51.866 INFO:teuthology.orchestra.run.smithi150.stdout:created 2024-05-10T04:02:31.605102+0000 2024-05-10T04:03:51.866 INFO:teuthology.orchestra.run.smithi150.stdout:modified 2024-05-10T04:02:57.183845+0000 2024-05-10T04:03:51.866 INFO:teuthology.orchestra.run.smithi150.stdout:tableserver 0 2024-05-10T04:03:51.866 INFO:teuthology.orchestra.run.smithi150.stdout:root 0 2024-05-10T04:03:51.866 INFO:teuthology.orchestra.run.smithi150.stdout:session_timeout 300 2024-05-10T04:03:51.867 INFO:teuthology.orchestra.run.smithi150.stdout:session_autoclose 300 2024-05-10T04:03:51.867 INFO:teuthology.orchestra.run.smithi150.stdout:max_file_size 1099511627776 2024-05-10T04:03:51.867 INFO:teuthology.orchestra.run.smithi150.stdout:max_xattr_size 65536 2024-05-10T04:03:51.867 INFO:teuthology.orchestra.run.smithi150.stdout:required_client_features {} 2024-05-10T04:03:51.867 INFO:teuthology.orchestra.run.smithi150.stdout:last_failure 0 2024-05-10T04:03:51.867 INFO:teuthology.orchestra.run.smithi150.stdout:last_failure_osd_epoch 0 2024-05-10T04:03:51.867 INFO:teuthology.orchestra.run.smithi150.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T04:03:51.867 INFO:teuthology.orchestra.run.smithi150.stdout:max_mds 3 2024-05-10T04:03:51.867 INFO:teuthology.orchestra.run.smithi150.stdout:in 0,1,2 2024-05-10T04:03:51.867 INFO:teuthology.orchestra.run.smithi150.stdout:up {0=24557,1=24556,2=24535} 2024-05-10T04:03:51.868 INFO:teuthology.orchestra.run.smithi150.stdout:failed 2024-05-10T04:03:51.868 INFO:teuthology.orchestra.run.smithi150.stdout:damaged 2024-05-10T04:03:51.868 INFO:teuthology.orchestra.run.smithi150.stdout:stopped 2024-05-10T04:03:51.868 INFO:teuthology.orchestra.run.smithi150.stdout:data_pools [3,4] 2024-05-10T04:03:51.868 INFO:teuthology.orchestra.run.smithi150.stdout:metadata_pool 2 2024-05-10T04:03:51.868 INFO:teuthology.orchestra.run.smithi150.stdout:inline_data disabled 2024-05-10T04:03:51.868 INFO:teuthology.orchestra.run.smithi150.stdout:balancer 2024-05-10T04:03:51.868 INFO:teuthology.orchestra.run.smithi150.stdout:bal_rank_mask -1 2024-05-10T04:03:51.868 INFO:teuthology.orchestra.run.smithi150.stdout:standby_count_wanted 0 2024-05-10T04:03:51.868 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b{0:24557} state up:active seq 3 addr [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.f{0:24523} state up:standby-replay seq 1 addr [v2:172.21.15.181:6834/501599730,v1:172.21.15.181:6835/501599730] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.i{1:24556} state up:active seq 10 addr [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.e{1:24503} state up:standby-replay seq 1 addr [v2:172.21.15.161:6834/1403383813,v1:172.21.15.161:6835/1403383813] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c{2:24535} state up:active seq 11 addr [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.l{2:24484} state up:standby-replay seq 1 addr [v2:172.21.15.181:6832/2987418700,v1:172.21.15.181:6833/2987418700] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout:Standby daemons: 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.j{-1:14592} state up:standby seq 1 addr [v2:172.21.15.150:6834/1387516089,v1:172.21.15.150:6835/1387516089] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d{-1:14604} state up:standby seq 1 addr [v2:172.21.15.150:6836/3462983502,v1:172.21.15.150:6837/3462983502] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.h{-1:14646} state up:standby seq 1 addr [v2:172.21.15.161:6836/1619036473,v1:172.21.15.161:6837/1619036473] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.g{-1:14664} state up:standby seq 1 addr [v2:172.21.15.150:6838/2082606207,v1:172.21.15.150:6839/2082606207] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a{-1:14676} state up:standby seq 1 addr [v2:172.21.15.150:6840/591989720,v1:172.21.15.150:6841/591989720] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:03:51.869 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.k{-1:24467} state up:standby seq 1 addr [v2:172.21.15.161:6832/3107886390,v1:172.21.15.161:6833/3107886390] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:03:51.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.865+0000 7f84c9969700 1 -- 172.21.15.150:0/2867708307 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f84ac0a11b0 msgr2=0x7f84ac0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:51.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.865+0000 7f84c9969700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f84ac0a11b0 0x7f84ac0a36c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f84c40696a0 tx=0x7f84b8009260 comp rx=0 tx=0).stop 2024-05-10T04:03:51.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.865+0000 7f84c9969700 1 -- 172.21.15.150:0/2867708307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84c4101cb0 msgr2=0x7f84c419d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:51.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.865+0000 7f84c9969700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84c4101cb0 0x7f84c419d370 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f84b4005740 tx=0x7f84b4009580 comp rx=0 tx=0).stop 2024-05-10T04:03:51.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.865+0000 7f84c9969700 1 -- 172.21.15.150:0/2867708307 shutdown_connections 2024-05-10T04:03:51.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.865+0000 7f84c9969700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f84ac0a11b0 0x7f84ac0a36c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:51.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.865+0000 7f84c9969700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84c4102650 0x7f84c419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:51.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.865+0000 7f84c9969700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4100b80 0x7f84c419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:51.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.865+0000 7f84c9969700 1 --2- 172.21.15.150:0/2867708307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84c4101cb0 0x7f84c419d370 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:51.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.865+0000 7f84c9969700 1 -- 172.21.15.150:0/2867708307 >> 172.21.15.150:0/2867708307 conn(0x7f84c40fc1c0 msgr2=0x7f84c4104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:51.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.865+0000 7f84c9969700 1 -- 172.21.15.150:0/2867708307 shutdown_connections 2024-05-10T04:03:51.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:51.865+0000 7f84c9969700 1 -- 172.21.15.150:0/2867708307 wait complete. 2024-05-10T04:03:51.872 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 24 2024-05-10T04:03:52.275 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-10T04:03:52.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:52 smithi150 ceph-mon[42477]: pgmap v268: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T04:03:52.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:52 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2867708307' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T04:03:52.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:52 smithi181 ceph-mon[46602]: pgmap v268: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T04:03:52.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:52 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2867708307' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T04:03:52.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:52 smithi161 ceph-mon[46918]: pgmap v268: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T04:03:52.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:52 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2867708307' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T04:03:52.501 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:03:53.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.505+0000 7f95910cb700 1 -- 172.21.15.150:0/1343692528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c102650 msgr2=0x7f958c108f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:53.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.505+0000 7f95910cb700 1 --2- 172.21.15.150:0/1343692528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c102650 0x7f958c108f00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f958400b0a0 tx=0x7f958400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:53.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.505+0000 7f95910cb700 1 -- 172.21.15.150:0/1343692528 shutdown_connections 2024-05-10T04:03:53.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.505+0000 7f95910cb700 1 --2- 172.21.15.150:0/1343692528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c102650 0x7f958c108f00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:53.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.505+0000 7f95910cb700 1 --2- 172.21.15.150:0/1343692528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f958c100b80 0x7f958c100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:53.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.505+0000 7f95910cb700 1 --2- 172.21.15.150:0/1343692528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f958c101cb0 0x7f958c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:53.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.505+0000 7f95910cb700 1 -- 172.21.15.150:0/1343692528 >> 172.21.15.150:0/1343692528 conn(0x7f958c0fc1c0 msgr2=0x7f958c0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:53.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.505+0000 7f95910cb700 1 -- 172.21.15.150:0/1343692528 shutdown_connections 2024-05-10T04:03:53.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.505+0000 7f95910cb700 1 -- 172.21.15.150:0/1343692528 wait complete. 2024-05-10T04:03:53.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.506+0000 7f95910cb700 1 Processor -- start 2024-05-10T04:03:53.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f95910cb700 1 -- start start 2024-05-10T04:03:53.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f95910cb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f958c100b80 0x7f958c197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f95910cb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101cb0 0x7f958c1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f95910cb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f958c102650 0x7f958c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f95910cb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c198ac0 con 0x7f958c101cb0 2024-05-10T04:03:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f95910cb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c198c00 con 0x7f958c102650 2024-05-10T04:03:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f95910cb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c198dc0 con 0x7f958c100b80 2024-05-10T04:03:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f958a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101cb0 0x7f958c1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f958a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101cb0 0x7f958c1976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38502/0 (socket says 172.21.15.150:38502) 2024-05-10T04:03:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f958a59c700 1 -- 172.21.15.150:0/1162146436 learned_addr learned my addr 172.21.15.150:0/1162146436 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f958ad9d700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f958c100b80 0x7f958c197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f958b59e700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f958c102650 0x7f958c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f958a59c700 1 -- 172.21.15.150:0/1162146436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f958c100b80 msgr2=0x7f958c197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f958a59c700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f958c100b80 0x7f958c197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f958a59c700 1 -- 172.21.15.150:0/1162146436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f958c102650 msgr2=0x7f958c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:53.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f958a59c700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f958c102650 0x7f958c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:53.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f958a59c700 1 -- 172.21.15.150:0/1162146436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9584009d00 con 0x7f958c101cb0 2024-05-10T04:03:53.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.507+0000 7f958b59e700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f958c102650 0x7f958c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:53.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.508+0000 7f958a59c700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101cb0 0x7f958c1976b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f957400b530 tx=0x7f957400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:53.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.508+0000 7f9583fff700 1 -- 172.21.15.150:0/1162146436 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9574011830 con 0x7f958c101cb0 2024-05-10T04:03:53.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.508+0000 7f95910cb700 1 -- 172.21.15.150:0/1162146436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958c1990a0 con 0x7f958c101cb0 2024-05-10T04:03:53.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.508+0000 7f9583fff700 1 -- 172.21.15.150:0/1162146436 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9574011e50 con 0x7f958c101cb0 2024-05-10T04:03:53.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.508+0000 7f9583fff700 1 -- 172.21.15.150:0/1162146436 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957400f2e0 con 0x7f958c101cb0 2024-05-10T04:03:53.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.508+0000 7f95910cb700 1 -- 172.21.15.150:0/1162146436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958c19f900 con 0x7f958c101cb0 2024-05-10T04:03:53.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.509+0000 7f95910cb700 1 -- 172.21.15.150:0/1162146436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958c107da0 con 0x7f958c101cb0 2024-05-10T04:03:53.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.510+0000 7f9583fff700 1 -- 172.21.15.150:0/1162146436 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9574010380 con 0x7f958c101cb0 2024-05-10T04:03:53.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.511+0000 7f9583fff700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f95780a11b0 0x7f95780a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:53.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.511+0000 7f958ad9d700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f95780a11b0 0x7f95780a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:53.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.511+0000 7f9583fff700 1 -- 172.21.15.150:0/1162146436 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f95740ccd70 con 0x7f958c101cb0 2024-05-10T04:03:53.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.511+0000 7f958ad9d700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f95780a11b0 0x7f95780a36c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f958c101bb0 tx=0x7f957c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:53.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.515+0000 7f9583fff700 1 -- 172.21.15.150:0/1162146436 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f95740961a0 con 0x7f958c101cb0 2024-05-10T04:03:53.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.704+0000 7f95910cb700 1 -- 172.21.15.150:0/1162146436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f958c0ffe80 con 0x7f958c101cb0 2024-05-10T04:03:53.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.705+0000 7f9583fff700 1 -- 172.21.15.150:0/1162146436 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5628 (secure 0 0 0) 0x7f95740997c0 con 0x7f958c101cb0 2024-05-10T04:03:53.707 INFO:teuthology.orchestra.run.smithi150.stdout:epoch 87 2024-05-10T04:03:53.708 INFO:teuthology.orchestra.run.smithi150.stdout:fsid 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T04:03:53.708 INFO:teuthology.orchestra.run.smithi150.stdout:created 2024-05-10T03:56:39.689538+0000 2024-05-10T04:03:53.708 INFO:teuthology.orchestra.run.smithi150.stdout:modified 2024-05-10T04:03:30.185138+0000 2024-05-10T04:03:53.708 INFO:teuthology.orchestra.run.smithi150.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-10T04:03:53.708 INFO:teuthology.orchestra.run.smithi150.stdout:crush_version 27 2024-05-10T04:03:53.708 INFO:teuthology.orchestra.run.smithi150.stdout:full_ratio 0.95 2024-05-10T04:03:53.708 INFO:teuthology.orchestra.run.smithi150.stdout:backfillfull_ratio 0.9 2024-05-10T04:03:53.708 INFO:teuthology.orchestra.run.smithi150.stdout:nearfull_ratio 0.85 2024-05-10T04:03:53.708 INFO:teuthology.orchestra.run.smithi150.stdout:require_min_compat_client luminous 2024-05-10T04:03:53.708 INFO:teuthology.orchestra.run.smithi150.stdout:min_compat_client luminous 2024-05-10T04:03:53.708 INFO:teuthology.orchestra.run.smithi150.stdout:require_osd_release reef 2024-05-10T04:03:53.709 INFO:teuthology.orchestra.run.smithi150.stdout:stretch_mode_enabled false 2024-05-10T04:03:53.709 INFO:teuthology.orchestra.run.smithi150.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-10T04:03:53.709 INFO:teuthology.orchestra.run.smithi150.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-10T04:03:53.709 INFO:teuthology.orchestra.run.smithi150.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-10T04:03:53.709 INFO:teuthology.orchestra.run.smithi150.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-10T04:03:53.709 INFO:teuthology.orchestra.run.smithi150.stdout:max_osd 12 2024-05-10T04:03:53.709 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6802/2214000215,v1:172.21.15.150:6803/2214000215] [v2:172.21.15.150:6804/2214000215,v1:172.21.15.150:6805/2214000215] exists,up e6088d05-bf1a-4fd0-be86-fde95b461d3a 2024-05-10T04:03:53.709 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/1606950735,v1:172.21.15.161:6801/1606950735] [v2:172.21.15.161:6802/1606950735,v1:172.21.15.161:6803/1606950735] exists,up cef6c663-3fc6-4ea4-9dd8-e95845b8bac8 2024-05-10T04:03:53.709 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6800/4035594145,v1:172.21.15.181:6801/4035594145] [v2:172.21.15.181:6802/4035594145,v1:172.21.15.181:6803/4035594145] exists,up 27256076-4b91-40ac-a69b-9e20e0f37b4c 2024-05-10T04:03:53.709 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6810/1770832004,v1:172.21.15.150:6811/1770832004] [v2:172.21.15.150:6812/1770832004,v1:172.21.15.150:6813/1770832004] exists,up 116fe668-1fb0-478b-a7ea-fd1542885ec9 2024-05-10T04:03:53.709 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4 up in weight 1 up_from 32 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/765138503,v1:172.21.15.161:6809/765138503] [v2:172.21.15.161:6810/765138503,v1:172.21.15.161:6811/765138503] exists,up 827ce60d-b43f-45e0-a1e3-d40de3a1b5e3 2024-05-10T04:03:53.709 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5 up in weight 1 up_from 37 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6808/1962584780,v1:172.21.15.181:6809/1962584780] [v2:172.21.15.181:6810/1962584780,v1:172.21.15.181:6811/1962584780] exists,up bb8856d9-daa0-4119-9da6-84cc2c9c5dbc 2024-05-10T04:03:53.710 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6 up in weight 1 up_from 44 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6818/3579837504,v1:172.21.15.150:6819/3579837504] [v2:172.21.15.150:6820/3579837504,v1:172.21.15.150:6821/3579837504] exists,up 8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8 2024-05-10T04:03:53.710 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7 up in weight 1 up_from 49 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/1528703649,v1:172.21.15.161:6817/1528703649] [v2:172.21.15.161:6818/1528703649,v1:172.21.15.161:6819/1528703649] exists,up 4e7b2d25-da3d-4d56-966a-13e6d9c2278e 2024-05-10T04:03:53.710 INFO:teuthology.orchestra.run.smithi150.stdout:osd.8 up in weight 1 up_from 56 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6816/3605634813,v1:172.21.15.181:6817/3605634813] [v2:172.21.15.181:6818/3605634813,v1:172.21.15.181:6819/3605634813] exists,up 259306a4-2c3c-4aff-9f6b-38cf3256b090 2024-05-10T04:03:53.710 INFO:teuthology.orchestra.run.smithi150.stdout:osd.9 up in weight 1 up_from 64 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6826/383677934,v1:172.21.15.150:6827/383677934] [v2:172.21.15.150:6828/383677934,v1:172.21.15.150:6829/383677934] exists,up db540925-be5f-4601-8cb9-392f6b547d68 2024-05-10T04:03:53.710 INFO:teuthology.orchestra.run.smithi150.stdout:osd.10 up in weight 1 up_from 68 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/1965594933,v1:172.21.15.161:6825/1965594933] [v2:172.21.15.161:6826/1965594933,v1:172.21.15.161:6827/1965594933] exists,up c3281319-74d6-4de1-ba77-6671f6d2d5fc 2024-05-10T04:03:53.710 INFO:teuthology.orchestra.run.smithi150.stdout:osd.11 up in weight 1 up_from 76 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6824/1224976921,v1:172.21.15.181:6825/1224976921] [v2:172.21.15.181:6826/1224976921,v1:172.21.15.181:6827/1224976921] exists,up a60f21a5-86c6-43b9-aea1-fa8517d04b1d 2024-05-10T04:03:53.710 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 2.a [2,11] 2024-05-10T04:03:53.710 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 2.10 [2,4] 2024-05-10T04:03:53.710 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 2.1d [8,11] 2024-05-10T04:03:53.710 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 3.1 [4,9] 2024-05-10T04:03:53.710 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 3.1e [3,9] 2024-05-10T04:03:53.710 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 3.3c [4,9] 2024-05-10T04:03:53.711 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 4.2 [9,1] 2024-05-10T04:03:53.711 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 4.c [9,10] 2024-05-10T04:03:53.711 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 4.1b [9,8] 2024-05-10T04:03:53.711 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 4.1d [5,8] 2024-05-10T04:03:53.711 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:0/3457770606 expires 2024-05-11T03:57:29.013448+0000 2024-05-10T04:03:53.711 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:6801/999775795 expires 2024-05-11T03:57:29.013448+0000 2024-05-10T04:03:53.711 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:0/977507934 expires 2024-05-11T03:57:08.747290+0000 2024-05-10T04:03:53.711 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:6800/3392737437 expires 2024-05-11T03:57:08.747290+0000 2024-05-10T04:03:53.711 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:0/3909466387 expires 2024-05-11T03:57:29.013448+0000 2024-05-10T04:03:53.711 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:0/2393499570 expires 2024-05-11T03:57:29.013448+0000 2024-05-10T04:03:53.711 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:6800/999775795 expires 2024-05-11T03:57:29.013448+0000 2024-05-10T04:03:53.711 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:6801/3392737437 expires 2024-05-11T03:57:08.747290+0000 2024-05-10T04:03:53.711 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:0/679926778 expires 2024-05-11T03:57:08.747290+0000 2024-05-10T04:03:53.712 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:0/431676611 expires 2024-05-11T03:57:08.747290+0000 2024-05-10T04:03:53.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.707+0000 7f95910cb700 1 -- 172.21.15.150:0/1162146436 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f95780a11b0 msgr2=0x7f95780a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:53.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.707+0000 7f95910cb700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f95780a11b0 0x7f95780a36c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f958c101bb0 tx=0x7f957c00a380 comp rx=0 tx=0).stop 2024-05-10T04:03:53.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.707+0000 7f95910cb700 1 -- 172.21.15.150:0/1162146436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101cb0 msgr2=0x7f958c1976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:53.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.707+0000 7f95910cb700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101cb0 0x7f958c1976b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f957400b530 tx=0x7f957400b840 comp rx=0 tx=0).stop 2024-05-10T04:03:53.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.707+0000 7f95910cb700 1 -- 172.21.15.150:0/1162146436 shutdown_connections 2024-05-10T04:03:53.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.707+0000 7f95910cb700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f95780a11b0 0x7f95780a36c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:53.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.707+0000 7f95910cb700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f958c102650 0x7f958c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:53.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.707+0000 7f95910cb700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f958c100b80 0x7f958c197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:53.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.707+0000 7f95910cb700 1 --2- 172.21.15.150:0/1162146436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101cb0 0x7f958c1976b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:53.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.707+0000 7f95910cb700 1 -- 172.21.15.150:0/1162146436 >> 172.21.15.150:0/1162146436 conn(0x7f958c0fc1c0 msgr2=0x7f958c104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:53.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.708+0000 7f95910cb700 1 -- 172.21.15.150:0/1162146436 shutdown_connections 2024-05-10T04:03:53.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:53.708+0000 7f95910cb700 1 -- 172.21.15.150:0/1162146436 wait complete. 2024-05-10T04:03:54.065 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-10T04:03:54.076 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:03:54.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.213+0000 7f69e481c700 1 -- 172.21.15.150:0/3234563458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e0102580 msgr2=0x7f69e0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:54.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.213+0000 7f69e481c700 1 --2- 172.21.15.150:0/3234563458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e0102580 0x7f69e0108e30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f69d800b0a0 tx=0x7f69d800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.213+0000 7f69e481c700 1 -- 172.21.15.150:0/3234563458 shutdown_connections 2024-05-10T04:03:54.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.213+0000 7f69e481c700 1 --2- 172.21.15.150:0/3234563458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e0102580 0x7f69e0108e30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.213+0000 7f69e481c700 1 --2- 172.21.15.150:0/3234563458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69e0100ac0 0x7f69e0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.213+0000 7f69e481c700 1 --2- 172.21.15.150:0/3234563458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69e0101be0 0x7f69e0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.213+0000 7f69e481c700 1 -- 172.21.15.150:0/3234563458 >> 172.21.15.150:0/3234563458 conn(0x7f69e00fc0d0 msgr2=0x7f69e00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:54.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.213+0000 7f69e481c700 1 -- 172.21.15.150:0/3234563458 shutdown_connections 2024-05-10T04:03:54.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.213+0000 7f69e481c700 1 -- 172.21.15.150:0/3234563458 wait complete. 2024-05-10T04:03:54.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69e481c700 1 Processor -- start 2024-05-10T04:03:54.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69e481c700 1 -- start start 2024-05-10T04:03:54.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69e481c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69e0100ac0 0x7f69e0068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:54.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69e481c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69e0101be0 0x7f69e019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:54.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69e481c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e0102580 0x7f69e019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69e481c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e019fbb0 con 0x7f69e0102580 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69e481c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e019fd20 con 0x7f69e0101be0 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69e481c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e019fee0 con 0x7f69e0100ac0 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69ded9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e0102580 0x7f69e019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69ded9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e0102580 0x7f69e019f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38516/0 (socket says 172.21.15.150:38516) 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69ded9d700 1 -- 172.21.15.150:0/2762346035 learned_addr learned my addr 172.21.15.150:0/2762346035 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69de59c700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69e0100ac0 0x7f69e0068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69ddd9b700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69e0101be0 0x7f69e019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69ded9d700 1 -- 172.21.15.150:0/2762346035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69e0100ac0 msgr2=0x7f69e0068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69ded9d700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69e0100ac0 0x7f69e0068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69ded9d700 1 -- 172.21.15.150:0/2762346035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69e0101be0 msgr2=0x7f69e019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69ded9d700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69e0101be0 0x7f69e019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69ded9d700 1 -- 172.21.15.150:0/2762346035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d8009d00 con 0x7f69e0102580 2024-05-10T04:03:54.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69ded9d700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e0102580 0x7f69e019f460 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f69d8006070 tx=0x7f69d80094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:54.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69d77fe700 1 -- 172.21.15.150:0/2762346035 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d800e040 con 0x7f69e0102580 2024-05-10T04:03:54.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69d77fe700 1 -- 172.21.15.150:0/2762346035 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d8004e20 con 0x7f69e0102580 2024-05-10T04:03:54.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69e481c700 1 -- 172.21.15.150:0/2762346035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e01a0100 con 0x7f69e0102580 2024-05-10T04:03:54.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.214+0000 7f69d77fe700 1 -- 172.21.15.150:0/2762346035 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d801d480 con 0x7f69e0102580 2024-05-10T04:03:54.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.215+0000 7f69e481c700 1 -- 172.21.15.150:0/2762346035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e01a0580 con 0x7f69e0102580 2024-05-10T04:03:54.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.215+0000 7f69e481c700 1 -- 172.21.15.150:0/2762346035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e0107cd0 con 0x7f69e0102580 2024-05-10T04:03:54.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.217+0000 7f69d77fe700 1 -- 172.21.15.150:0/2762346035 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f69d80040b0 con 0x7f69e0102580 2024-05-10T04:03:54.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.218+0000 7f69d77fe700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69cc0a11b0 0x7f69cc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:54.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.218+0000 7f69d77fe700 1 -- 172.21.15.150:0/2762346035 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f69d80ce8d0 con 0x7f69e0102580 2024-05-10T04:03:54.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.218+0000 7f69de59c700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69cc0a11b0 0x7f69cc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:54.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.218+0000 7f69de59c700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69cc0a11b0 0x7f69cc0a36c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f69d0005950 tx=0x7f69d00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:54.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.222+0000 7f69d77fe700 1 -- 172.21.15.150:0/2762346035 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f69d8097c20 con 0x7f69e0102580 2024-05-10T04:03:54.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.420+0000 7f69e481c700 1 -- 172.21.15.150:0/2762346035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69e010c2f0 con 0x7f69e0102580 2024-05-10T04:03:54.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.422+0000 7f69d77fe700 1 -- 172.21.15.150:0/2762346035 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24554 (secure 0 0 0) 0x7f69d809b240 con 0x7f69e0102580 2024-05-10T04:03:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:54.425 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":87,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:03:30.185138+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:03:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.424+0000 7f69e481c700 1 -- 172.21.15.150:0/2762346035 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69cc0a11b0 msgr2=0x7f69cc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.424+0000 7f69e481c700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69cc0a11b0 0x7f69cc0a36c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f69d0005950 tx=0x7f69d00058e0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.424+0000 7f69e481c700 1 -- 172.21.15.150:0/2762346035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e0102580 msgr2=0x7f69e019f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.424+0000 7f69e481c700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e0102580 0x7f69e019f460 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f69d8006070 tx=0x7f69d80094f0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.424+0000 7f69e481c700 1 -- 172.21.15.150:0/2762346035 shutdown_connections 2024-05-10T04:03:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.424+0000 7f69e481c700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f69cc0a11b0 0x7f69cc0a36c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.424+0000 7f69e481c700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69e0102580 0x7f69e019f460 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.424+0000 7f69e481c700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69e0100ac0 0x7f69e0068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.424+0000 7f69e481c700 1 --2- 172.21.15.150:0/2762346035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69e0101be0 0x7f69e019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.424+0000 7f69e481c700 1 -- 172.21.15.150:0/2762346035 >> 172.21.15.150:0/2762346035 conn(0x7f69e00fc0d0 msgr2=0x7f69e0104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.424+0000 7f69e481c700 1 -- 172.21.15.150:0/2762346035 shutdown_connections 2024-05-10T04:03:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.424+0000 7f69e481c700 1 -- 172.21.15.150:0/2762346035 wait complete. 2024-05-10T04:03:54.438 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:54 smithi150 ceph-mon[42477]: pgmap v269: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:54.439 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:54 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1162146436' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T04:03:54.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:02:30.076694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:02:30.760710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:02:33.850714+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:03:54.440 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:03:54.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:54 smithi181 ceph-mon[46602]: pgmap v269: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:54.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:54 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1162146436' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T04:03:54.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:54 smithi161 ceph-mon[46918]: pgmap v269: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:54 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1162146436' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T04:03:54.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.574+0000 7f408ea21700 1 -- 172.21.15.150:0/4149002158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4088101be0 msgr2=0x7f4088101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:54.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.574+0000 7f408ea21700 1 --2- 172.21.15.150:0/4149002158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4088101be0 0x7f4088101ff0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f407c009990 tx=0x7f407c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.574+0000 7f408ea21700 1 -- 172.21.15.150:0/4149002158 shutdown_connections 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.574+0000 7f408ea21700 1 --2- 172.21.15.150:0/4149002158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4088102580 0x7f4088108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.574+0000 7f408ea21700 1 --2- 172.21.15.150:0/4149002158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4088100ac0 0x7f4088100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.574+0000 7f408ea21700 1 --2- 172.21.15.150:0/4149002158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4088101be0 0x7f4088101ff0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.574+0000 7f408ea21700 1 -- 172.21.15.150:0/4149002158 >> 172.21.15.150:0/4149002158 conn(0x7f40880fc0d0 msgr2=0x7f40880fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.574+0000 7f408ea21700 1 -- 172.21.15.150:0/4149002158 shutdown_connections 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.574+0000 7f408ea21700 1 -- 172.21.15.150:0/4149002158 wait complete. 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f408ea21700 1 Processor -- start 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f408ea21700 1 -- start start 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f408ea21700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4088100ac0 0x7f4088068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f408ea21700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4088101be0 0x7f408819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f408ea21700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4088102580 0x7f408819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f408ea21700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408819f9a0 con 0x7f4088101be0 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f408ea21700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408819fb10 con 0x7f4088100ac0 2024-05-10T04:03:54.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f408ea21700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408819fcd0 con 0x7f4088102580 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f40877fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4088101be0 0x7f408819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f40877fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4088101be0 0x7f408819cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38540/0 (socket says 172.21.15.150:38540) 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f40877fe700 1 -- 172.21.15.150:0/3924365175 learned_addr learned my addr 172.21.15.150:0/3924365175 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f40877fe700 1 -- 172.21.15.150:0/3924365175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4088102580 msgr2=0x7f408819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f40877fe700 1 --2- 172.21.15.150:0/3924365175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4088102580 0x7f408819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f40877fe700 1 -- 172.21.15.150:0/3924365175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4088100ac0 msgr2=0x7f4088068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f40877fe700 1 --2- 172.21.15.150:0/3924365175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4088100ac0 0x7f4088068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.575+0000 7f40877fe700 1 -- 172.21.15.150:0/3924365175 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c0095f0 con 0x7f4088101be0 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.576+0000 7f40877fe700 1 --2- 172.21.15.150:0/3924365175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4088101be0 0x7f408819cc90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f407c0098e0 tx=0x7f407c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.576+0000 7f40857fa700 1 -- 172.21.15.150:0/3924365175 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c01d070 con 0x7f4088101be0 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.576+0000 7f40857fa700 1 -- 172.21.15.150:0/3924365175 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f407c00fb50 con 0x7f4088101be0 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.576+0000 7f408ea21700 1 -- 172.21.15.150:0/3924365175 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408819ff50 con 0x7f4088101be0 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.576+0000 7f40857fa700 1 -- 172.21.15.150:0/3924365175 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c017650 con 0x7f4088101be0 2024-05-10T04:03:54.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.576+0000 7f408ea21700 1 -- 172.21.15.150:0/3924365175 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40881a0370 con 0x7f4088101be0 2024-05-10T04:03:54.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.577+0000 7f408ea21700 1 -- 172.21.15.150:0/3924365175 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4088107cd0 con 0x7f4088101be0 2024-05-10T04:03:54.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.578+0000 7f40857fa700 1 -- 172.21.15.150:0/3924365175 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f407c00fcc0 con 0x7f4088101be0 2024-05-10T04:03:54.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.579+0000 7f40857fa700 1 --2- 172.21.15.150:0/3924365175 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f40700a12e0 0x7f40700a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:54.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.579+0000 7f4087fff700 1 --2- 172.21.15.150:0/3924365175 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f40700a12e0 0x7f40700a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:54.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.580+0000 7f40857fa700 1 -- 172.21.15.150:0/3924365175 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f407c0ce870 con 0x7f4088101be0 2024-05-10T04:03:54.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.580+0000 7f4087fff700 1 --2- 172.21.15.150:0/3924365175 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f40700a12e0 0x7f40700a37f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4078005fd0 tx=0x7f4078005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:54.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.583+0000 7f40857fa700 1 -- 172.21.15.150:0/3924365175 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f407c097c20 con 0x7f4088101be0 2024-05-10T04:03:54.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.777+0000 7f408ea21700 1 -- 172.21.15.150:0/3924365175 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f408810c570 con 0x7f4088101be0 2024-05-10T04:03:54.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.778+0000 7f40857fa700 1 -- 172.21.15.150:0/3924365175 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f407c09b240 con 0x7f4088101be0 2024-05-10T04:03:54.780 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:03:54.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.780+0000 7f408ea21700 1 -- 172.21.15.150:0/3924365175 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f40700a12e0 msgr2=0x7f40700a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:54.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.780+0000 7f408ea21700 1 --2- 172.21.15.150:0/3924365175 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f40700a12e0 0x7f40700a37f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4078005fd0 tx=0x7f4078005d00 comp rx=0 tx=0).stop 2024-05-10T04:03:54.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.780+0000 7f408ea21700 1 -- 172.21.15.150:0/3924365175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4088101be0 msgr2=0x7f408819cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:54.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.780+0000 7f408ea21700 1 --2- 172.21.15.150:0/3924365175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4088101be0 0x7f408819cc90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f407c0098e0 tx=0x7f407c009580 comp rx=0 tx=0).stop 2024-05-10T04:03:54.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.780+0000 7f408ea21700 1 -- 172.21.15.150:0/3924365175 shutdown_connections 2024-05-10T04:03:54.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.780+0000 7f408ea21700 1 --2- 172.21.15.150:0/3924365175 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f40700a12e0 0x7f40700a37f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.780+0000 7f408ea21700 1 --2- 172.21.15.150:0/3924365175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4088102580 0x7f408819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.780+0000 7f408ea21700 1 --2- 172.21.15.150:0/3924365175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4088100ac0 0x7f4088068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.780+0000 7f408ea21700 1 --2- 172.21.15.150:0/3924365175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4088101be0 0x7f408819cc90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.780+0000 7f408ea21700 1 -- 172.21.15.150:0/3924365175 >> 172.21.15.150:0/3924365175 conn(0x7f40880fc0d0 msgr2=0x7f4088104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:54.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.780+0000 7f408ea21700 1 -- 172.21.15.150:0/3924365175 shutdown_connections 2024-05-10T04:03:54.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.780+0000 7f408ea21700 1 -- 172.21.15.150:0/3924365175 wait complete. 2024-05-10T04:03:54.795 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:03:54.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.930+0000 7fd09a228700 1 -- 172.21.15.150:0/1053794024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd094100ac0 msgr2=0x7fd094100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:54.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.930+0000 7fd09a228700 1 --2- 172.21.15.150:0/1053794024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd094100ac0 0x7fd094100ed0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd084009990 tx=0x7fd084009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.930+0000 7fd09a228700 1 -- 172.21.15.150:0/1053794024 shutdown_connections 2024-05-10T04:03:54.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.930+0000 7fd09a228700 1 --2- 172.21.15.150:0/1053794024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd094102580 0x7fd094108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.930+0000 7fd09a228700 1 --2- 172.21.15.150:0/1053794024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd094100ac0 0x7fd094100ed0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.930+0000 7fd09a228700 1 --2- 172.21.15.150:0/1053794024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd094101be0 0x7fd094101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.930+0000 7fd09a228700 1 -- 172.21.15.150:0/1053794024 >> 172.21.15.150:0/1053794024 conn(0x7fd0940fc0d0 msgr2=0x7fd0940fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:54.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.930+0000 7fd09a228700 1 -- 172.21.15.150:0/1053794024 shutdown_connections 2024-05-10T04:03:54.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.930+0000 7fd09a228700 1 -- 172.21.15.150:0/1053794024 wait complete. 2024-05-10T04:03:54.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.930+0000 7fd09a228700 1 Processor -- start 2024-05-10T04:03:54.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd09a228700 1 -- start start 2024-05-10T04:03:54.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd09a228700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd094100ac0 0x7fd094196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd09a228700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd094101be0 0x7fd094197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd09a228700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd094102580 0x7fd09419e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd09a228700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09419e890 con 0x7fd094100ac0 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd09a228700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09419e9d0 con 0x7fd094102580 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd09a228700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09419eb60 con 0x7fd094101be0 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd094100ac0 0x7fd094196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd094100ac0 0x7fd094196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38556/0 (socket says 172.21.15.150:38556) 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd0937fe700 1 -- 172.21.15.150:0/1069637223 learned_addr learned my addr 172.21.15.150:0/1069637223 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd093fff700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd094102580 0x7fd09419e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd092ffd700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd094101be0 0x7fd094197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd0937fe700 1 -- 172.21.15.150:0/1069637223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd094101be0 msgr2=0x7fd094197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd0937fe700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd094101be0 0x7fd094197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd0937fe700 1 -- 172.21.15.150:0/1069637223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd094102580 msgr2=0x7fd09419e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd0937fe700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd094102580 0x7fd09419e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd0937fe700 1 -- 172.21.15.150:0/1069637223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0840095f0 con 0x7fd094100ac0 2024-05-10T04:03:54.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.931+0000 7fd093fff700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd094102580 0x7fd09419e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:54.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.932+0000 7fd0937fe700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd094100ac0 0x7fd094196df0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd084000c00 tx=0x7fd08400f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:54.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.932+0000 7fd090ff9700 1 -- 172.21.15.150:0/1069637223 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08401d070 con 0x7fd094100ac0 2024-05-10T04:03:54.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.932+0000 7fd090ff9700 1 -- 172.21.15.150:0/1069637223 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd08400fcc0 con 0x7fd094100ac0 2024-05-10T04:03:54.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.932+0000 7fd09a228700 1 -- 172.21.15.150:0/1069637223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09419ede0 con 0x7fd094100ac0 2024-05-10T04:03:54.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.932+0000 7fd090ff9700 1 -- 172.21.15.150:0/1069637223 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084017650 con 0x7fd094100ac0 2024-05-10T04:03:54.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.932+0000 7fd09a228700 1 -- 172.21.15.150:0/1069637223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09419f1b0 con 0x7fd094100ac0 2024-05-10T04:03:54.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.933+0000 7fd09a228700 1 -- 172.21.15.150:0/1069637223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd094107cd0 con 0x7fd094100ac0 2024-05-10T04:03:54.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.935+0000 7fd090ff9700 1 -- 172.21.15.150:0/1069637223 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd084022470 con 0x7fd094100ac0 2024-05-10T04:03:54.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.935+0000 7fd090ff9700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd07c0a1260 0x7fd07c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:54.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.935+0000 7fd090ff9700 1 -- 172.21.15.150:0/1069637223 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd0840ce640 con 0x7fd094100ac0 2024-05-10T04:03:54.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.935+0000 7fd092ffd700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd07c0a1260 0x7fd07c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:54.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.936+0000 7fd092ffd700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd07c0a1260 0x7fd07c0a3770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd094197e90 tx=0x7fd088005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:54.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:54.939+0000 7fd090ff9700 1 -- 172.21.15.150:0/1069637223 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd0840979f0 con 0x7fd094100ac0 2024-05-10T04:03:55.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.136+0000 7fd09a228700 1 -- 172.21.15.150:0/1069637223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd09410c430 con 0x7fd094100ac0 2024-05-10T04:03:55.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.137+0000 7fd090ff9700 1 -- 172.21.15.150:0/1069637223 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd08409b010 con 0x7fd094100ac0 2024-05-10T04:03:55.139 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:03:55.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.139+0000 7fd09a228700 1 -- 172.21.15.150:0/1069637223 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd07c0a1260 msgr2=0x7fd07c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:55.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.139+0000 7fd09a228700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd07c0a1260 0x7fd07c0a3770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd094197e90 tx=0x7fd088005d70 comp rx=0 tx=0).stop 2024-05-10T04:03:55.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.139+0000 7fd09a228700 1 -- 172.21.15.150:0/1069637223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd094100ac0 msgr2=0x7fd094196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:55.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.139+0000 7fd09a228700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd094100ac0 0x7fd094196df0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd084000c00 tx=0x7fd08400f720 comp rx=0 tx=0).stop 2024-05-10T04:03:55.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.139+0000 7fd09a228700 1 -- 172.21.15.150:0/1069637223 shutdown_connections 2024-05-10T04:03:55.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.139+0000 7fd09a228700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd07c0a1260 0x7fd07c0a3770 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.139+0000 7fd09a228700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd094102580 0x7fd09419e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.139+0000 7fd09a228700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd094100ac0 0x7fd094196df0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.139+0000 7fd09a228700 1 --2- 172.21.15.150:0/1069637223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd094101be0 0x7fd094197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.139+0000 7fd09a228700 1 -- 172.21.15.150:0/1069637223 >> 172.21.15.150:0/1069637223 conn(0x7fd0940fc0d0 msgr2=0x7fd094104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:55.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.139+0000 7fd09a228700 1 -- 172.21.15.150:0/1069637223 shutdown_connections 2024-05-10T04:03:55.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.139+0000 7fd09a228700 1 -- 172.21.15.150:0/1069637223 wait complete. 2024-05-10T04:03:55.155 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:03:55.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.290+0000 7f8530ef1700 1 -- 172.21.15.150:0/3778957537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c100570 msgr2=0x7f852c10a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:55.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.290+0000 7f8530ef1700 1 --2- 172.21.15.150:0/3778957537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c100570 0x7f852c10a190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f852400b0a0 tx=0x7f852400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.290+0000 7f8530ef1700 1 -- 172.21.15.150:0/3778957537 shutdown_connections 2024-05-10T04:03:55.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.290+0000 7f8530ef1700 1 --2- 172.21.15.150:0/3778957537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c100570 0x7f852c10a190 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.290+0000 7f8530ef1700 1 --2- 172.21.15.150:0/3778957537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c1031e0 0x7f852c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.290+0000 7f8530ef1700 1 --2- 172.21.15.150:0/3778957537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f852c0ffc20 0x7f852c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.290+0000 7f8530ef1700 1 -- 172.21.15.150:0/3778957537 >> 172.21.15.150:0/3778957537 conn(0x7f852c0fb8f0 msgr2=0x7f852c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:55.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.290+0000 7f8530ef1700 1 -- 172.21.15.150:0/3778957537 shutdown_connections 2024-05-10T04:03:55.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.290+0000 7f8530ef1700 1 -- 172.21.15.150:0/3778957537 wait complete. 2024-05-10T04:03:55.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f8530ef1700 1 Processor -- start 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f8530ef1700 1 -- start start 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f8530ef1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c0ffc20 0x7f852c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f8530ef1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c100570 0x7f852c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f8530ef1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f852c1031e0 0x7f852c19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f8530ef1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c19fa90 con 0x7f852c0ffc20 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f8530ef1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c19fbd0 con 0x7f852c100570 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f8530ef1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c19fd60 con 0x7f852c1031e0 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f852a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c0ffc20 0x7f852c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f852a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c0ffc20 0x7f852c197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38560/0 (socket says 172.21.15.150:38560) 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f852a59c700 1 -- 172.21.15.150:0/2232133664 learned_addr learned my addr 172.21.15.150:0/2232133664 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f852ad9d700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f852c1031e0 0x7f852c19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f8529d9b700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c100570 0x7f852c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f852a59c700 1 -- 172.21.15.150:0/2232133664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f852c1031e0 msgr2=0x7f852c19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f852a59c700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f852c1031e0 0x7f852c19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f852a59c700 1 -- 172.21.15.150:0/2232133664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c100570 msgr2=0x7f852c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:55.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f852a59c700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c100570 0x7f852c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f852a59c700 1 -- 172.21.15.150:0/2232133664 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8524009d00 con 0x7f852c0ffc20 2024-05-10T04:03:55.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.291+0000 7f8529d9b700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c100570 0x7f852c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:55.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.292+0000 7f852a59c700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c0ffc20 0x7f852c197ff0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d980 tx=0x7f851c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:55.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.292+0000 7f851b7fe700 1 -- 172.21.15.150:0/2232133664 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00b950 con 0x7f852c0ffc20 2024-05-10T04:03:55.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.292+0000 7f851b7fe700 1 -- 172.21.15.150:0/2232133664 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f851c007620 con 0x7f852c0ffc20 2024-05-10T04:03:55.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.292+0000 7f8530ef1700 1 -- 172.21.15.150:0/2232133664 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852c19ffe0 con 0x7f852c0ffc20 2024-05-10T04:03:55.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.292+0000 7f851b7fe700 1 -- 172.21.15.150:0/2232133664 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00f3c0 con 0x7f852c0ffc20 2024-05-10T04:03:55.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.292+0000 7f8530ef1700 1 -- 172.21.15.150:0/2232133664 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c1a0490 con 0x7f852c0ffc20 2024-05-10T04:03:55.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.293+0000 7f8530ef1700 1 -- 172.21.15.150:0/2232133664 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c109030 con 0x7f852c0ffc20 2024-05-10T04:03:55.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.294+0000 7f851b7fe700 1 -- 172.21.15.150:0/2232133664 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f851c0105d0 con 0x7f852c0ffc20 2024-05-10T04:03:55.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.295+0000 7f851b7fe700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f85140a1260 0x7f85140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:55.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.295+0000 7f851b7fe700 1 -- 172.21.15.150:0/2232133664 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f851c0cd3e0 con 0x7f852c0ffc20 2024-05-10T04:03:55.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.295+0000 7f8529d9b700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f85140a1260 0x7f85140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:55.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.295+0000 7f8529d9b700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f85140a1260 0x7f85140a3770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f852c199090 tx=0x7f852000a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:55.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.298+0000 7f851b7fe700 1 -- 172.21.15.150:0/2232133664 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f851c096700 con 0x7f852c0ffc20 2024-05-10T04:03:55.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:55 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2762346035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:55.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:55 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3924365175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:03:55.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:55 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1069637223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:03:55.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:55 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2762346035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:55.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:55 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3924365175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:03:55.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:55 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1069637223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:03:55.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:55 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2762346035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:55.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:55 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3924365175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:03:55.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:55 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1069637223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:03:55.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.500+0000 7f8530ef1700 1 -- 172.21.15.150:0/2232133664 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f852c10d3d0 con 0x7f852c0ffc20 2024-05-10T04:03:55.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.500+0000 7f851b7fe700 1 -- 172.21.15.150:0/2232133664 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f851c099d20 con 0x7f852c0ffc20 2024-05-10T04:03:55.503 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:03:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.503+0000 7f8530ef1700 1 -- 172.21.15.150:0/2232133664 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f85140a1260 msgr2=0x7f85140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.503+0000 7f8530ef1700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f85140a1260 0x7f85140a3770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f852c199090 tx=0x7f852000a680 comp rx=0 tx=0).stop 2024-05-10T04:03:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.503+0000 7f8530ef1700 1 -- 172.21.15.150:0/2232133664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c0ffc20 msgr2=0x7f852c197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.503+0000 7f8530ef1700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c0ffc20 0x7f852c197ff0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d980 tx=0x7f851c00dc90 comp rx=0 tx=0).stop 2024-05-10T04:03:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.503+0000 7f8530ef1700 1 -- 172.21.15.150:0/2232133664 shutdown_connections 2024-05-10T04:03:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.503+0000 7f8530ef1700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f85140a1260 0x7f85140a3770 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.503+0000 7f8530ef1700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f852c1031e0 0x7f852c19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.503+0000 7f8530ef1700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f852c0ffc20 0x7f852c197ff0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.503+0000 7f8530ef1700 1 --2- 172.21.15.150:0/2232133664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c100570 0x7f852c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.503+0000 7f8530ef1700 1 -- 172.21.15.150:0/2232133664 >> 172.21.15.150:0/2232133664 conn(0x7f852c0fb8f0 msgr2=0x7f852c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:55.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.503+0000 7f8530ef1700 1 -- 172.21.15.150:0/2232133664 shutdown_connections 2024-05-10T04:03:55.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.503+0000 7f8530ef1700 1 -- 172.21.15.150:0/2232133664 wait complete. 2024-05-10T04:03:55.518 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:03:55.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.653+0000 7fa06edf6700 1 -- 172.21.15.150:0/3106494337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa068100ae0 msgr2=0x7fa068100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:55.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.653+0000 7fa06edf6700 1 --2- 172.21.15.150:0/3106494337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa068100ae0 0x7fa068100ef0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa064009990 tx=0x7fa064009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.653+0000 7fa06edf6700 1 -- 172.21.15.150:0/3106494337 shutdown_connections 2024-05-10T04:03:55.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.653+0000 7fa06edf6700 1 --2- 172.21.15.150:0/3106494337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0681025a0 0x7fa068108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.653+0000 7fa06edf6700 1 --2- 172.21.15.150:0/3106494337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa068100ae0 0x7fa068100ef0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.653+0000 7fa06edf6700 1 --2- 172.21.15.150:0/3106494337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa068101c00 0x7fa068102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.653+0000 7fa06edf6700 1 -- 172.21.15.150:0/3106494337 >> 172.21.15.150:0/3106494337 conn(0x7fa0680fc0f0 msgr2=0x7fa0680fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:55.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.653+0000 7fa06edf6700 1 -- 172.21.15.150:0/3106494337 shutdown_connections 2024-05-10T04:03:55.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.653+0000 7fa06edf6700 1 -- 172.21.15.150:0/3106494337 wait complete. 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.654+0000 7fa06edf6700 1 Processor -- start 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.654+0000 7fa06edf6700 1 -- start start 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.654+0000 7fa06edf6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa068100ae0 0x7fa0681048a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.654+0000 7fa06edf6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa068101c00 0x7fa068102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.654+0000 7fa06edf6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0681025a0 0x7fa068103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.654+0000 7fa06edf6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa068103b60 con 0x7fa0681025a0 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.654+0000 7fa06edf6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa068103cd0 con 0x7fa068100ae0 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.654+0000 7fa06edf6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0681a2060 con 0x7fa068101c00 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.654+0000 7fa06d393700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0681025a0 0x7fa068103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.654+0000 7fa06d393700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0681025a0 0x7fa068103480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38578/0 (socket says 172.21.15.150:38578) 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.654+0000 7fa06d393700 1 -- 172.21.15.150:0/202448838 learned_addr learned my addr 172.21.15.150:0/202448838 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.654+0000 7fa05ffff700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa068101c00 0x7fa068102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.655+0000 7fa06cb92700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa068100ae0 0x7fa0681048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:55.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.655+0000 7fa06d393700 1 -- 172.21.15.150:0/202448838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa068101c00 msgr2=0x7fa068102f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:55.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.655+0000 7fa06d393700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa068101c00 0x7fa068102f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.655+0000 7fa06d393700 1 -- 172.21.15.150:0/202448838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa068100ae0 msgr2=0x7fa0681048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:55.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.655+0000 7fa06d393700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa068100ae0 0x7fa0681048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.655+0000 7fa06d393700 1 -- 172.21.15.150:0/202448838 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0640095f0 con 0x7fa0681025a0 2024-05-10T04:03:55.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.655+0000 7fa06d393700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0681025a0 0x7fa068103480 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa06000ef30 tx=0x7fa06000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:55.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.655+0000 7fa05dffb700 1 -- 172.21.15.150:0/202448838 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060018070 con 0x7fa0681025a0 2024-05-10T04:03:55.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.655+0000 7fa05dffb700 1 -- 172.21.15.150:0/202448838 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa06000f040 con 0x7fa0681025a0 2024-05-10T04:03:55.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.655+0000 7fa06edf6700 1 -- 172.21.15.150:0/202448838 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0681a2260 con 0x7fa0681025a0 2024-05-10T04:03:55.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.655+0000 7fa05dffb700 1 -- 172.21.15.150:0/202448838 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000cdb0 con 0x7fa0681025a0 2024-05-10T04:03:55.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.655+0000 7fa06edf6700 1 -- 172.21.15.150:0/202448838 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0681a26f0 con 0x7fa0681025a0 2024-05-10T04:03:55.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.656+0000 7fa06edf6700 1 -- 172.21.15.150:0/202448838 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa068068730 con 0x7fa0681025a0 2024-05-10T04:03:55.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.658+0000 7fa05dffb700 1 -- 172.21.15.150:0/202448838 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa060003a00 con 0x7fa0681025a0 2024-05-10T04:03:55.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.658+0000 7fa05dffb700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa0500a1260 0x7fa0500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:55.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.658+0000 7fa06cb92700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa0500a1260 0x7fa0500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:55.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.658+0000 7fa05dffb700 1 -- 172.21.15.150:0/202448838 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa0600cca40 con 0x7fa0681025a0 2024-05-10T04:03:55.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.659+0000 7fa06cb92700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa0500a1260 0x7fa0500a3770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa06400b7b0 tx=0x7fa06401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:55.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.662+0000 7fa05dffb700 1 -- 172.21.15.150:0/202448838 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa060095df0 con 0x7fa0681025a0 2024-05-10T04:03:55.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.860+0000 7fa06edf6700 1 -- 172.21.15.150:0/202448838 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa0681a2a10 con 0x7fa0681025a0 2024-05-10T04:03:55.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.860+0000 7fa05dffb700 1 -- 172.21.15.150:0/202448838 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa060099410 con 0x7fa0681025a0 2024-05-10T04:03:55.862 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:03:55.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.862+0000 7fa06edf6700 1 -- 172.21.15.150:0/202448838 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa0500a1260 msgr2=0x7fa0500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:55.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.862+0000 7fa06edf6700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa0500a1260 0x7fa0500a3770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa06400b7b0 tx=0x7fa06401a040 comp rx=0 tx=0).stop 2024-05-10T04:03:55.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.862+0000 7fa06edf6700 1 -- 172.21.15.150:0/202448838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0681025a0 msgr2=0x7fa068103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:55.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.862+0000 7fa06edf6700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0681025a0 0x7fa068103480 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa06000ef30 tx=0x7fa06000c3a0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.862+0000 7fa06edf6700 1 -- 172.21.15.150:0/202448838 shutdown_connections 2024-05-10T04:03:55.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.862+0000 7fa06edf6700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa0500a1260 0x7fa0500a3770 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.862+0000 7fa06edf6700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0681025a0 0x7fa068103480 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.862+0000 7fa06edf6700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa068100ae0 0x7fa0681048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.862+0000 7fa06edf6700 1 --2- 172.21.15.150:0/202448838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa068101c00 0x7fa068102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:55.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.862+0000 7fa06edf6700 1 -- 172.21.15.150:0/202448838 >> 172.21.15.150:0/202448838 conn(0x7fa0680fc0f0 msgr2=0x7fa0680fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:55.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.862+0000 7fa06edf6700 1 -- 172.21.15.150:0/202448838 shutdown_connections 2024-05-10T04:03:55.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:55.862+0000 7fa06edf6700 1 -- 172.21.15.150:0/202448838 wait complete. 2024-05-10T04:03:55.878 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:03:56.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.009+0000 7f13b08e0700 1 -- 172.21.15.150:0/2196954728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13ac100ac0 msgr2=0x7f13ac100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.009+0000 7f13b08e0700 1 --2- 172.21.15.150:0/2196954728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13ac100ac0 0x7f13ac100ed0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f139c009990 tx=0x7f139c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.009+0000 7f13b08e0700 1 -- 172.21.15.150:0/2196954728 shutdown_connections 2024-05-10T04:03:56.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.009+0000 7f13b08e0700 1 --2- 172.21.15.150:0/2196954728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13ac102580 0x7f13ac108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.009+0000 7f13b08e0700 1 --2- 172.21.15.150:0/2196954728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13ac100ac0 0x7f13ac100ed0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.009+0000 7f13b08e0700 1 --2- 172.21.15.150:0/2196954728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ac101be0 0x7f13ac101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.009+0000 7f13b08e0700 1 -- 172.21.15.150:0/2196954728 >> 172.21.15.150:0/2196954728 conn(0x7f13ac0fc0d0 msgr2=0x7f13ac0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:56.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.009+0000 7f13b08e0700 1 -- 172.21.15.150:0/2196954728 shutdown_connections 2024-05-10T04:03:56.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.009+0000 7f13b08e0700 1 -- 172.21.15.150:0/2196954728 wait complete. 2024-05-10T04:03:56.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13b08e0700 1 Processor -- start 2024-05-10T04:03:56.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13b08e0700 1 -- start start 2024-05-10T04:03:56.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13b08e0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13ac100ac0 0x7f13ac104880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:56.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13b08e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ac101be0 0x7f13ac102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:56.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13b08e0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13ac102580 0x7f13ac103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:56.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13b08e0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac103b40 con 0x7f13ac102580 2024-05-10T04:03:56.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13b08e0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac103cb0 con 0x7f13ac100ac0 2024-05-10T04:03:56.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13b08e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac1a20a0 con 0x7f13ac101be0 2024-05-10T04:03:56.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13aad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13ac102580 0x7f13ac103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:56.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13aad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13ac102580 0x7f13ac103460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38582/0 (socket says 172.21.15.150:38582) 2024-05-10T04:03:56.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13aad9d700 1 -- 172.21.15.150:0/3360188404 learned_addr learned my addr 172.21.15.150:0/3360188404 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13a9d9b700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ac101be0 0x7f13ac102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13aa59c700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13ac100ac0 0x7f13ac104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13aad9d700 1 -- 172.21.15.150:0/3360188404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ac101be0 msgr2=0x7f13ac102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13aad9d700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ac101be0 0x7f13ac102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13aad9d700 1 -- 172.21.15.150:0/3360188404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13ac100ac0 msgr2=0x7f13ac104880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13aad9d700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13ac100ac0 0x7f13ac104880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13aad9d700 1 -- 172.21.15.150:0/3360188404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c0095f0 con 0x7f13ac102580 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13a9d9b700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ac101be0 0x7f13ac102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.010+0000 7f13aad9d700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13ac102580 0x7f13ac103460 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f13a400ef30 tx=0x7f13a400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.011+0000 7f139b7fe700 1 -- 172.21.15.150:0/3360188404 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4018070 con 0x7f13ac102580 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.011+0000 7f139b7fe700 1 -- 172.21.15.150:0/3360188404 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13a400f040 con 0x7f13ac102580 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.011+0000 7f13b08e0700 1 -- 172.21.15.150:0/3360188404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ac1a22a0 con 0x7f13ac102580 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.011+0000 7f139b7fe700 1 -- 172.21.15.150:0/3360188404 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4003dc0 con 0x7f13ac102580 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.011+0000 7f13b08e0700 1 -- 172.21.15.150:0/3360188404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ac1a2730 con 0x7f13ac102580 2024-05-10T04:03:56.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.012+0000 7f13b08e0700 1 -- 172.21.15.150:0/3360188404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ac0694d0 con 0x7f13ac102580 2024-05-10T04:03:56.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.013+0000 7f139b7fe700 1 -- 172.21.15.150:0/3360188404 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f13a400caa0 con 0x7f13ac102580 2024-05-10T04:03:56.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.013+0000 7f139b7fe700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13940a11b0 0x7f13940a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:56.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.013+0000 7f139b7fe700 1 -- 172.21.15.150:0/3360188404 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f13a40cd2b0 con 0x7f13ac102580 2024-05-10T04:03:56.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.013+0000 7f13aa59c700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13940a11b0 0x7f13940a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:56.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.014+0000 7f13aa59c700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13940a11b0 0x7f13940a36c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f139c00b2e0 tx=0x7f139c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:56.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.017+0000 7f139b7fe700 1 -- 172.21.15.150:0/3360188404 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f13a4096700 con 0x7f13ac102580 2024-05-10T04:03:56.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.262+0000 7f13b08e0700 1 -- 172.21.15.150:0/3360188404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13ac10c2a0 con 0x7f13ac102580 2024-05-10T04:03:56.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.262+0000 7f139b7fe700 1 -- 172.21.15.150:0/3360188404 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10886 (secure 0 0 0) 0x7f13a4099d20 con 0x7f13ac102580 2024-05-10T04:03:56.264 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:56.265 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:02:57.183845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:03:56.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.264+0000 7f13b08e0700 1 -- 172.21.15.150:0/3360188404 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13940a11b0 msgr2=0x7f13940a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.264+0000 7f13b08e0700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13940a11b0 0x7f13940a36c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f139c00b2e0 tx=0x7f139c005fb0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.264+0000 7f13b08e0700 1 -- 172.21.15.150:0/3360188404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13ac102580 msgr2=0x7f13ac103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.264+0000 7f13b08e0700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13ac102580 0x7f13ac103460 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f13a400ef30 tx=0x7f13a400c450 comp rx=0 tx=0).stop 2024-05-10T04:03:56.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.264+0000 7f13b08e0700 1 -- 172.21.15.150:0/3360188404 shutdown_connections 2024-05-10T04:03:56.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.264+0000 7f13b08e0700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13940a11b0 0x7f13940a36c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.264+0000 7f13b08e0700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13ac102580 0x7f13ac103460 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.265+0000 7f13b08e0700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13ac100ac0 0x7f13ac104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.265+0000 7f13b08e0700 1 --2- 172.21.15.150:0/3360188404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ac101be0 0x7f13ac102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.265+0000 7f13b08e0700 1 -- 172.21.15.150:0/3360188404 >> 172.21.15.150:0/3360188404 conn(0x7f13ac0fc0d0 msgr2=0x7f13ac0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:56.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.265+0000 7f13b08e0700 1 -- 172.21.15.150:0/3360188404 shutdown_connections 2024-05-10T04:03:56.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.265+0000 7f13b08e0700 1 -- 172.21.15.150:0/3360188404 wait complete. 2024-05-10T04:03:56.268 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 24 2024-05-10T04:03:56.280 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:03:56.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.410+0000 7fc5f4867700 1 -- 172.21.15.150:0/2316742679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f0100ac0 msgr2=0x7fc5f0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.410+0000 7fc5f4867700 1 --2- 172.21.15.150:0/2316742679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f0100ac0 0x7fc5f0100ed0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009990 tx=0x7fc5e0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.410+0000 7fc5f4867700 1 -- 172.21.15.150:0/2316742679 shutdown_connections 2024-05-10T04:03:56.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.410+0000 7fc5f4867700 1 --2- 172.21.15.150:0/2316742679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5f0102580 0x7fc5f0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.410+0000 7fc5f4867700 1 --2- 172.21.15.150:0/2316742679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f0100ac0 0x7fc5f0100ed0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.410+0000 7fc5f4867700 1 --2- 172.21.15.150:0/2316742679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f0101be0 0x7fc5f0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.410+0000 7fc5f4867700 1 -- 172.21.15.150:0/2316742679 >> 172.21.15.150:0/2316742679 conn(0x7fc5f00fc0d0 msgr2=0x7fc5f00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:56.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.410+0000 7fc5f4867700 1 -- 172.21.15.150:0/2316742679 shutdown_connections 2024-05-10T04:03:56.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.410+0000 7fc5f4867700 1 -- 172.21.15.150:0/2316742679 wait complete. 2024-05-10T04:03:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.411+0000 7fc5f4867700 1 Processor -- start 2024-05-10T04:03:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.411+0000 7fc5f4867700 1 -- start start 2024-05-10T04:03:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.411+0000 7fc5f4867700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f0100ac0 0x7fc5f0198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.411+0000 7fc5f4867700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5f0101be0 0x7fc5f0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.411+0000 7fc5f4867700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f0102580 0x7fc5f019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.411+0000 7fc5f4867700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f019fb00 con 0x7fc5f0102580 2024-05-10T04:03:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.411+0000 7fc5f4867700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f019fc70 con 0x7fc5f0101be0 2024-05-10T04:03:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.411+0000 7fc5f4867700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f019fe30 con 0x7fc5f0100ac0 2024-05-10T04:03:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5eed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f0102580 0x7fc5f019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:56.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5eed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f0102580 0x7fc5f019f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38592/0 (socket says 172.21.15.150:38592) 2024-05-10T04:03:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5eed9d700 1 -- 172.21.15.150:0/2483193938 learned_addr learned my addr 172.21.15.150:0/2483193938 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5ee59c700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f0100ac0 0x7fc5f0198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5edd9b700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5f0101be0 0x7fc5f0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5eed9d700 1 -- 172.21.15.150:0/2483193938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f0100ac0 msgr2=0x7fc5f0198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5eed9d700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f0100ac0 0x7fc5f0198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5eed9d700 1 -- 172.21.15.150:0/2483193938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5f0101be0 msgr2=0x7fc5f0198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5eed9d700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5f0101be0 0x7fc5f0198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5eed9d700 1 -- 172.21.15.150:0/2483193938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e00095f0 con 0x7fc5f0102580 2024-05-10T04:03:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5eed9d700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f0102580 0x7fc5f019f4f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8007ee0 tx=0x7fc5e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5e77fe700 1 -- 172.21.15.150:0/2483193938 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8017070 con 0x7fc5f0102580 2024-05-10T04:03:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5e77fe700 1 -- 172.21.15.150:0/2483193938 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e800f040 con 0x7fc5f0102580 2024-05-10T04:03:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5f4867700 1 -- 172.21.15.150:0/2483193938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f01a0110 con 0x7fc5f0102580 2024-05-10T04:03:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5e77fe700 1 -- 172.21.15.150:0/2483193938 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8005740 con 0x7fc5f0102580 2024-05-10T04:03:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.412+0000 7fc5f4867700 1 -- 172.21.15.150:0/2483193938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f01a0650 con 0x7fc5f0102580 2024-05-10T04:03:56.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.413+0000 7fc5f4867700 1 -- 172.21.15.150:0/2483193938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f00694d0 con 0x7fc5f0102580 2024-05-10T04:03:56.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.414+0000 7fc5e77fe700 1 -- 172.21.15.150:0/2483193938 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc5e800dc60 con 0x7fc5f0102580 2024-05-10T04:03:56.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.415+0000 7fc5e77fe700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc5dc0a11b0 0x7fc5dc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:56.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.415+0000 7fc5ee59c700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc5dc0a11b0 0x7fc5dc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:56.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.415+0000 7fc5e77fe700 1 -- 172.21.15.150:0/2483193938 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc5e80ccb80 con 0x7fc5f0102580 2024-05-10T04:03:56.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.416+0000 7fc5ee59c700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc5dc0a11b0 0x7fc5dc0a36c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0006010 tx=0x7fc5e001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:56.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.419+0000 7fc5e77fe700 1 -- 172.21.15.150:0/2483193938 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc5e8095ea0 con 0x7fc5f0102580 2024-05-10T04:03:56.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:56 smithi150 ceph-mon[42477]: pgmap v270: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T04:03:56.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:56 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2232133664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:03:56.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:56 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/202448838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:03:56.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:56 smithi181 ceph-mon[46602]: pgmap v270: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T04:03:56.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:56 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2232133664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:03:56.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:56 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/202448838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:03:56.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:56 smithi161 ceph-mon[46918]: pgmap v270: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T04:03:56.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:56 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2232133664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:03:56.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:56 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/202448838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:03:56.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.621+0000 7fc5f4867700 1 -- 172.21.15.150:0/2483193938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5f01a0910 con 0x7fc5f0102580 2024-05-10T04:03:56.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.623+0000 7fc5e77fe700 1 -- 172.21.15.150:0/2483193938 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24554 (secure 0 0 0) 0x7fc5e80994c0 con 0x7fc5f0102580 2024-05-10T04:03:56.625 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:56.625 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":87,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:03:30.185138+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:03:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.625+0000 7fc5f4867700 1 -- 172.21.15.150:0/2483193938 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc5dc0a11b0 msgr2=0x7fc5dc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.625+0000 7fc5f4867700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc5dc0a11b0 0x7fc5dc0a36c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0006010 tx=0x7fc5e001a040 comp rx=0 tx=0).stop 2024-05-10T04:03:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.625+0000 7fc5f4867700 1 -- 172.21.15.150:0/2483193938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f0102580 msgr2=0x7fc5f019f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.625+0000 7fc5f4867700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f0102580 0x7fc5f019f4f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8007ee0 tx=0x7fc5e800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.625+0000 7fc5f4867700 1 -- 172.21.15.150:0/2483193938 shutdown_connections 2024-05-10T04:03:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.625+0000 7fc5f4867700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc5dc0a11b0 0x7fc5dc0a36c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.625+0000 7fc5f4867700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f0102580 0x7fc5f019f4f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.625+0000 7fc5f4867700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f0100ac0 0x7fc5f0198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.625+0000 7fc5f4867700 1 --2- 172.21.15.150:0/2483193938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5f0101be0 0x7fc5f0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.625+0000 7fc5f4867700 1 -- 172.21.15.150:0/2483193938 >> 172.21.15.150:0/2483193938 conn(0x7fc5f00fc0d0 msgr2=0x7fc5f00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.625+0000 7fc5f4867700 1 -- 172.21.15.150:0/2483193938 shutdown_connections 2024-05-10T04:03:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.625+0000 7fc5f4867700 1 -- 172.21.15.150:0/2483193938 wait complete. 2024-05-10T04:03:56.640 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:02:30.076694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:02:30.760710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:02:33.850714+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:03:56.641 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:03:56.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.774+0000 7f4ed17cb700 1 -- 172.21.15.150:0/436527849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ecc100af0 msgr2=0x7f4ecc100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.774+0000 7f4ed17cb700 1 --2- 172.21.15.150:0/436527849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ecc100af0 0x7f4ecc100f00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009990 tx=0x7f4ec0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.774+0000 7f4ed17cb700 1 -- 172.21.15.150:0/436527849 shutdown_connections 2024-05-10T04:03:56.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.774+0000 7f4ed17cb700 1 --2- 172.21.15.150:0/436527849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc1025b0 0x7f4ecc108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.774+0000 7f4ed17cb700 1 --2- 172.21.15.150:0/436527849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ecc100af0 0x7f4ecc100f00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.774+0000 7f4ed17cb700 1 --2- 172.21.15.150:0/436527849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc101c10 0x7f4ecc102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.774+0000 7f4ed17cb700 1 -- 172.21.15.150:0/436527849 >> 172.21.15.150:0/436527849 conn(0x7f4ecc0fc0e0 msgr2=0x7f4ecc0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:56.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.774+0000 7f4ed17cb700 1 -- 172.21.15.150:0/436527849 shutdown_connections 2024-05-10T04:03:56.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.774+0000 7f4ed17cb700 1 -- 172.21.15.150:0/436527849 wait complete. 2024-05-10T04:03:56.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.774+0000 7f4ed17cb700 1 Processor -- start 2024-05-10T04:03:56.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ed17cb700 1 -- start start 2024-05-10T04:03:56.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ed17cb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ecc100af0 0x7f4ecc196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:56.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ed17cb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc101c10 0x7f4ecc1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:56.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ed17cb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc1025b0 0x7f4ecc19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:56.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ed17cb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc19e890 con 0x7f4ecc100af0 2024-05-10T04:03:56.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ed17cb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc19ea00 con 0x7f4ecc1025b0 2024-05-10T04:03:56.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ed17cb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc19ebc0 con 0x7f4ecc101c10 2024-05-10T04:03:56.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ecaffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ecc100af0 0x7f4ecc196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:56.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ecaffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ecc100af0 0x7f4ecc196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38608/0 (socket says 172.21.15.150:38608) 2024-05-10T04:03:56.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ecaffd700 1 -- 172.21.15.150:0/2652452612 learned_addr learned my addr 172.21.15.150:0/2652452612 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:56.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4eca7fc700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc101c10 0x7f4ecc1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:56.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ecb7fe700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc1025b0 0x7f4ecc19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:56.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ecaffd700 1 -- 172.21.15.150:0/2652452612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc101c10 msgr2=0x7f4ecc1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ecaffd700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc101c10 0x7f4ecc1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ecaffd700 1 -- 172.21.15.150:0/2652452612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc1025b0 msgr2=0x7f4ecc19e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ecaffd700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc1025b0 0x7f4ecc19e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ecaffd700 1 -- 172.21.15.150:0/2652452612 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec00095f0 con 0x7f4ecc100af0 2024-05-10T04:03:56.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.775+0000 7f4ecaffd700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ecc100af0 0x7f4ecc196db0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0006010 tx=0x7f4ec000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:56.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.776+0000 7f4eb3fff700 1 -- 172.21.15.150:0/2652452612 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec001d070 con 0x7f4ecc100af0 2024-05-10T04:03:56.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.776+0000 7f4ed17cb700 1 -- 172.21.15.150:0/2652452612 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc19ee40 con 0x7f4ecc100af0 2024-05-10T04:03:56.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.776+0000 7f4eb3fff700 1 -- 172.21.15.150:0/2652452612 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec0003960 con 0x7f4ecc100af0 2024-05-10T04:03:56.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.776+0000 7f4eb3fff700 1 -- 172.21.15.150:0/2652452612 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0005450 con 0x7f4ecc100af0 2024-05-10T04:03:56.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.776+0000 7f4ed17cb700 1 -- 172.21.15.150:0/2652452612 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc19f200 con 0x7f4ecc100af0 2024-05-10T04:03:56.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.777+0000 7f4ed17cb700 1 -- 172.21.15.150:0/2652452612 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc107d00 con 0x7f4ecc100af0 2024-05-10T04:03:56.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.778+0000 7f4eb3fff700 1 -- 172.21.15.150:0/2652452612 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4ec00055b0 con 0x7f4ecc100af0 2024-05-10T04:03:56.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.779+0000 7f4eb3fff700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4eb40a1260 0x7f4eb40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:56.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.779+0000 7f4eca7fc700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4eb40a1260 0x7f4eb40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:56.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.779+0000 7f4eb3fff700 1 -- 172.21.15.150:0/2652452612 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4ec00ce350 con 0x7f4ecc100af0 2024-05-10T04:03:56.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.779+0000 7f4eca7fc700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4eb40a1260 0x7f4eb40a3770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc197e50 tx=0x7f4ebc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:56.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.782+0000 7f4eb3fff700 1 -- 172.21.15.150:0/2652452612 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4ec00976a0 con 0x7f4ecc100af0 2024-05-10T04:03:56.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.979+0000 7f4ed17cb700 1 -- 172.21.15.150:0/2652452612 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4ecc10c2d0 con 0x7f4ecc100af0 2024-05-10T04:03:56.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.980+0000 7f4eb3fff700 1 -- 172.21.15.150:0/2652452612 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ec009acc0 con 0x7f4ecc100af0 2024-05-10T04:03:56.982 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:03:56.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.982+0000 7f4ed17cb700 1 -- 172.21.15.150:0/2652452612 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4eb40a1260 msgr2=0x7f4eb40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.982+0000 7f4ed17cb700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4eb40a1260 0x7f4eb40a3770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc197e50 tx=0x7f4ebc00b410 comp rx=0 tx=0).stop 2024-05-10T04:03:56.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.982+0000 7f4ed17cb700 1 -- 172.21.15.150:0/2652452612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ecc100af0 msgr2=0x7f4ecc196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:56.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.982+0000 7f4ed17cb700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ecc100af0 0x7f4ecc196db0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0006010 tx=0x7f4ec000f690 comp rx=0 tx=0).stop 2024-05-10T04:03:56.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.982+0000 7f4ed17cb700 1 -- 172.21.15.150:0/2652452612 shutdown_connections 2024-05-10T04:03:56.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.982+0000 7f4ed17cb700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f4eb40a1260 0x7f4eb40a3770 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.982+0000 7f4ed17cb700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc1025b0 0x7f4ecc19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.982+0000 7f4ed17cb700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ecc100af0 0x7f4ecc196db0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.982+0000 7f4ed17cb700 1 --2- 172.21.15.150:0/2652452612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc101c10 0x7f4ecc1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:56.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.982+0000 7f4ed17cb700 1 -- 172.21.15.150:0/2652452612 >> 172.21.15.150:0/2652452612 conn(0x7f4ecc0fc0e0 msgr2=0x7f4ecc104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:56.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.982+0000 7f4ed17cb700 1 -- 172.21.15.150:0/2652452612 shutdown_connections 2024-05-10T04:03:56.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:56.982+0000 7f4ed17cb700 1 -- 172.21.15.150:0/2652452612 wait complete. 2024-05-10T04:03:56.997 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:03:57.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.131+0000 7f232d857700 1 -- 172.21.15.150:0/1508958126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2328107a30 msgr2=0x7f2328109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:57.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.131+0000 7f232d857700 1 --2- 172.21.15.150:0/1508958126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2328107a30 0x7f2328109e50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f232000b0a0 tx=0x7f232000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.131+0000 7f232d857700 1 -- 172.21.15.150:0/1508958126 shutdown_connections 2024-05-10T04:03:57.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.131+0000 7f232d857700 1 --2- 172.21.15.150:0/1508958126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2328107a30 0x7f2328109e50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.131+0000 7f232d857700 1 --2- 172.21.15.150:0/1508958126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23280fffb0 0x7f23281003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.131+0000 7f232d857700 1 --2- 172.21.15.150:0/1508958126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23281050d0 0x7f23281074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.131+0000 7f232d857700 1 -- 172.21.15.150:0/1508958126 >> 172.21.15.150:0/1508958126 conn(0x7f23280fb880 msgr2=0x7f23280fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:57.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.131+0000 7f232d857700 1 -- 172.21.15.150:0/1508958126 shutdown_connections 2024-05-10T04:03:57.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.131+0000 7f232d857700 1 -- 172.21.15.150:0/1508958126 wait complete. 2024-05-10T04:03:57.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.131+0000 7f232d857700 1 Processor -- start 2024-05-10T04:03:57.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f232d857700 1 -- start start 2024-05-10T04:03:57.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f232d857700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23280fffb0 0x7f2328197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:57.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f232d857700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23281050d0 0x7f23281984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:57.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f232d857700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2328107a30 0x7f232819f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:57.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f232d857700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232819fa80 con 0x7f23280fffb0 2024-05-10T04:03:57.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f232d857700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232819fbf0 con 0x7f23281050d0 2024-05-10T04:03:57.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f232d857700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232819fdb0 con 0x7f2328107a30 2024-05-10T04:03:57.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f2326ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23280fffb0 0x7f2328197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:57.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f2326ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23280fffb0 0x7f2328197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38626/0 (socket says 172.21.15.150:38626) 2024-05-10T04:03:57.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f2326ffd700 1 -- 172.21.15.150:0/326791952 learned_addr learned my addr 172.21.15.150:0/326791952 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:57.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f23277fe700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2328107a30 0x7f232819f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:57.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f23267fc700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23281050d0 0x7f23281984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:57.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f2326ffd700 1 -- 172.21.15.150:0/326791952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2328107a30 msgr2=0x7f232819f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:57.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f2326ffd700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2328107a30 0x7f232819f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f2326ffd700 1 -- 172.21.15.150:0/326791952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23281050d0 msgr2=0x7f23281984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:57.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f2326ffd700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23281050d0 0x7f23281984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f2326ffd700 1 -- 172.21.15.150:0/326791952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2320009d00 con 0x7f23280fffb0 2024-05-10T04:03:57.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f2326ffd700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23280fffb0 0x7f2328197fa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f231800e940 tx=0x7f231800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:57.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f232c855700 1 -- 172.21.15.150:0/326791952 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231800cbd0 con 0x7f23280fffb0 2024-05-10T04:03:57.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f232d857700 1 -- 172.21.15.150:0/326791952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23281a0090 con 0x7f23280fffb0 2024-05-10T04:03:57.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f232c855700 1 -- 172.21.15.150:0/326791952 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23180044e0 con 0x7f23280fffb0 2024-05-10T04:03:57.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.132+0000 7f232c855700 1 -- 172.21.15.150:0/326791952 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318003dc0 con 0x7f23280fffb0 2024-05-10T04:03:57.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.133+0000 7f232d857700 1 -- 172.21.15.150:0/326791952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23281a05a0 con 0x7f23280fffb0 2024-05-10T04:03:57.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.133+0000 7f232d857700 1 -- 172.21.15.150:0/326791952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23280fd430 con 0x7f23280fffb0 2024-05-10T04:03:57.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.135+0000 7f232c855700 1 -- 172.21.15.150:0/326791952 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f2318003f20 con 0x7f23280fffb0 2024-05-10T04:03:57.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.135+0000 7f232c855700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f23100a1260 0x7f23100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:57.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.136+0000 7f23267fc700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f23100a1260 0x7f23100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:57.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.136+0000 7f232c855700 1 -- 172.21.15.150:0/326791952 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2318014070 con 0x7f23280fffb0 2024-05-10T04:03:57.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.136+0000 7f23267fc700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f23100a1260 0x7f23100a3770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f231c005950 tx=0x7f231c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:57.141 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.139+0000 7f232c855700 1 -- 172.21.15.150:0/326791952 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f23180965a0 con 0x7f23280fffb0 2024-05-10T04:03:57.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.341+0000 7f232d857700 1 -- 172.21.15.150:0/326791952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23280fce10 con 0x7f23280fffb0 2024-05-10T04:03:57.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.342+0000 7f232c855700 1 -- 172.21.15.150:0/326791952 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2318017e90 con 0x7f23280fffb0 2024-05-10T04:03:57.344 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:03:57.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.344+0000 7f232d857700 1 -- 172.21.15.150:0/326791952 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f23100a1260 msgr2=0x7f23100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:57.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.344+0000 7f232d857700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f23100a1260 0x7f23100a3770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f231c005950 tx=0x7f231c0058e0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.344+0000 7f232d857700 1 -- 172.21.15.150:0/326791952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23280fffb0 msgr2=0x7f2328197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:57.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.344+0000 7f232d857700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23280fffb0 0x7f2328197fa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f231800e940 tx=0x7f231800ed00 comp rx=0 tx=0).stop 2024-05-10T04:03:57.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.344+0000 7f232d857700 1 -- 172.21.15.150:0/326791952 shutdown_connections 2024-05-10T04:03:57.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.344+0000 7f232d857700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f23100a1260 0x7f23100a3770 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.344+0000 7f232d857700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2328107a30 0x7f232819f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.344+0000 7f232d857700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23280fffb0 0x7f2328197fa0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.344+0000 7f232d857700 1 --2- 172.21.15.150:0/326791952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23281050d0 0x7f23281984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.344+0000 7f232d857700 1 -- 172.21.15.150:0/326791952 >> 172.21.15.150:0/326791952 conn(0x7f23280fb880 msgr2=0x7f2328105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:57.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.344+0000 7f232d857700 1 -- 172.21.15.150:0/326791952 shutdown_connections 2024-05-10T04:03:57.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.344+0000 7f232d857700 1 -- 172.21.15.150:0/326791952 wait complete. 2024-05-10T04:03:57.358 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:57 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3360188404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:57.358 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:57 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2483193938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:57.359 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:57 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2652452612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:03:57.359 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:03:57.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:57 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3360188404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:57.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:57 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2483193938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:57.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:57 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2652452612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:03:57.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:57 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3360188404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:57.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:57 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2483193938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:57.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:57 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2652452612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:03:57.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.494+0000 7fdb749a1700 1 -- 172.21.15.150:0/29723337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb700ffc20 msgr2=0x7fdb70100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:57.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.494+0000 7fdb749a1700 1 --2- 172.21.15.150:0/29723337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb700ffc20 0x7fdb70100030 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009990 tx=0x7fdb64009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.494+0000 7fdb749a1700 1 -- 172.21.15.150:0/29723337 shutdown_connections 2024-05-10T04:03:57.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.494+0000 7fdb749a1700 1 --2- 172.21.15.150:0/29723337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb70100570 0x7fdb7010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.494+0000 7fdb749a1700 1 --2- 172.21.15.150:0/29723337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb701031e0 0x7fdb701035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.494+0000 7fdb749a1700 1 --2- 172.21.15.150:0/29723337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb700ffc20 0x7fdb70100030 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.494+0000 7fdb749a1700 1 -- 172.21.15.150:0/29723337 >> 172.21.15.150:0/29723337 conn(0x7fdb700fb8f0 msgr2=0x7fdb700fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:57.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.494+0000 7fdb749a1700 1 -- 172.21.15.150:0/29723337 shutdown_connections 2024-05-10T04:03:57.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.494+0000 7fdb749a1700 1 -- 172.21.15.150:0/29723337 wait complete. 2024-05-10T04:03:57.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.495+0000 7fdb749a1700 1 Processor -- start 2024-05-10T04:03:57.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.495+0000 7fdb749a1700 1 -- start start 2024-05-10T04:03:57.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.495+0000 7fdb749a1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb700ffc20 0x7fdb70197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:57.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.495+0000 7fdb749a1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb70100570 0x7fdb70198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:57.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.495+0000 7fdb749a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb701031e0 0x7fdb7019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:57.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.495+0000 7fdb749a1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7019fa00 con 0x7fdb70100570 2024-05-10T04:03:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.495+0000 7fdb749a1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7019fb70 con 0x7fdb700ffc20 2024-05-10T04:03:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.495+0000 7fdb749a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7019fd30 con 0x7fdb701031e0 2024-05-10T04:03:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.495+0000 7fdb6dd9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb70100570 0x7fdb70198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.495+0000 7fdb6dd9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb70100570 0x7fdb70198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38650/0 (socket says 172.21.15.150:38650) 2024-05-10T04:03:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.495+0000 7fdb6dd9b700 1 -- 172.21.15.150:0/1879029364 learned_addr learned my addr 172.21.15.150:0/1879029364 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.495+0000 7fdb6e59c700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb700ffc20 0x7fdb70197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb6dd9b700 1 -- 172.21.15.150:0/1879029364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb701031e0 msgr2=0x7fdb7019f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb6ed9d700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb701031e0 0x7fdb7019f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb6dd9b700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb701031e0 0x7fdb7019f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb6dd9b700 1 -- 172.21.15.150:0/1879029364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb700ffc20 msgr2=0x7fdb70197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb6dd9b700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb700ffc20 0x7fdb70197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb6dd9b700 1 -- 172.21.15.150:0/1879029364 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb640095f0 con 0x7fdb70100570 2024-05-10T04:03:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb6ed9d700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb701031e0 0x7fdb7019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:57.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb6e59c700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb700ffc20 0x7fdb70197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:57.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb6dd9b700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb70100570 0x7fdb70198460 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdb640095c0 tx=0x7fdb6400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:57.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb5f7fe700 1 -- 172.21.15.150:0/1879029364 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6401d070 con 0x7fdb70100570 2024-05-10T04:03:57.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb5f7fe700 1 -- 172.21.15.150:0/1879029364 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb64022470 con 0x7fdb70100570 2024-05-10T04:03:57.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb749a1700 1 -- 172.21.15.150:0/1879029364 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7019ffb0 con 0x7fdb70100570 2024-05-10T04:03:57.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb5f7fe700 1 -- 172.21.15.150:0/1879029364 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64017650 con 0x7fdb70100570 2024-05-10T04:03:57.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.496+0000 7fdb749a1700 1 -- 172.21.15.150:0/1879029364 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb701a0320 con 0x7fdb70100570 2024-05-10T04:03:57.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.497+0000 7fdb749a1700 1 -- 172.21.15.150:0/1879029364 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb70109030 con 0x7fdb70100570 2024-05-10T04:03:57.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.499+0000 7fdb5f7fe700 1 -- 172.21.15.150:0/1879029364 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fdb640225e0 con 0x7fdb70100570 2024-05-10T04:03:57.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.500+0000 7fdb5f7fe700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdb580a1260 0x7fdb580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:57.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.500+0000 7fdb6e59c700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdb580a1260 0x7fdb580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:57.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.500+0000 7fdb5f7fe700 1 -- 172.21.15.150:0/1879029364 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdb640ce520 con 0x7fdb70100570 2024-05-10T04:03:57.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.500+0000 7fdb6e59c700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdb580a1260 0x7fdb580a3770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdb60005fd0 tx=0x7fdb60005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:57.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.503+0000 7fdb5f7fe700 1 -- 172.21.15.150:0/1879029364 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdb64097900 con 0x7fdb70100570 2024-05-10T04:03:57.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.700+0000 7fdb749a1700 1 -- 172.21.15.150:0/1879029364 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb7010d560 con 0x7fdb70100570 2024-05-10T04:03:57.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.701+0000 7fdb5f7fe700 1 -- 172.21.15.150:0/1879029364 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb64022900 con 0x7fdb70100570 2024-05-10T04:03:57.703 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:03:57.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.703+0000 7fdb749a1700 1 -- 172.21.15.150:0/1879029364 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdb580a1260 msgr2=0x7fdb580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:57.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.703+0000 7fdb749a1700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdb580a1260 0x7fdb580a3770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdb60005fd0 tx=0x7fdb60005dd0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.703+0000 7fdb749a1700 1 -- 172.21.15.150:0/1879029364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb70100570 msgr2=0x7fdb70198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:57.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.703+0000 7fdb749a1700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb70100570 0x7fdb70198460 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdb640095c0 tx=0x7fdb6400f690 comp rx=0 tx=0).stop 2024-05-10T04:03:57.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.703+0000 7fdb749a1700 1 -- 172.21.15.150:0/1879029364 shutdown_connections 2024-05-10T04:03:57.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.703+0000 7fdb749a1700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdb580a1260 0x7fdb580a3770 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.703+0000 7fdb749a1700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb701031e0 0x7fdb7019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.703+0000 7fdb749a1700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb700ffc20 0x7fdb70197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.703+0000 7fdb749a1700 1 --2- 172.21.15.150:0/1879029364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb70100570 0x7fdb70198460 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.703+0000 7fdb749a1700 1 -- 172.21.15.150:0/1879029364 >> 172.21.15.150:0/1879029364 conn(0x7fdb700fb8f0 msgr2=0x7fdb70106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:57.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.703+0000 7fdb749a1700 1 -- 172.21.15.150:0/1879029364 shutdown_connections 2024-05-10T04:03:57.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.703+0000 7fdb749a1700 1 -- 172.21.15.150:0/1879029364 wait complete. 2024-05-10T04:03:57.718 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:03:57.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.852+0000 7f617bf51700 1 -- 172.21.15.150:0/4049877975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6174102030 msgr2=0x7f6174102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.852+0000 7f617bf51700 1 --2- 172.21.15.150:0/4049877975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6174102030 0x7f6174102460 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6170009990 tx=0x7f6170009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.852+0000 7f617bf51700 1 -- 172.21.15.150:0/4049877975 shutdown_connections 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.852+0000 7f617bf51700 1 --2- 172.21.15.150:0/4049877975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6174101680 0x7f6174108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.852+0000 7f617bf51700 1 --2- 172.21.15.150:0/4049877975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6174102030 0x7f6174102460 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.852+0000 7f617bf51700 1 --2- 172.21.15.150:0/4049877975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174100d30 0x7f6174101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.852+0000 7f617bf51700 1 -- 172.21.15.150:0/4049877975 >> 172.21.15.150:0/4049877975 conn(0x7f61740fb8f0 msgr2=0x7f61740fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.852+0000 7f617bf51700 1 -- 172.21.15.150:0/4049877975 shutdown_connections 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.852+0000 7f617bf51700 1 -- 172.21.15.150:0/4049877975 wait complete. 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.852+0000 7f617bf51700 1 Processor -- start 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.852+0000 7f617bf51700 1 -- start start 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f617bf51700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6174100d30 0x7f6174196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f617bf51700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6174101680 0x7f6174197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f617bf51700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174102030 0x7f617419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:57.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f617bf51700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617419e8f0 con 0x7f6174101680 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f617bf51700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617419ea60 con 0x7f6174100d30 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f617bf51700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617419ec20 con 0x7f6174102030 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f61794ec700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6174101680 0x7f6174197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f61794ec700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6174101680 0x7f6174197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38674/0 (socket says 172.21.15.150:38674) 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f61794ec700 1 -- 172.21.15.150:0/437943385 learned_addr learned my addr 172.21.15.150:0/437943385 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f617a4ee700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174102030 0x7f617419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f6179ced700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6174100d30 0x7f6174196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f61794ec700 1 -- 172.21.15.150:0/437943385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174102030 msgr2=0x7f617419e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f61794ec700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174102030 0x7f617419e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f61794ec700 1 -- 172.21.15.150:0/437943385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6174100d30 msgr2=0x7f6174196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f61794ec700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6174100d30 0x7f6174196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f61794ec700 1 -- 172.21.15.150:0/437943385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700095f0 con 0x7f6174101680 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f6179ced700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6174100d30 0x7f6174196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:57.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f617a4ee700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174102030 0x7f617419e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:57.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f61794ec700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6174101680 0x7f6174197350 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f616800d950 tx=0x7f616800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:57.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f6166ffd700 1 -- 172.21.15.150:0/437943385 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168009b40 con 0x7f6174101680 2024-05-10T04:03:57.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f617bf51700 1 -- 172.21.15.150:0/437943385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617419ef00 con 0x7f6174101680 2024-05-10T04:03:57.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f6166ffd700 1 -- 172.21.15.150:0/437943385 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6168009ca0 con 0x7f6174101680 2024-05-10T04:03:57.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f6166ffd700 1 -- 172.21.15.150:0/437943385 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800f620 con 0x7f6174101680 2024-05-10T04:03:57.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.853+0000 7f617bf51700 1 -- 172.21.15.150:0/437943385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617419f330 con 0x7f6174101680 2024-05-10T04:03:57.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.854+0000 7f617bf51700 1 -- 172.21.15.150:0/437943385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6174107e30 con 0x7f6174101680 2024-05-10T04:03:57.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.856+0000 7f6166ffd700 1 -- 172.21.15.150:0/437943385 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6168010460 con 0x7f6174101680 2024-05-10T04:03:57.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.857+0000 7f6166ffd700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f61600a1260 0x7f61600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:57.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.857+0000 7f6179ced700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f61600a1260 0x7f61600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:57.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.857+0000 7f6166ffd700 1 -- 172.21.15.150:0/437943385 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f61680cd1a0 con 0x7f6174101680 2024-05-10T04:03:57.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.857+0000 7f6179ced700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f61600a1260 0x7f61600a3770 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6170000c00 tx=0x7f6170009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:57.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:57.861+0000 7f6166ffd700 1 -- 172.21.15.150:0/437943385 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f61680964c0 con 0x7f6174101680 2024-05-10T04:03:58.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.056+0000 7f617bf51700 1 -- 172.21.15.150:0/437943385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f617410c2a0 con 0x7f6174101680 2024-05-10T04:03:58.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.057+0000 7f6166ffd700 1 -- 172.21.15.150:0/437943385 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6168020020 con 0x7f6174101680 2024-05-10T04:03:58.059 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:03:58.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.059+0000 7f617bf51700 1 -- 172.21.15.150:0/437943385 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f61600a1260 msgr2=0x7f61600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.059+0000 7f617bf51700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f61600a1260 0x7f61600a3770 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6170000c00 tx=0x7f6170009510 comp rx=0 tx=0).stop 2024-05-10T04:03:58.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.059+0000 7f617bf51700 1 -- 172.21.15.150:0/437943385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6174101680 msgr2=0x7f6174197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.059+0000 7f617bf51700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6174101680 0x7f6174197350 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f616800d950 tx=0x7f616800dd10 comp rx=0 tx=0).stop 2024-05-10T04:03:58.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.059+0000 7f617bf51700 1 -- 172.21.15.150:0/437943385 shutdown_connections 2024-05-10T04:03:58.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.059+0000 7f617bf51700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f61600a1260 0x7f61600a3770 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.059+0000 7f617bf51700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174102030 0x7f617419e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.059+0000 7f617bf51700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6174100d30 0x7f6174196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.059+0000 7f617bf51700 1 --2- 172.21.15.150:0/437943385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6174101680 0x7f6174197350 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.059+0000 7f617bf51700 1 -- 172.21.15.150:0/437943385 >> 172.21.15.150:0/437943385 conn(0x7f61740fb8f0 msgr2=0x7f6174104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:58.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.059+0000 7f617bf51700 1 -- 172.21.15.150:0/437943385 shutdown_connections 2024-05-10T04:03:58.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.060+0000 7f617bf51700 1 -- 172.21.15.150:0/437943385 wait complete. 2024-05-10T04:03:58.075 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:03:58.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.207+0000 7ffa7314b700 1 -- 172.21.15.150:0/3374711592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa6c101700 msgr2=0x7ffa6c109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.207+0000 7ffa7314b700 1 --2- 172.21.15.150:0/3374711592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa6c101700 0x7ffa6c109010 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400b0a0 tx=0x7ffa6400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.207+0000 7ffa7314b700 1 -- 172.21.15.150:0/3374711592 shutdown_connections 2024-05-10T04:03:58.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.207+0000 7ffa7314b700 1 --2- 172.21.15.150:0/3374711592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa6c101700 0x7ffa6c109010 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.207+0000 7ffa7314b700 1 --2- 172.21.15.150:0/3374711592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa6c1020a0 0x7ffa6c1024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.207+0000 7ffa7314b700 1 --2- 172.21.15.150:0/3374711592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa6c100db0 0x7ffa6c1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.207+0000 7ffa7314b700 1 -- 172.21.15.150:0/3374711592 >> 172.21.15.150:0/3374711592 conn(0x7ffa6c0fb8f0 msgr2=0x7ffa6c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:58.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.207+0000 7ffa7314b700 1 -- 172.21.15.150:0/3374711592 shutdown_connections 2024-05-10T04:03:58.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.207+0000 7ffa7314b700 1 -- 172.21.15.150:0/3374711592 wait complete. 2024-05-10T04:03:58.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.207+0000 7ffa7314b700 1 Processor -- start 2024-05-10T04:03:58.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.207+0000 7ffa7314b700 1 -- start start 2024-05-10T04:03:58.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa7314b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa6c100db0 0x7ffa6c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa7314b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa6c101700 0x7ffa6c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa7314b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa6c1020a0 0x7ffa6c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa7314b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c19faf0 con 0x7ffa6c1020a0 2024-05-10T04:03:58.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa7314b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c19fc60 con 0x7ffa6c100db0 2024-05-10T04:03:58.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa7314b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c19fe20 con 0x7ffa6c101700 2024-05-10T04:03:58.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa716e8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa6c1020a0 0x7ffa6c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa716e8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa6c1020a0 0x7ffa6c19f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38690/0 (socket says 172.21.15.150:38690) 2024-05-10T04:03:58.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa716e8700 1 -- 172.21.15.150:0/62695132 learned_addr learned my addr 172.21.15.150:0/62695132 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:58.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa70ee7700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa6c100db0 0x7ffa6c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa6bfff700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa6c101700 0x7ffa6c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa716e8700 1 -- 172.21.15.150:0/62695132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa6c101700 msgr2=0x7ffa6c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa716e8700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa6c101700 0x7ffa6c198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa716e8700 1 -- 172.21.15.150:0/62695132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa6c100db0 msgr2=0x7ffa6c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa716e8700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa6c100db0 0x7ffa6c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa716e8700 1 -- 172.21.15.150:0/62695132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa64009d00 con 0x7ffa6c1020a0 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa70ee7700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa6c100db0 0x7ffa6c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa6bfff700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa6c101700 0x7ffa6c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa716e8700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa6c1020a0 0x7ffa6c19f4e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006070 tx=0x7ffa64004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa69ffb700 1 -- 172.21.15.150:0/62695132 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6400e040 con 0x7ffa6c1020a0 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa69ffb700 1 -- 172.21.15.150:0/62695132 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa6400cec0 con 0x7ffa6c1020a0 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa7314b700 1 -- 172.21.15.150:0/62695132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6c1a00a0 con 0x7ffa6c1020a0 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.208+0000 7ffa69ffb700 1 -- 172.21.15.150:0/62695132 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64004020 con 0x7ffa6c1020a0 2024-05-10T04:03:58.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.209+0000 7ffa7314b700 1 -- 172.21.15.150:0/62695132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6c1a0520 con 0x7ffa6c1020a0 2024-05-10T04:03:58.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.209+0000 7ffa7314b700 1 -- 172.21.15.150:0/62695132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6c0678b0 con 0x7ffa6c1020a0 2024-05-10T04:03:58.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.211+0000 7ffa69ffb700 1 -- 172.21.15.150:0/62695132 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ffa6401d690 con 0x7ffa6c1020a0 2024-05-10T04:03:58.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.211+0000 7ffa69ffb700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffa5c0a11b0 0x7ffa5c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.212+0000 7ffa69ffb700 1 -- 172.21.15.150:0/62695132 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffa640ced80 con 0x7ffa6c1020a0 2024-05-10T04:03:58.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.212+0000 7ffa70ee7700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffa5c0a11b0 0x7ffa5c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.212+0000 7ffa70ee7700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffa5c0a11b0 0x7ffa5c0a36c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffa60005950 tx=0x7ffa60005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:58.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.215+0000 7ffa69ffb700 1 -- 172.21.15.150:0/62695132 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffa64098130 con 0x7ffa6c1020a0 2024-05-10T04:03:58.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.459+0000 7ffa7314b700 1 -- 172.21.15.150:0/62695132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa6c10c3b0 con 0x7ffa6c1020a0 2024-05-10T04:03:58.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.460+0000 7ffa69ffb700 1 -- 172.21.15.150:0/62695132 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10886 (secure 0 0 0) 0x7ffa6409b750 con 0x7ffa6c1020a0 2024-05-10T04:03:58.462 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:58.462 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:02:57.183845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:03:58.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:58 smithi150 ceph-mon[42477]: pgmap v271: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:58.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:58 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/326791952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:03:58.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:58 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1879029364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:03:58.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:58 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/437943385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:03:58.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.462+0000 7ffa7314b700 1 -- 172.21.15.150:0/62695132 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffa5c0a11b0 msgr2=0x7ffa5c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.462+0000 7ffa7314b700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffa5c0a11b0 0x7ffa5c0a36c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffa60005950 tx=0x7ffa60005f50 comp rx=0 tx=0).stop 2024-05-10T04:03:58.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.462+0000 7ffa7314b700 1 -- 172.21.15.150:0/62695132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa6c1020a0 msgr2=0x7ffa6c19f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.462+0000 7ffa7314b700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa6c1020a0 0x7ffa6c19f4e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006070 tx=0x7ffa64004d30 comp rx=0 tx=0).stop 2024-05-10T04:03:58.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.462+0000 7ffa7314b700 1 -- 172.21.15.150:0/62695132 shutdown_connections 2024-05-10T04:03:58.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.462+0000 7ffa7314b700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffa5c0a11b0 0x7ffa5c0a36c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.462+0000 7ffa7314b700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa6c1020a0 0x7ffa6c19f4e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.462+0000 7ffa7314b700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa6c100db0 0x7ffa6c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.462+0000 7ffa7314b700 1 --2- 172.21.15.150:0/62695132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa6c101700 0x7ffa6c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.462+0000 7ffa7314b700 1 -- 172.21.15.150:0/62695132 >> 172.21.15.150:0/62695132 conn(0x7ffa6c0fb8f0 msgr2=0x7ffa6c0fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:58.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.462+0000 7ffa7314b700 1 -- 172.21.15.150:0/62695132 shutdown_connections 2024-05-10T04:03:58.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.462+0000 7ffa7314b700 1 -- 172.21.15.150:0/62695132 wait complete. 2024-05-10T04:03:58.466 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 24 2024-05-10T04:03:58.478 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:03:58.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:58 smithi181 ceph-mon[46602]: pgmap v271: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:58.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:58 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/326791952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:03:58.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:58 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1879029364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:03:58.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:58 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/437943385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:03:58.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:58 smithi161 ceph-mon[46918]: pgmap v271: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:58.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:58 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/326791952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:03:58.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:58 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1879029364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:03:58.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:58 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/437943385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:03:58.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.607+0000 7fbdb4d11700 1 -- 172.21.15.150:0/300780971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdb0101be0 msgr2=0x7fbdb0101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.607+0000 7fbdb4d11700 1 --2- 172.21.15.150:0/300780971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdb0101be0 0x7fbdb0101ff0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009990 tx=0x7fbda4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.608+0000 7fbdb4d11700 1 -- 172.21.15.150:0/300780971 shutdown_connections 2024-05-10T04:03:58.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.608+0000 7fbdb4d11700 1 --2- 172.21.15.150:0/300780971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdb0102580 0x7fbdb0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.608+0000 7fbdb4d11700 1 --2- 172.21.15.150:0/300780971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdb0100ac0 0x7fbdb0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.608+0000 7fbdb4d11700 1 --2- 172.21.15.150:0/300780971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdb0101be0 0x7fbdb0101ff0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.608+0000 7fbdb4d11700 1 -- 172.21.15.150:0/300780971 >> 172.21.15.150:0/300780971 conn(0x7fbdb00fc0d0 msgr2=0x7fbdb00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:58.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.608+0000 7fbdb4d11700 1 -- 172.21.15.150:0/300780971 shutdown_connections 2024-05-10T04:03:58.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.608+0000 7fbdb4d11700 1 -- 172.21.15.150:0/300780971 wait complete. 2024-05-10T04:03:58.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.608+0000 7fbdb4d11700 1 Processor -- start 2024-05-10T04:03:58.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.608+0000 7fbdb4d11700 1 -- start start 2024-05-10T04:03:58.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdb4d11700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdb0100ac0 0x7fbdb0068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdb4d11700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdb0101be0 0x7fbdb019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdb4d11700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdb0102580 0x7fbdb019f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdb4d11700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb019f9f0 con 0x7fbdb0102580 2024-05-10T04:03:58.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdb4d11700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb019fb60 con 0x7fbdb0100ac0 2024-05-10T04:03:58.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdb4d11700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb019fd20 con 0x7fbdb0101be0 2024-05-10T04:03:58.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdaed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdb0102580 0x7fbdb019f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdaed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdb0102580 0x7fbdb019f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38702/0 (socket says 172.21.15.150:38702) 2024-05-10T04:03:58.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdaed9d700 1 -- 172.21.15.150:0/3655322212 learned_addr learned my addr 172.21.15.150:0/3655322212 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:58.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdae59c700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdb0100ac0 0x7fbdb0068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdadd9b700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdb0101be0 0x7fbdb019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdaed9d700 1 -- 172.21.15.150:0/3655322212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdb0101be0 msgr2=0x7fbdb019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdaed9d700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdb0101be0 0x7fbdb019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdaed9d700 1 -- 172.21.15.150:0/3655322212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdb0100ac0 msgr2=0x7fbdb0068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdaed9d700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdb0100ac0 0x7fbdb0068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdaed9d700 1 -- 172.21.15.150:0/3655322212 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda40095f0 con 0x7fbdb0102580 2024-05-10T04:03:58.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdae59c700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdb0100ac0 0x7fbdb0068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:58.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdaed9d700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdb0102580 0x7fbdb019f2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbda8007ee0 tx=0x7fbda800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:58.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbd9f7fe700 1 -- 172.21.15.150:0/3655322212 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8017070 con 0x7fbdb0102580 2024-05-10T04:03:58.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbd9f7fe700 1 -- 172.21.15.150:0/3655322212 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbda800f040 con 0x7fbdb0102580 2024-05-10T04:03:58.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbd9f7fe700 1 -- 172.21.15.150:0/3655322212 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8005740 con 0x7fbdb0102580 2024-05-10T04:03:58.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdb4d11700 1 -- 172.21.15.150:0/3655322212 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb019ffa0 con 0x7fbdb0102580 2024-05-10T04:03:58.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.609+0000 7fbdb4d11700 1 -- 172.21.15.150:0/3655322212 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb01a0400 con 0x7fbdb0102580 2024-05-10T04:03:58.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.610+0000 7fbdb4d11700 1 -- 172.21.15.150:0/3655322212 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb0107cd0 con 0x7fbdb0102580 2024-05-10T04:03:58.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.612+0000 7fbd9f7fe700 1 -- 172.21.15.150:0/3655322212 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fbda80058a0 con 0x7fbdb0102580 2024-05-10T04:03:58.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.612+0000 7fbd9f7fe700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbd980a1230 0x7fbd980a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.612+0000 7fbdae59c700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbd980a1230 0x7fbd980a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.612+0000 7fbd9f7fe700 1 -- 172.21.15.150:0/3655322212 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbda80ccb80 con 0x7fbdb0102580 2024-05-10T04:03:58.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.613+0000 7fbdae59c700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbd980a1230 0x7fbd980a3740 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0101ae0 tx=0x7fbda000a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:58.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.616+0000 7fbd9f7fe700 1 -- 172.21.15.150:0/3655322212 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbda8095ea0 con 0x7fbdb0102580 2024-05-10T04:03:58.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.812+0000 7fbdb4d11700 1 -- 172.21.15.150:0/3655322212 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdb00694d0 con 0x7fbdb0102580 2024-05-10T04:03:58.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.814+0000 7fbd9f7fe700 1 -- 172.21.15.150:0/3655322212 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24554 (secure 0 0 0) 0x7fbda80994c0 con 0x7fbdb0102580 2024-05-10T04:03:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:03:58.817 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":87,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:03:30.185138+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:03:58.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.816+0000 7fbdb4d11700 1 -- 172.21.15.150:0/3655322212 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbd980a1230 msgr2=0x7fbd980a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.816+0000 7fbdb4d11700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbd980a1230 0x7fbd980a3740 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0101ae0 tx=0x7fbda000a680 comp rx=0 tx=0).stop 2024-05-10T04:03:58.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.816+0000 7fbdb4d11700 1 -- 172.21.15.150:0/3655322212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdb0102580 msgr2=0x7fbdb019f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.816+0000 7fbdb4d11700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdb0102580 0x7fbdb019f2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbda8007ee0 tx=0x7fbda800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.816+0000 7fbdb4d11700 1 -- 172.21.15.150:0/3655322212 shutdown_connections 2024-05-10T04:03:58.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.816+0000 7fbdb4d11700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbd980a1230 0x7fbd980a3740 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.816+0000 7fbdb4d11700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdb0102580 0x7fbdb019f2f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.816+0000 7fbdb4d11700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdb0100ac0 0x7fbdb0068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.816+0000 7fbdb4d11700 1 --2- 172.21.15.150:0/3655322212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdb0101be0 0x7fbdb019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.816+0000 7fbdb4d11700 1 -- 172.21.15.150:0/3655322212 >> 172.21.15.150:0/3655322212 conn(0x7fbdb00fc0d0 msgr2=0x7fbdb0104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:58.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.816+0000 7fbdb4d11700 1 -- 172.21.15.150:0/3655322212 shutdown_connections 2024-05-10T04:03:58.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.816+0000 7fbdb4d11700 1 -- 172.21.15.150:0/3655322212 wait complete. 2024-05-10T04:03:58.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-10T04:02:31.605102+0000', 'modified': '2024-05-10T04:02:57.183845+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24557, 'mds_1': 24556, 'mds_2': 24535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24484': {'gid': 24484, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6833/2987418700', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6832', 'nonce': 2987418700}, {'type': 'v1', 'addr': '172.21.15.181:6833', 'nonce': 2987418700}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24503': {'gid': 24503, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/1403383813', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 1403383813}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 1403383813}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6835/501599730', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 501599730}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 501599730}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.181:6837/1798187480', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 1798187480}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1798187480}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24556': {'gid': 24556, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.181:6839/1415332890', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1415332890}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1415332890}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6839/267491715', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 267491715}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 267491715}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T04:03:58.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-10T04:03:58.833 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T04:03:58.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.965+0000 7fb17a774700 1 -- 172.21.15.150:0/1902781327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb174101be0 msgr2=0x7fb174101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.965+0000 7fb17a774700 1 --2- 172.21.15.150:0/1902781327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb174101be0 0x7fb174101ff0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb168009990 tx=0x7fb168009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.965+0000 7fb17a774700 1 -- 172.21.15.150:0/1902781327 shutdown_connections 2024-05-10T04:03:58.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.965+0000 7fb17a774700 1 --2- 172.21.15.150:0/1902781327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb174102580 0x7fb174108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.965+0000 7fb17a774700 1 --2- 172.21.15.150:0/1902781327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb174100ac0 0x7fb174100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.965+0000 7fb17a774700 1 --2- 172.21.15.150:0/1902781327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb174101be0 0x7fb174101ff0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.965+0000 7fb17a774700 1 -- 172.21.15.150:0/1902781327 >> 172.21.15.150:0/1902781327 conn(0x7fb1740fc0d0 msgr2=0x7fb1740fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:58.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.965+0000 7fb17a774700 1 -- 172.21.15.150:0/1902781327 shutdown_connections 2024-05-10T04:03:58.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.965+0000 7fb17a774700 1 -- 172.21.15.150:0/1902781327 wait complete. 2024-05-10T04:03:58.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb17a774700 1 Processor -- start 2024-05-10T04:03:58.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb17a774700 1 -- start start 2024-05-10T04:03:58.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb17a774700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb174100ac0 0x7fb17419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb17a774700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb174101be0 0x7fb17419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb17a774700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb174102580 0x7fb1741a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb17a774700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1741a1b00 con 0x7fb174102580 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb17a774700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1741a1c70 con 0x7fb174100ac0 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb17a774700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1741a1e30 con 0x7fb174101be0 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb178d11700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb174102580 0x7fb1741a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb173fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb174100ac0 0x7fb17419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb178d11700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb174102580 0x7fb1741a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38714/0 (socket says 172.21.15.150:38714) 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb178d11700 1 -- 172.21.15.150:0/3981606446 learned_addr learned my addr 172.21.15.150:0/3981606446 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb1737fe700 1 --2- 172.21.15.150:0/3981606446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb174101be0 0x7fb17419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb178d11700 1 -- 172.21.15.150:0/3981606446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb174101be0 msgr2=0x7fb17419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb178d11700 1 --2- 172.21.15.150:0/3981606446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb174101be0 0x7fb17419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb178d11700 1 -- 172.21.15.150:0/3981606446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb174100ac0 msgr2=0x7fb17419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb178d11700 1 --2- 172.21.15.150:0/3981606446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb174100ac0 0x7fb17419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.966+0000 7fb178d11700 1 -- 172.21.15.150:0/3981606446 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680095f0 con 0x7fb174102580 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.967+0000 7fb178d11700 1 --2- 172.21.15.150:0/3981606446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb174102580 0x7fb1741a14f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00ef30 tx=0x7fb16c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.967+0000 7fb1717fa700 1 -- 172.21.15.150:0/3981606446 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c018070 con 0x7fb174102580 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.967+0000 7fb17a774700 1 -- 172.21.15.150:0/3981606446 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1741a20b0 con 0x7fb174102580 2024-05-10T04:03:58.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.967+0000 7fb1717fa700 1 -- 172.21.15.150:0/3981606446 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb16c00f040 con 0x7fb174102580 2024-05-10T04:03:58.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.967+0000 7fb1717fa700 1 -- 172.21.15.150:0/3981606446 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00cd00 con 0x7fb174102580 2024-05-10T04:03:58.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.967+0000 7fb17a774700 1 -- 172.21.15.150:0/3981606446 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1741a25c0 con 0x7fb174102580 2024-05-10T04:03:58.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.969+0000 7fb1717fa700 1 -- 172.21.15.150:0/3981606446 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb16c004000 con 0x7fb174102580 2024-05-10T04:03:58.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.970+0000 7fb1717fa700 1 --2- 172.21.15.150:0/3981606446 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb15c0a1260 0x7fb15c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.970+0000 7fb173fff700 1 --2- 172.21.15.150:0/3981606446 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb15c0a1260 0x7fb15c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.970+0000 7fb1717fa700 1 -- 172.21.15.150:0/3981606446 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb16c0cc860 con 0x7fb174102580 2024-05-10T04:03:58.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.970+0000 7fb173fff700 1 --2- 172.21.15.150:0/3981606446 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb15c0a1260 0x7fb15c0a3770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb16400b3c0 tx=0x7fb16400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:58.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.970+0000 7fb18aab8b80 1 Processor -- start 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.970+0000 7fb18aab8b80 1 -- start start 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb18aab8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 0x555eb258daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb18aab8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258e520 0x555eb258e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb18aab8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 0x555eb2595770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb18aab8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb24b4100 con 0x555eb258d650 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb18aab8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb24b1ac0 con 0x555eb258ee70 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb18aab8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb24ad3b0 con 0x555eb258e520 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb173fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 0x555eb258daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb173fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 0x555eb258daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38728/0 (socket says 172.21.15.150:38728) 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb173fff700 1 -- 172.21.15.150:0/3773838025 learned_addr learned my addr 172.21.15.150:0/3773838025 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb1737fe700 1 --2- 172.21.15.150:0/3773838025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258e520 0x555eb258e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb178d11700 1 --2- 172.21.15.150:0/3773838025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 0x555eb2595770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb173fff700 1 -- 172.21.15.150:0/3773838025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258e520 msgr2=0x555eb258e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:58.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb173fff700 1 --2- 172.21.15.150:0/3773838025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258e520 0x555eb258e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb173fff700 1 -- 172.21.15.150:0/3773838025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 msgr2=0x555eb2595770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb173fff700 1 --2- 172.21.15.150:0/3773838025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 0x555eb2595770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb173fff700 1 -- 172.21.15.150:0/3773838025 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eb232e360 con 0x555eb258d650 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb1737fe700 1 --2- 172.21.15.150:0/3773838025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258e520 0x555eb258e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb173fff700 1 --2- 172.21.15.150:0/3773838025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 0x555eb258daa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb164009d30 tx=0x7fb164014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5075ca8208ae68b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.971+0000 7fb15a7fc700 1 -- 172.21.15.150:0/3773838025 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164010040 con 0x555eb258d650 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.972+0000 7fb15a7fc700 1 -- 172.21.15.150:0/3773838025 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb164004580 con 0x555eb258d650 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.972+0000 7fb15a7fc700 1 -- 172.21.15.150:0/3773838025 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164018b80 con 0x555eb258d650 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.972+0000 7fb18aab8b80 1 -- 172.21.15.150:0/3773838025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 msgr2=0x555eb258daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.972+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/3773838025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 0x555eb258daa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb164009d30 tx=0x7fb164014000 comp rx=0 tx=0).stop 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.972+0000 7fb18aab8b80 1 -- 172.21.15.150:0/3773838025 shutdown_connections 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.972+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/3773838025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 0x555eb2595770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.972+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/3773838025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 0x555eb258daa0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.972+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/3773838025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258e520 0x555eb258e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.972+0000 7fb18aab8b80 1 -- 172.21.15.150:0/3773838025 >> 172.21.15.150:0/3773838025 conn(0x555eb24efa10 msgr2=0x555eb2587f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.972+0000 7fb18aab8b80 1 -- 172.21.15.150:0/3773838025 shutdown_connections 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.972+0000 7fb18aab8b80 1 -- 172.21.15.150:0/3773838025 wait complete. 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.973+0000 7fb18aab8b80 1 -- start start 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.973+0000 7fb18aab8b80 1 Processor -- start 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.973+0000 7fb18aab8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258d650 0x555eb2597660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.973+0000 7fb18aab8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 0x555eb2597ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.973+0000 7fb18aab8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 0x555eb259eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.973+0000 7fb18aab8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb2449510 con 0x555eb258e520 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.973+0000 7fb18aab8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb24491b0 con 0x555eb258ee70 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.973+0000 7fb18aab8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb2330650 con 0x555eb258d650 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.973+0000 7fb18aab8b80 10 client.0 _ll_register_callbacks cb 0x555eb2452ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.973+0000 7fb1737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 0x555eb2597ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.973+0000 7fb1737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 0x555eb2597ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38732/0 (socket says 172.21.15.150:38732) 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.973+0000 7fb1737fe700 1 -- 172.21.15.150:0/3751976669 learned_addr learned my addr 172.21.15.150:0/3751976669 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb178d11700 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 0x555eb259eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb173fff700 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258d650 0x555eb2597660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb1737fe700 1 -- 172.21.15.150:0/3751976669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258d650 msgr2=0x555eb2597660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb1737fe700 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258d650 0x555eb2597660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb1737fe700 1 -- 172.21.15.150:0/3751976669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 msgr2=0x555eb259eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb1737fe700 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 0x555eb259eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb1737fe700 1 -- 172.21.15.150:0/3751976669 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb164009a10 con 0x555eb258e520 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb173fff700 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258d650 0x555eb2597660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb1737fe700 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 0x555eb2597ba0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb168015000 tx=0x7fb16800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a27203e74d1a6d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb1537fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb1537fe700 1 -- 172.21.15.150:0/3751976669 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168004700 con 0x555eb258e520 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb1537fe700 1 -- 172.21.15.150:0/3751976669 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb1680056b0 con 0x555eb258e520 2024-05-10T04:03:58.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb18aab8b80 1 -- 172.21.15.150:0/3751976669 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555eb2406220 con 0x555eb258e520 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb1537fe700 1 -- 172.21.15.150:0/3751976669 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb168004860 con 0x555eb258e520 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb18aab8b80 10 client.15432 fetch_fsmap learned FSMap version 24 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb18aab8b80 1 -- 172.21.15.150:0/3751976669 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555eb22dab10 con 0x555eb258e520 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.974+0000 7fb1537fe700 1 -- 172.21.15.150:0/3751976669 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555eb2406220 con 0x555eb258e520 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.975+0000 7fb1537fe700 1 -- 172.21.15.150:0/3751976669 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb16800fae0 con 0x555eb258e520 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.975+0000 7fb18aab8b80 10 client.15432 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.975+0000 7fb18aab8b80 10 client.15432 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.975+0000 7fb18aab8b80 20 client.15432 populate_metadata read hostname 'smithi150' 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.975+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x555eb25a2740 0x555eb25a7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.975+0000 7fb18aab8b80 4 client.15432 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.975+0000 7fb18aab8b80 1 -- 172.21.15.150:0/3751976669 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555eb255c730 con 0x555eb25a2740 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.975+0000 7fb173fff700 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x555eb25a2740 0x555eb25a7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.976+0000 7fb1537fe700 10 client.15432 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.976+0000 7fb173fff700 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x555eb25a2740 0x555eb25a7a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=191126d505d493ef server_cookie=566359ebcfbd216e in_seq=0 out_seq=0 2024-05-10T04:03:58.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.976+0000 7fb1537fe700 1 -- 172.21.15.150:0/3751976669 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555eb255c730 con 0x555eb25a2740 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.976+0000 7fb1537fe700 10 client.15432 handle_command_reply: tid=0 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.976+0000 7fb18aab8b80 1 client.15432 shutdown 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.976+0000 7fb18aab8b80 1 -- 172.21.15.150:0/3751976669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 msgr2=0x555eb2597ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.976+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 0x555eb2597ba0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb168015000 tx=0x7fb16800bfa0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.977+0000 7fb18aab8b80 1 -- 172.21.15.150:0/3751976669 shutdown_connections 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.977+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 0x555eb259eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.977+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x555eb25a2740 0x555eb25a7a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.977+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258d650 0x555eb2597660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.977+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/3751976669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 0x555eb2597ba0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.977+0000 7fb1537fe700 0 client.15432 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.977+0000 7fb18aab8b80 1 -- 172.21.15.150:0/3751976669 >> 172.21.15.150:0/3751976669 conn(0x555eb24efa10 msgr2=0x555eb2587f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.977+0000 7fb18aab8b80 1 -- 172.21.15.150:0/3751976669 shutdown_connections 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.977+0000 7fb18aab8b80 1 -- 172.21.15.150:0/3751976669 wait complete. 2024-05-10T04:03:58.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.977+0000 7fb18aab8b80 20 client.15432 trim_cache size 0 max 16384 2024-05-10T04:03:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb18aab8b80 1 Processor -- start 2024-05-10T04:03:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb18aab8b80 1 -- start start 2024-05-10T04:03:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb18aab8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 0x555eb259cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb18aab8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258e520 0x555eb259cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb18aab8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258ee70 0x555eb25959c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb18aab8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb244c5a0 con 0x555eb258d650 2024-05-10T04:03:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb18aab8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb2584ce0 con 0x555eb258e520 2024-05-10T04:03:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb18aab8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb259fc00 con 0x555eb258ee70 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb173fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 0x555eb259cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb173fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 0x555eb259cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38740/0 (socket says 172.21.15.150:38740) 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb173fff700 1 -- 172.21.15.150:0/1970537968 learned_addr learned my addr 172.21.15.150:0/1970537968 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb178d11700 1 --2- 172.21.15.150:0/1970537968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258ee70 0x555eb25959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb1737fe700 1 --2- 172.21.15.150:0/1970537968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258e520 0x555eb259cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb173fff700 1 -- 172.21.15.150:0/1970537968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258ee70 msgr2=0x555eb25959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb173fff700 1 --2- 172.21.15.150:0/1970537968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258ee70 0x555eb25959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb173fff700 1 -- 172.21.15.150:0/1970537968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258e520 msgr2=0x555eb259cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb173fff700 1 --2- 172.21.15.150:0/1970537968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258e520 0x555eb259cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb173fff700 1 -- 172.21.15.150:0/1970537968 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eb2473510 con 0x555eb258d650 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb178d11700 1 --2- 172.21.15.150:0/1970537968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258ee70 0x555eb25959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.995+0000 7fb1737fe700 1 --2- 172.21.15.150:0/1970537968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258e520 0x555eb259cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.996+0000 7fb173fff700 1 --2- 172.21.15.150:0/1970537968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 0x555eb259cab0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x555eb258d330 tx=0x7fb164005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f63de978e03381 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.996+0000 7fb15affd700 1 -- 172.21.15.150:0/1970537968 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164038030 con 0x555eb258d650 2024-05-10T04:03:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.996+0000 7fb15affd700 1 -- 172.21.15.150:0/1970537968 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb16400be90 con 0x555eb258d650 2024-05-10T04:03:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.996+0000 7fb15affd700 1 -- 172.21.15.150:0/1970537968 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164034070 con 0x555eb258d650 2024-05-10T04:03:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 -- 172.21.15.150:0/1970537968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 msgr2=0x555eb259cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/1970537968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 0x555eb259cab0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x555eb258d330 tx=0x7fb164005c20 comp rx=0 tx=0).stop 2024-05-10T04:03:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 -- 172.21.15.150:0/1970537968 shutdown_connections 2024-05-10T04:03:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/1970537968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258ee70 0x555eb25959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/1970537968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258d650 0x555eb259cab0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/1970537968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258e520 0x555eb259cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 -- 172.21.15.150:0/1970537968 >> 172.21.15.150:0/1970537968 conn(0x555eb24efa10 msgr2=0x555eb2591750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 -- 172.21.15.150:0/1970537968 shutdown_connections 2024-05-10T04:03:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 -- 172.21.15.150:0/1970537968 wait complete. 2024-05-10T04:03:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 -- start start 2024-05-10T04:03:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 Processor -- start 2024-05-10T04:03:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258d650 0x555eb259c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 0x555eb259cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 0x555eb259b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb259b780 con 0x555eb258e520 2024-05-10T04:03:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb259b8f0 con 0x555eb258ee70 2024-05-10T04:03:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb259bab0 con 0x555eb258d650 2024-05-10T04:03:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.997+0000 7fb18aab8b80 10 client.0 _ll_register_callbacks cb 0x555eb2452ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:03:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 0x555eb259cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 0x555eb259cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38748/0 (socket says 172.21.15.150:38748) 2024-05-10T04:03:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1737fe700 1 -- 172.21.15.150:0/4227124404 learned_addr learned my addr 172.21.15.150:0/4227124404 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb173fff700 1 --2- 172.21.15.150:0/4227124404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258d650 0x555eb259c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb178d11700 1 --2- 172.21.15.150:0/4227124404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 0x555eb259b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1737fe700 1 -- 172.21.15.150:0/4227124404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258d650 msgr2=0x555eb259c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1737fe700 1 --2- 172.21.15.150:0/4227124404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258d650 0x555eb259c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1737fe700 1 -- 172.21.15.150:0/4227124404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 msgr2=0x555eb259b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1737fe700 1 --2- 172.21.15.150:0/4227124404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 0x555eb259b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1737fe700 1 -- 172.21.15.150:0/4227124404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555eb232e360 con 0x555eb258e520 2024-05-10T04:03:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1737fe700 1 --2- 172.21.15.150:0/4227124404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 0x555eb259cf10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb16800ff40 tx=0x7fb168005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d55c92120ddf7d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1537fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:03:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1537fe700 1 -- 172.21.15.150:0/4227124404 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168023c50 con 0x555eb258e520 2024-05-10T04:03:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1537fe700 1 -- 172.21.15.150:0/4227124404 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb168000e40 con 0x555eb258e520 2024-05-10T04:03:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb18aab8b80 1 -- 172.21.15.150:0/4227124404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb164004580 con 0x555eb258e520 2024-05-10T04:03:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1537fe700 1 -- 172.21.15.150:0/4227124404 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb16800b360 con 0x555eb258e520 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb18aab8b80 10 client.15438 fetch_fsmap learned FSMap version 24 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb18aab8b80 1 -- 172.21.15.150:0/4227124404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555eb2332c80 con 0x555eb258e520 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.998+0000 7fb1537fe700 1 -- 172.21.15.150:0/4227124404 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb164004580 con 0x555eb258e520 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.999+0000 7fb1537fe700 1 -- 172.21.15.150:0/4227124404 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb168004590 con 0x555eb258e520 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.999+0000 7fb18aab8b80 10 client.15438 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.999+0000 7fb18aab8b80 10 client.15438 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.999+0000 7fb18aab8b80 20 client.15438 populate_metadata read hostname 'smithi150' 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.999+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/4227124404 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x555eb25fbd70 0x555eb25fa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.999+0000 7fb18aab8b80 4 client.15438 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.999+0000 7fb18aab8b80 1 -- 172.21.15.150:0/4227124404 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555eb24a5e70 con 0x555eb25fbd70 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.999+0000 7fb173fff700 1 --2- 172.21.15.150:0/4227124404 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x555eb25fbd70 0x555eb25fa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.999+0000 7fb1537fe700 10 client.15438 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:58.999+0000 7fb173fff700 1 --2- 172.21.15.150:0/4227124404 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x555eb25fbd70 0x555eb25fa380 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c4e024819b82125 server_cookie=c50b9cdd66a3c97d in_seq=0 out_seq=0 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb1537fe700 1 -- 172.21.15.150:0/4227124404 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x555eb24a5e70 con 0x555eb25fbd70 2024-05-10T04:03:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb1537fe700 10 client.15438 handle_command_reply: tid=0 2024-05-10T04:03:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb18aab8b80 1 client.15438 shutdown 2024-05-10T04:03:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb18aab8b80 1 -- 172.21.15.150:0/4227124404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 msgr2=0x555eb259cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/4227124404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 0x555eb259cf10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb16800ff40 tx=0x7fb168005760 comp rx=0 tx=0).stop 2024-05-10T04:03:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb18aab8b80 1 -- 172.21.15.150:0/4227124404 shutdown_connections 2024-05-10T04:03:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/4227124404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eb258ee70 0x555eb259b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/4227124404 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x555eb25fbd70 0x555eb25fa380 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/4227124404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eb258d650 0x555eb259c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb18aab8b80 1 --2- 172.21.15.150:0/4227124404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555eb258e520 0x555eb259cf10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb1537fe700 0 client.15438 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:03:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb18aab8b80 1 -- 172.21.15.150:0/4227124404 >> 172.21.15.150:0/4227124404 conn(0x555eb24efa10 msgr2=0x555eb2591750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb18aab8b80 1 -- 172.21.15.150:0/4227124404 shutdown_connections 2024-05-10T04:03:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb18aab8b80 1 -- 172.21.15.150:0/4227124404 wait complete. 2024-05-10T04:03:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb18aab8b80 20 client.15438 trim_cache size 0 max 16384 2024-05-10T04:03:59.009 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:03:59.009 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "325b4c82-0e81-11ef-bc98-c7b262605968", 2024-05-10T04:03:59.009 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 2, 2024-05-10T04:03:59.009 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24535, 2024-05-10T04:03:59.009 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:03:59.009 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:03:59.009 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:03:59.009 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 62.829580317999998, 2024-05-10T04:03:59.009 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 24, 2024-05-10T04:03:59.009 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 87, 2024-05-10T04:03:59.009 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 85, 2024-05-10T04:03:59.009 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 101.787421194 2024-05-10T04:03:59.010 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:03:59.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb17a774700 1 -- 172.21.15.150:0/3981606446 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb15c0a1260 msgr2=0x7fb15c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb17a774700 1 --2- 172.21.15.150:0/3981606446 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb15c0a1260 0x7fb15c0a3770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb16400b3c0 tx=0x7fb16400d040 comp rx=0 tx=0).stop 2024-05-10T04:03:59.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb17a774700 1 -- 172.21.15.150:0/3981606446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb174102580 msgr2=0x7fb1741a14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.000+0000 7fb17a774700 1 --2- 172.21.15.150:0/3981606446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb174102580 0x7fb1741a14f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00ef30 tx=0x7fb16c00c3a0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.001+0000 7fb17a774700 1 -- 172.21.15.150:0/3981606446 shutdown_connections 2024-05-10T04:03:59.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.001+0000 7fb17a774700 1 --2- 172.21.15.150:0/3981606446 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb15c0a1260 0x7fb15c0a3770 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.001+0000 7fb17a774700 1 --2- 172.21.15.150:0/3981606446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb174102580 0x7fb1741a14f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.001+0000 7fb17a774700 1 --2- 172.21.15.150:0/3981606446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb174100ac0 0x7fb17419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.001+0000 7fb17a774700 1 --2- 172.21.15.150:0/3981606446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb174101be0 0x7fb17419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.001+0000 7fb17a774700 1 -- 172.21.15.150:0/3981606446 >> 172.21.15.150:0/3981606446 conn(0x7fb1740fc0d0 msgr2=0x7fb174104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.001+0000 7fb17a774700 1 -- 172.21.15.150:0/3981606446 shutdown_connections 2024-05-10T04:03:59.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.001+0000 7fb17a774700 1 -- 172.21.15.150:0/3981606446 wait complete. 2024-05-10T04:03:59.016 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-10T04:03:59.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.148+0000 7f1318705700 1 -- 172.21.15.150:0/3266562042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1310100d30 msgr2=0x7f1310101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.148+0000 7f1318705700 1 --2- 172.21.15.150:0/3266562042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1310100d30 0x7f1310101140 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f130c009990 tx=0x7f130c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.148+0000 7f1318705700 1 -- 172.21.15.150:0/3266562042 shutdown_connections 2024-05-10T04:03:59.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.148+0000 7f1318705700 1 --2- 172.21.15.150:0/3266562042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1310101680 0x7f1310108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.148+0000 7f1318705700 1 --2- 172.21.15.150:0/3266562042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1310102030 0x7f1310102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.148+0000 7f1318705700 1 --2- 172.21.15.150:0/3266562042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1310100d30 0x7f1310101140 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.148+0000 7f1318705700 1 -- 172.21.15.150:0/3266562042 >> 172.21.15.150:0/3266562042 conn(0x7f13100fb8f0 msgr2=0x7f13100fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.148+0000 7f1318705700 1 -- 172.21.15.150:0/3266562042 shutdown_connections 2024-05-10T04:03:59.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.148+0000 7f1318705700 1 -- 172.21.15.150:0/3266562042 wait complete. 2024-05-10T04:03:59.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.149+0000 7f1318705700 1 Processor -- start 2024-05-10T04:03:59.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.149+0000 7f1318705700 1 -- start start 2024-05-10T04:03:59.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.149+0000 7f1318705700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1310100d30 0x7f1310196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.149+0000 7f1318705700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1310101680 0x7f1310197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.149+0000 7f1318705700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1310102030 0x7f131019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.149+0000 7f1318705700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131019e8d0 con 0x7f1310100d30 2024-05-10T04:03:59.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.149+0000 7f1318705700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131019ea10 con 0x7f1310101680 2024-05-10T04:03:59.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.149+0000 7f1318705700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131019eba0 con 0x7f1310102030 2024-05-10T04:03:59.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.149+0000 7f13164a1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1310100d30 0x7f1310196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.149+0000 7f13164a1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1310100d30 0x7f1310196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38762/0 (socket says 172.21.15.150:38762) 2024-05-10T04:03:59.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.149+0000 7f13164a1700 1 -- 172.21.15.150:0/366431757 learned_addr learned my addr 172.21.15.150:0/366431757 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:59.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.149+0000 7f1315ca0700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1310101680 0x7f1310197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.149+0000 7f1316ca2700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1310102030 0x7f131019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.150+0000 7f13164a1700 1 -- 172.21.15.150:0/366431757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1310102030 msgr2=0x7f131019e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.150+0000 7f13164a1700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1310102030 0x7f131019e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.150+0000 7f13164a1700 1 -- 172.21.15.150:0/366431757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1310101680 msgr2=0x7f1310197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.150+0000 7f13164a1700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1310101680 0x7f1310197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.150+0000 7f13164a1700 1 -- 172.21.15.150:0/366431757 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c0095f0 con 0x7f1310100d30 2024-05-10T04:03:59.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.150+0000 7f13164a1700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1310100d30 0x7f1310196e30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f130000d9b0 tx=0x7f130000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.150+0000 7f13077fe700 1 -- 172.21.15.150:0/366431757 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300009b40 con 0x7f1310100d30 2024-05-10T04:03:59.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.150+0000 7f13077fe700 1 -- 172.21.15.150:0/366431757 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f130000be80 con 0x7f1310100d30 2024-05-10T04:03:59.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.150+0000 7f13077fe700 1 -- 172.21.15.150:0/366431757 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130000f620 con 0x7f1310100d30 2024-05-10T04:03:59.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.150+0000 7f1318705700 1 -- 172.21.15.150:0/366431757 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131019ee80 con 0x7f1310100d30 2024-05-10T04:03:59.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.150+0000 7f1318705700 1 -- 172.21.15.150:0/366431757 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131019f250 con 0x7f1310100d30 2024-05-10T04:03:59.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.153+0000 7f13077fe700 1 -- 172.21.15.150:0/366431757 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f130000f780 con 0x7f1310100d30 2024-05-10T04:03:59.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.154+0000 7f13077fe700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f12fc0a1260 0x7f12fc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.154+0000 7f13077fe700 1 -- 172.21.15.150:0/366431757 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f13000cc970 con 0x7f1310100d30 2024-05-10T04:03:59.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.154+0000 7f1315ca0700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f12fc0a1260 0x7f12fc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.154+0000 7f1328a49b80 1 Processor -- start 2024-05-10T04:03:59.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.154+0000 7f1328a49b80 1 -- start start 2024-05-10T04:03:59.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.154+0000 7f1315ca0700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f12fc0a1260 0x7f12fc0a3770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f130c006010 tx=0x7f130c00be20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1328a49b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f01650 0x558076f01aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1328a49b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 0x558076f02930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1328a49b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02e70 0x558076f09770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1328a49b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558076e28100 con 0x558076f02520 2024-05-10T04:03:59.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1328a49b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558076e25ac0 con 0x558076f02e70 2024-05-10T04:03:59.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1328a49b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558076e213b0 con 0x558076f01650 2024-05-10T04:03:59.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1315ca0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 0x558076f02930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1315ca0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 0x558076f02930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38770/0 (socket says 172.21.15.150:38770) 2024-05-10T04:03:59.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1315ca0700 1 -- 172.21.15.150:0/3388315769 learned_addr learned my addr 172.21.15.150:0/3388315769 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:59.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1316ca2700 1 --2- 172.21.15.150:0/3388315769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02e70 0x558076f09770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f13164a1700 1 --2- 172.21.15.150:0/3388315769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f01650 0x558076f01aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1315ca0700 1 -- 172.21.15.150:0/3388315769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f01650 msgr2=0x558076f01aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1315ca0700 1 --2- 172.21.15.150:0/3388315769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f01650 0x558076f01aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1315ca0700 1 -- 172.21.15.150:0/3388315769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02e70 msgr2=0x558076f09770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1315ca0700 1 --2- 172.21.15.150:0/3388315769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02e70 0x558076f09770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1315ca0700 1 -- 172.21.15.150:0/3388315769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558076ca2360 con 0x558076f02520 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.155+0000 7f1315ca0700 1 --2- 172.21.15.150:0/3388315769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 0x558076f02930 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f130c005f50 tx=0x7f130c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e301b5437e43ab00 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.156+0000 7f1304ff9700 1 -- 172.21.15.150:0/3388315769 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c01d070 con 0x558076f02520 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.156+0000 7f1304ff9700 1 -- 172.21.15.150:0/3388315769 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f130c00fd80 con 0x558076f02520 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.156+0000 7f1304ff9700 1 -- 172.21.15.150:0/3388315769 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c005330 con 0x558076f02520 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.156+0000 7f1328a49b80 1 -- 172.21.15.150:0/3388315769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 msgr2=0x558076f02930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.156+0000 7f1328a49b80 1 --2- 172.21.15.150:0/3388315769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 0x558076f02930 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f130c005f50 tx=0x7f130c01d000 comp rx=0 tx=0).stop 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.156+0000 7f1328a49b80 1 -- 172.21.15.150:0/3388315769 shutdown_connections 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.156+0000 7f1328a49b80 1 --2- 172.21.15.150:0/3388315769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02e70 0x558076f09770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.156+0000 7f1328a49b80 1 --2- 172.21.15.150:0/3388315769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f01650 0x558076f01aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.156+0000 7f1328a49b80 1 --2- 172.21.15.150:0/3388315769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 0x558076f02930 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.156+0000 7f1328a49b80 1 -- 172.21.15.150:0/3388315769 >> 172.21.15.150:0/3388315769 conn(0x558076e63a10 msgr2=0x558076efbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f1328a49b80 1 -- 172.21.15.150:0/3388315769 shutdown_connections 2024-05-10T04:03:59.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f1328a49b80 1 -- 172.21.15.150:0/3388315769 wait complete. 2024-05-10T04:03:59.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f1328a49b80 1 -- start start 2024-05-10T04:03:59.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f1328a49b80 1 Processor -- start 2024-05-10T04:03:59.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f1328a49b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f01650 0x558076f0b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f1328a49b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02520 0x558076f0bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f1328a49b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f02e70 0x558076f12a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f1328a49b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558076dbd510 con 0x558076f01650 2024-05-10T04:03:59.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f1328a49b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558076dbd1b0 con 0x558076f02520 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f1328a49b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558076ca4650 con 0x558076f02e70 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f1328a49b80 10 client.0 _ll_register_callbacks cb 0x558076dc6ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f13164a1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f01650 0x558076f0b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f13164a1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f01650 0x558076f0b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38786/0 (socket says 172.21.15.150:38786) 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.157+0000 7f13164a1700 1 -- 172.21.15.150:0/143754310 learned_addr learned my addr 172.21.15.150:0/143754310 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f1315ca0700 1 --2- 172.21.15.150:0/143754310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02520 0x558076f0bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f1316ca2700 1 --2- 172.21.15.150:0/143754310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f02e70 0x558076f12a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f13164a1700 1 -- 172.21.15.150:0/143754310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f02e70 msgr2=0x558076f12a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f13164a1700 1 --2- 172.21.15.150:0/143754310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f02e70 0x558076f12a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f13164a1700 1 -- 172.21.15.150:0/143754310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02520 msgr2=0x558076f0bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f13164a1700 1 --2- 172.21.15.150:0/143754310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02520 0x558076f0bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f13164a1700 1 -- 172.21.15.150:0/143754310 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1310109940 con 0x558076f01650 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f13164a1700 1 --2- 172.21.15.150:0/143754310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f01650 0x558076f0b5c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1300000c00 tx=0x7f1300009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53191f1d603072dd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f12f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f12f9ffb700 1 -- 172.21.15.150:0/143754310 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300098fe0 con 0x558076f01650 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f12f9ffb700 1 -- 172.21.15.150:0/143754310 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f130009d720 con 0x558076f01650 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f1328a49b80 1 -- 172.21.15.150:0/143754310 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558076d7a220 con 0x558076f01650 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f12f9ffb700 1 -- 172.21.15.150:0/143754310 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13000a1d70 con 0x558076f01650 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f1328a49b80 10 client.15450 fetch_fsmap learned FSMap version 24 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f1328a49b80 1 -- 172.21.15.150:0/143754310 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558076c4eb10 con 0x558076f01650 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.158+0000 7f12f9ffb700 1 -- 172.21.15.150:0/143754310 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558076d7a220 con 0x558076f01650 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.159+0000 7f12f9ffb700 1 -- 172.21.15.150:0/143754310 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f13000b90b0 con 0x558076f01650 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.159+0000 7f1328a49b80 10 client.15450 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.159+0000 7f1328a49b80 10 client.15450 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.159+0000 7f1328a49b80 20 client.15450 populate_metadata read hostname 'smithi150' 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.159+0000 7f1328a49b80 1 --2- 172.21.15.150:0/143754310 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x558076f16bc0 0x558076f1b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.159+0000 7f1328a49b80 4 client.15450 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:03:59.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.159+0000 7f1328a49b80 1 -- 172.21.15.150:0/143754310 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558076ed0730 con 0x558076f16bc0 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.159+0000 7f1315ca0700 1 --2- 172.21.15.150:0/143754310 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x558076f16bc0 0x558076f1b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.159+0000 7f12f9ffb700 10 client.15450 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.159+0000 7f1315ca0700 1 --2- 172.21.15.150:0/143754310 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x558076f16bc0 0x558076f1b8f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b2052c2a1b37be9 server_cookie=360af1bf38e3736 in_seq=0 out_seq=0 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f12f9ffb700 1 -- 172.21.15.150:0/143754310 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558076ed0730 con 0x558076f16bc0 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f12f9ffb700 10 client.15450 handle_command_reply: tid=0 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f1328a49b80 1 client.15450 shutdown 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f1328a49b80 1 -- 172.21.15.150:0/143754310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f01650 msgr2=0x558076f0b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f1328a49b80 1 --2- 172.21.15.150:0/143754310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f01650 0x558076f0b5c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1300000c00 tx=0x7f1300009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f1328a49b80 1 -- 172.21.15.150:0/143754310 shutdown_connections 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f1328a49b80 1 --2- 172.21.15.150:0/143754310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f02e70 0x558076f12a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f1328a49b80 1 --2- 172.21.15.150:0/143754310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f01650 0x558076f0b5c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f1328a49b80 1 --2- 172.21.15.150:0/143754310 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x558076f16bc0 0x558076f1b8f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f1328a49b80 1 --2- 172.21.15.150:0/143754310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02520 0x558076f0bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f12f9ffb700 0 client.15450 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:03:59.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f1328a49b80 1 -- 172.21.15.150:0/143754310 >> 172.21.15.150:0/143754310 conn(0x558076e63a10 msgr2=0x558076efbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f1328a49b80 1 -- 172.21.15.150:0/143754310 shutdown_connections 2024-05-10T04:03:59.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f1328a49b80 1 -- 172.21.15.150:0/143754310 wait complete. 2024-05-10T04:03:59.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.160+0000 7f1328a49b80 20 client.15450 trim_cache size 0 max 16384 2024-05-10T04:03:59.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.178+0000 7f1328a49b80 1 Processor -- start 2024-05-10T04:03:59.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.178+0000 7f1328a49b80 1 -- start start 2024-05-10T04:03:59.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.178+0000 7f1328a49b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f01650 0x558076f05750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.178+0000 7f1328a49b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 0x558076f0a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.178+0000 7f1328a49b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f02e70 0x558076f10a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.178+0000 7f1328a49b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558076f14770 con 0x558076f02520 2024-05-10T04:03:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.178+0000 7f1328a49b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558076ddf2b0 con 0x558076f01650 2024-05-10T04:03:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.178+0000 7f1328a49b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558076e1e010 con 0x558076f02e70 2024-05-10T04:03:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.178+0000 7f1315ca0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 0x558076f0a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f1315ca0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 0x558076f0a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38796/0 (socket says 172.21.15.150:38796) 2024-05-10T04:03:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f1315ca0700 1 -- 172.21.15.150:0/2461016640 learned_addr learned my addr 172.21.15.150:0/2461016640 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f1316ca2700 1 --2- 172.21.15.150:0/2461016640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f02e70 0x558076f10a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f13164a1700 1 --2- 172.21.15.150:0/2461016640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f01650 0x558076f05750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f1315ca0700 1 -- 172.21.15.150:0/2461016640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f02e70 msgr2=0x558076f10a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f1315ca0700 1 --2- 172.21.15.150:0/2461016640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f02e70 0x558076f10a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f1315ca0700 1 -- 172.21.15.150:0/2461016640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f01650 msgr2=0x558076f05750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f1315ca0700 1 --2- 172.21.15.150:0/2461016640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f01650 0x558076f05750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f1315ca0700 1 -- 172.21.15.150:0/2461016640 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558076de7510 con 0x558076f02520 2024-05-10T04:03:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f13164a1700 1 --2- 172.21.15.150:0/2461016640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f01650 0x558076f05750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f1315ca0700 1 --2- 172.21.15.150:0/2461016640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 0x558076f0a7e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f130c009ce0 tx=0x7f130c00f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd538357b65d72f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f13057fa700 1 -- 172.21.15.150:0/2461016640 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c034040 con 0x558076f02520 2024-05-10T04:03:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f13057fa700 1 -- 172.21.15.150:0/2461016640 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f130c003cf0 con 0x558076f02520 2024-05-10T04:03:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.179+0000 7f13057fa700 1 -- 172.21.15.150:0/2461016640 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c038a90 con 0x558076f02520 2024-05-10T04:03:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.180+0000 7f1328a49b80 1 -- 172.21.15.150:0/2461016640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 msgr2=0x558076f0a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.180+0000 7f1328a49b80 1 --2- 172.21.15.150:0/2461016640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 0x558076f0a7e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f130c009ce0 tx=0x7f130c00f7c0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.180+0000 7f1328a49b80 1 -- 172.21.15.150:0/2461016640 shutdown_connections 2024-05-10T04:03:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.180+0000 7f1328a49b80 1 --2- 172.21.15.150:0/2461016640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f02e70 0x558076f10a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.180+0000 7f1328a49b80 1 --2- 172.21.15.150:0/2461016640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f01650 0x558076f05750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.180+0000 7f1328a49b80 1 --2- 172.21.15.150:0/2461016640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02520 0x558076f0a7e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.180+0000 7f1328a49b80 1 -- 172.21.15.150:0/2461016640 >> 172.21.15.150:0/2461016640 conn(0x558076e63a10 msgr2=0x558076ef8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.180+0000 7f1328a49b80 1 -- 172.21.15.150:0/2461016640 shutdown_connections 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.180+0000 7f1328a49b80 1 -- 172.21.15.150:0/2461016640 wait complete. 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.180+0000 7f1328a49b80 1 -- start start 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.180+0000 7f1328a49b80 1 Processor -- start 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1328a49b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f01650 0x558076f0ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1328a49b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02520 0x558076f0f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1328a49b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02e70 0x558076f0ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1328a49b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558076f104d0 con 0x558076f02e70 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1328a49b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558076f10640 con 0x558076f02520 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1328a49b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558076f10800 con 0x558076f01650 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1328a49b80 10 client.0 _ll_register_callbacks cb 0x558076dc6ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1316ca2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02e70 0x558076f0ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f13164a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f01650 0x558076f0ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1316ca2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02e70 0x558076f0ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38806/0 (socket says 172.21.15.150:38806) 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1316ca2700 1 -- 172.21.15.150:0/1467434545 learned_addr learned my addr 172.21.15.150:0/1467434545 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1315ca0700 1 --2- 172.21.15.150:0/1467434545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02520 0x558076f0f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1316ca2700 1 -- 172.21.15.150:0/1467434545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f01650 msgr2=0x558076f0ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1316ca2700 1 --2- 172.21.15.150:0/1467434545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f01650 0x558076f0ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1316ca2700 1 -- 172.21.15.150:0/1467434545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02520 msgr2=0x558076f0f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1316ca2700 1 --2- 172.21.15.150:0/1467434545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02520 0x558076f0f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1316ca2700 1 -- 172.21.15.150:0/1467434545 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558076ca2360 con 0x558076f02e70 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f13164a1700 1 --2- 172.21.15.150:0/1467434545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f01650 0x558076f0ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f1316ca2700 1 --2- 172.21.15.150:0/1467434545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02e70 0x558076f0ff90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1308007f10 tx=0x7f1308007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354bc0662b5acb19 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.181+0000 7f12f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f12f9ffb700 1 -- 172.21.15.150:0/1467434545 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130801a070 con 0x558076f02e70 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f12f9ffb700 1 -- 172.21.15.150:0/1467434545 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1308012bb0 con 0x558076f02e70 2024-05-10T04:03:59.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f1328a49b80 1 -- 172.21.15.150:0/1467434545 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f130c00fd80 con 0x558076f02e70 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f12f9ffb700 1 -- 172.21.15.150:0/1467434545 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1308012d70 con 0x558076f02e70 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f1328a49b80 10 client.15456 fetch_fsmap learned FSMap version 24 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f1328a49b80 1 -- 172.21.15.150:0/1467434545 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558076ca6c80 con 0x558076f02e70 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f12f9ffb700 1 -- 172.21.15.150:0/1467434545 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f130c00fd80 con 0x558076f02e70 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f12f9ffb700 1 -- 172.21.15.150:0/1467434545 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f130801e3c0 con 0x558076f02e70 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f1328a49b80 10 client.15456 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f1328a49b80 10 client.15456 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f1328a49b80 20 client.15456 populate_metadata read hostname 'smithi150' 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f1328a49b80 1 --2- 172.21.15.150:0/1467434545 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x558076f6fb60 0x558076f6e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f1328a49b80 4 client.15456 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f1328a49b80 1 -- 172.21.15.150:0/1467434545 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558076e19e70 con 0x558076f6fb60 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.182+0000 7f13164a1700 1 --2- 172.21.15.150:0/1467434545 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x558076f6fb60 0x558076f6e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f12f9ffb700 10 client.15456 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:03:59.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f13164a1700 1 --2- 172.21.15.150:0/1467434545 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x558076f6fb60 0x558076f6e0c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf10159a95b7b30b server_cookie=ac8778d5b200ec60 in_seq=0 out_seq=0 2024-05-10T04:03:59.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f12f9ffb700 1 -- 172.21.15.150:0/1467434545 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558076e19e70 con 0x558076f6fb60 2024-05-10T04:03:59.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f12f9ffb700 10 client.15456 handle_command_reply: tid=0 2024-05-10T04:03:59.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f1328a49b80 1 client.15456 shutdown 2024-05-10T04:03:59.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f1328a49b80 1 -- 172.21.15.150:0/1467434545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02e70 msgr2=0x558076f0ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f1328a49b80 1 --2- 172.21.15.150:0/1467434545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02e70 0x558076f0ff90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1308007f10 tx=0x7f1308007f40 comp rx=0 tx=0).stop 2024-05-10T04:03:59.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f1328a49b80 1 -- 172.21.15.150:0/1467434545 shutdown_connections 2024-05-10T04:03:59.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f1328a49b80 1 --2- 172.21.15.150:0/1467434545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558076f02e70 0x558076f0ff90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f1328a49b80 1 --2- 172.21.15.150:0/1467434545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558076f01650 0x558076f0ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f1328a49b80 1 --2- 172.21.15.150:0/1467434545 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x558076f6fb60 0x558076f6e0c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f1328a49b80 1 --2- 172.21.15.150:0/1467434545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558076f02520 0x558076f0f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f12f9ffb700 0 client.15456 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:03:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f1328a49b80 1 -- 172.21.15.150:0/1467434545 >> 172.21.15.150:0/1467434545 conn(0x558076e63a10 msgr2=0x558076ef8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f1328a49b80 1 -- 172.21.15.150:0/1467434545 shutdown_connections 2024-05-10T04:03:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f1328a49b80 1 -- 172.21.15.150:0/1467434545 wait complete. 2024-05-10T04:03:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.183+0000 7f1328a49b80 20 client.15456 trim_cache size 0 max 16384 2024-05-10T04:03:59.195 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:03:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "325b4c82-0e81-11ef-bc98-c7b262605968", 2024-05-10T04:03:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 1, 2024-05-10T04:03:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24556, 2024-05-10T04:03:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:03:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:03:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:03:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 64.019881788999996, 2024-05-10T04:03:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 24, 2024-05-10T04:03:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 87, 2024-05-10T04:03:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 85, 2024-05-10T04:03:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 98.364407338000007 2024-05-10T04:03:59.196 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:03:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.184+0000 7f1318705700 1 -- 172.21.15.150:0/366431757 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f12fc0a1260 msgr2=0x7f12fc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.184+0000 7f1318705700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f12fc0a1260 0x7f12fc0a3770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f130c006010 tx=0x7f130c00be20 comp rx=0 tx=0).stop 2024-05-10T04:03:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.184+0000 7f1318705700 1 -- 172.21.15.150:0/366431757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1310100d30 msgr2=0x7f1310196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.184+0000 7f1318705700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1310100d30 0x7f1310196e30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f130000d9b0 tx=0x7f130000dd70 comp rx=0 tx=0).stop 2024-05-10T04:03:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.184+0000 7f1318705700 1 -- 172.21.15.150:0/366431757 shutdown_connections 2024-05-10T04:03:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.184+0000 7f1318705700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f12fc0a1260 0x7f12fc0a3770 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.184+0000 7f1318705700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1310102030 0x7f131019e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.184+0000 7f1318705700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1310100d30 0x7f1310196e30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.184+0000 7f1318705700 1 --2- 172.21.15.150:0/366431757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1310101680 0x7f1310197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.184+0000 7f1318705700 1 -- 172.21.15.150:0/366431757 >> 172.21.15.150:0/366431757 conn(0x7f13100fb8f0 msgr2=0x7f1310104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.184+0000 7f1318705700 1 -- 172.21.15.150:0/366431757 shutdown_connections 2024-05-10T04:03:59.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.184+0000 7f1318705700 1 -- 172.21.15.150:0/366431757 wait complete. 2024-05-10T04:03:59.201 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-10T04:03:59.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.333+0000 7fd205031700 1 -- 172.21.15.150:0/3817433122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd200100ac0 msgr2=0x7fd200100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.333+0000 7fd205031700 1 --2- 172.21.15.150:0/3817433122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd200100ac0 0x7fd200100ed0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009990 tx=0x7fd1f8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.333+0000 7fd205031700 1 -- 172.21.15.150:0/3817433122 shutdown_connections 2024-05-10T04:03:59.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.333+0000 7fd205031700 1 --2- 172.21.15.150:0/3817433122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd200102580 0x7fd200108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.333+0000 7fd205031700 1 --2- 172.21.15.150:0/3817433122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd200100ac0 0x7fd200100ed0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.333+0000 7fd205031700 1 --2- 172.21.15.150:0/3817433122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd200101be0 0x7fd200101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.333+0000 7fd205031700 1 -- 172.21.15.150:0/3817433122 >> 172.21.15.150:0/3817433122 conn(0x7fd2000fc0d0 msgr2=0x7fd2000fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.333+0000 7fd205031700 1 -- 172.21.15.150:0/3817433122 shutdown_connections 2024-05-10T04:03:59.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.333+0000 7fd205031700 1 -- 172.21.15.150:0/3817433122 wait complete. 2024-05-10T04:03:59.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.334+0000 7fd205031700 1 Processor -- start 2024-05-10T04:03:59.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.334+0000 7fd205031700 1 -- start start 2024-05-10T04:03:59.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd205031700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd200100ac0 0x7fd200196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd205031700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd200101be0 0x7fd200197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd205031700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd200102580 0x7fd20019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd205031700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20019e920 con 0x7fd200101be0 2024-05-10T04:03:59.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd205031700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20019ea90 con 0x7fd200102580 2024-05-10T04:03:59.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd205031700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20019ec50 con 0x7fd200100ac0 2024-05-10T04:03:59.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd1fe59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd200101be0 0x7fd200197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd1fe59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd200101be0 0x7fd200197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35284/0 (socket says 172.21.15.150:35284) 2024-05-10T04:03:59.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd1fe59c700 1 -- 172.21.15.150:0/1707556692 learned_addr learned my addr 172.21.15.150:0/1707556692 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:59.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd1fed9d700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd200100ac0 0x7fd200196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd1ff59e700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd200102580 0x7fd20019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd1fe59c700 1 -- 172.21.15.150:0/1707556692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd200100ac0 msgr2=0x7fd200196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd200100ac0 0x7fd200196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd1fe59c700 1 -- 172.21.15.150:0/1707556692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd200102580 msgr2=0x7fd20019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd200102580 0x7fd20019e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd1fe59c700 1 -- 172.21.15.150:0/1707556692 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f80095f0 con 0x7fd200101be0 2024-05-10T04:03:59.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd200101be0 0x7fd200197380 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800b530 tx=0x7fd1e800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.335+0000 7fd1f7fff700 1 -- 172.21.15.150:0/1707556692 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8011830 con 0x7fd200101be0 2024-05-10T04:03:59.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.336+0000 7fd1f7fff700 1 -- 172.21.15.150:0/1707556692 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e8011e50 con 0x7fd200101be0 2024-05-10T04:03:59.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.336+0000 7fd205031700 1 -- 172.21.15.150:0/1707556692 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20019ef30 con 0x7fd200101be0 2024-05-10T04:03:59.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.336+0000 7fd1f7fff700 1 -- 172.21.15.150:0/1707556692 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e800f2e0 con 0x7fd200101be0 2024-05-10T04:03:59.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.336+0000 7fd205031700 1 -- 172.21.15.150:0/1707556692 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20019f3d0 con 0x7fd200101be0 2024-05-10T04:03:59.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.339+0000 7fd1f7fff700 1 -- 172.21.15.150:0/1707556692 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd1e800f440 con 0x7fd200101be0 2024-05-10T04:03:59.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.339+0000 7fd1f7fff700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd1ec0a1260 0x7fd1ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.339+0000 7fd1f7fff700 1 -- 172.21.15.150:0/1707556692 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd1e80cc660 con 0x7fd200101be0 2024-05-10T04:03:59.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.339+0000 7fd1fed9d700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd1ec0a1260 0x7fd1ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd215375b80 1 Processor -- start 2024-05-10T04:03:59.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd215375b80 1 -- start start 2024-05-10T04:03:59.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd1fed9d700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd1ec0a1260 0x7fd1ec0a3770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80098e0 tx=0x7fd1f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd215375b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb959860 0x561cfb959cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd215375b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb95ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd215375b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb95b080 0x561cfb961980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd215375b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb87e100 con 0x561cfb95a730 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd215375b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb87bac0 con 0x561cfb95b080 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd215375b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb8773b0 con 0x561cfb959860 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd1fe59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb95ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd1fe59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb95ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35296/0 (socket says 172.21.15.150:35296) 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd1fe59c700 1 -- 172.21.15.150:0/3429933963 learned_addr learned my addr 172.21.15.150:0/3429933963 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd1fed9d700 1 --2- 172.21.15.150:0/3429933963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb959860 0x561cfb959cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.340+0000 7fd1ff59e700 1 --2- 172.21.15.150:0/3429933963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb95b080 0x561cfb961980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.341+0000 7fd1fe59c700 1 -- 172.21.15.150:0/3429933963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb959860 msgr2=0x561cfb959cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.341+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/3429933963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb959860 0x561cfb959cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.341+0000 7fd1fe59c700 1 -- 172.21.15.150:0/3429933963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb95b080 msgr2=0x561cfb961980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.341+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/3429933963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb95b080 0x561cfb961980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.341+0000 7fd1fe59c700 1 -- 172.21.15.150:0/3429933963 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cfb6f8360 con 0x561cfb95a730 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.341+0000 7fd1ff59e700 1 --2- 172.21.15.150:0/3429933963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb95b080 0x561cfb961980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.341+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/3429933963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb95ab40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80048a0 tx=0x7fd1e8099360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695394ddb75814cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.341+0000 7fd1f57fa700 1 -- 172.21.15.150:0/3429933963 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e809d580 con 0x561cfb95a730 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.341+0000 7fd1f57fa700 1 -- 172.21.15.150:0/3429933963 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e80a1bd0 con 0x561cfb95a730 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.341+0000 7fd1f57fa700 1 -- 172.21.15.150:0/3429933963 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80ca020 con 0x561cfb95a730 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.342+0000 7fd215375b80 1 -- 172.21.15.150:0/3429933963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 msgr2=0x561cfb95ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.342+0000 7fd215375b80 1 --2- 172.21.15.150:0/3429933963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb95ab40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80048a0 tx=0x7fd1e8099360 comp rx=0 tx=0).stop 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.342+0000 7fd215375b80 1 -- 172.21.15.150:0/3429933963 shutdown_connections 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.342+0000 7fd215375b80 1 --2- 172.21.15.150:0/3429933963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb95b080 0x561cfb961980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.342+0000 7fd215375b80 1 --2- 172.21.15.150:0/3429933963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb959860 0x561cfb959cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.342+0000 7fd215375b80 1 --2- 172.21.15.150:0/3429933963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb95ab40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.342+0000 7fd215375b80 1 -- 172.21.15.150:0/3429933963 >> 172.21.15.150:0/3429933963 conn(0x561cfb8b9a10 msgr2=0x561cfb951f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.342+0000 7fd215375b80 1 -- 172.21.15.150:0/3429933963 shutdown_connections 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.342+0000 7fd215375b80 1 -- 172.21.15.150:0/3429933963 wait complete. 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.342+0000 7fd215375b80 1 -- start start 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.342+0000 7fd215375b80 1 Processor -- start 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd215375b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb959860 0x561cfb963900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd215375b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb963e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd215375b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95b080 0x561cfb96ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd215375b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb813510 con 0x561cfb95a730 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd215375b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb8131b0 con 0x561cfb959860 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd215375b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb6fa650 con 0x561cfb95b080 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd215375b80 10 client.0 _ll_register_callbacks cb 0x561cfb81cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1fe59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb963e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1fe59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb963e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35298/0 (socket says 172.21.15.150:35298) 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1fe59c700 1 -- 172.21.15.150:0/4098780999 learned_addr learned my addr 172.21.15.150:0/4098780999 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:59.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1ff59e700 1 --2- 172.21.15.150:0/4098780999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95b080 0x561cfb96ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1fed9d700 1 --2- 172.21.15.150:0/4098780999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb959860 0x561cfb963900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1fe59c700 1 -- 172.21.15.150:0/4098780999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95b080 msgr2=0x561cfb96ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/4098780999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95b080 0x561cfb96ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1fe59c700 1 -- 172.21.15.150:0/4098780999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb959860 msgr2=0x561cfb963900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/4098780999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb959860 0x561cfb963900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1fe59c700 1 -- 172.21.15.150:0/4098780999 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20019f3d0 con 0x561cfb95a730 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/4098780999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb963e40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800bb90 tx=0x7fd1e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca62fb7ca750c793 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1e67fc700 1 -- 172.21.15.150:0/4098780999 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e809d580 con 0x561cfb95a730 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd1e67fc700 1 -- 172.21.15.150:0/4098780999 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd1e80105a0 con 0x561cfb95a730 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.343+0000 7fd215375b80 1 -- 172.21.15.150:0/4098780999 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cfb7d0220 con 0x561cfb95a730 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.344+0000 7fd1e67fc700 1 -- 172.21.15.150:0/4098780999 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1e80660f0 con 0x561cfb95a730 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.344+0000 7fd215375b80 10 client.15468 fetch_fsmap learned FSMap version 24 2024-05-10T04:03:59.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.344+0000 7fd215375b80 1 -- 172.21.15.150:0/4098780999 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561cfb6a4b10 con 0x561cfb95a730 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.344+0000 7fd1e67fc700 1 -- 172.21.15.150:0/4098780999 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561cfb7d0220 con 0x561cfb95a730 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.344+0000 7fd1e67fc700 1 -- 172.21.15.150:0/4098780999 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd1e8098c40 con 0x561cfb95a730 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.344+0000 7fd215375b80 10 client.15468 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.344+0000 7fd215375b80 10 client.15468 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.344+0000 7fd215375b80 20 client.15468 populate_metadata read hostname 'smithi150' 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.344+0000 7fd215375b80 1 --2- 172.21.15.150:0/4098780999 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561cfb96f0c0 0x561cfb9737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.344+0000 7fd215375b80 4 client.15468 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.344+0000 7fd215375b80 1 -- 172.21.15.150:0/4098780999 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cfb926730 con 0x561cfb96f0c0 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.344+0000 7fd1fed9d700 1 --2- 172.21.15.150:0/4098780999 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561cfb96f0c0 0x561cfb9737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd1e67fc700 10 client.15468 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd1fed9d700 1 --2- 172.21.15.150:0/4098780999 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561cfb96f0c0 0x561cfb9737e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=143fd148028aad3c server_cookie=47ef7d6cd86a173f in_seq=0 out_seq=0 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd1e67fc700 1 -- 172.21.15.150:0/4098780999 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561cfb926730 con 0x561cfb96f0c0 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd1e67fc700 10 client.15468 handle_command_reply: tid=0 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd215375b80 1 client.15468 shutdown 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd215375b80 1 -- 172.21.15.150:0/4098780999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 msgr2=0x561cfb963e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd215375b80 1 --2- 172.21.15.150:0/4098780999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb963e40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800bb90 tx=0x7fd1e8000bc0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd215375b80 1 -- 172.21.15.150:0/4098780999 shutdown_connections 2024-05-10T04:03:59.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd215375b80 1 --2- 172.21.15.150:0/4098780999 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561cfb96f0c0 0x561cfb9737e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd215375b80 1 --2- 172.21.15.150:0/4098780999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95b080 0x561cfb96ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd215375b80 1 --2- 172.21.15.150:0/4098780999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb959860 0x561cfb963900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd215375b80 1 --2- 172.21.15.150:0/4098780999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb963e40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd1e67fc700 0 client.15468 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:03:59.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.345+0000 7fd215375b80 1 -- 172.21.15.150:0/4098780999 >> 172.21.15.150:0/4098780999 conn(0x561cfb8b9a10 msgr2=0x561cfb951f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.346+0000 7fd215375b80 1 -- 172.21.15.150:0/4098780999 shutdown_connections 2024-05-10T04:03:59.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.346+0000 7fd215375b80 1 -- 172.21.15.150:0/4098780999 wait complete. 2024-05-10T04:03:59.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.346+0000 7fd215375b80 20 client.15468 trim_cache size 0 max 16384 2024-05-10T04:03:59.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd215375b80 1 Processor -- start 2024-05-10T04:03:59.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd215375b80 1 -- start start 2024-05-10T04:03:59.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd215375b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb959860 0x561cfb962300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd215375b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb968d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd215375b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95b080 0x561cfb969290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd215375b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb8352b0 con 0x561cfb95a730 2024-05-10T04:03:59.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd215375b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb870150 con 0x561cfb959860 2024-05-10T04:03:59.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd215375b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb8165a0 con 0x561cfb95b080 2024-05-10T04:03:59.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd1fe59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb968d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd1fe59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb968d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35306/0 (socket says 172.21.15.150:35306) 2024-05-10T04:03:59.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd1fe59c700 1 -- 172.21.15.150:0/620647041 learned_addr learned my addr 172.21.15.150:0/620647041 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:59.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd1fed9d700 1 --2- 172.21.15.150:0/620647041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb959860 0x561cfb962300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd1ff59e700 1 --2- 172.21.15.150:0/620647041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95b080 0x561cfb969290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd1fe59c700 1 -- 172.21.15.150:0/620647041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95b080 msgr2=0x561cfb969290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/620647041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95b080 0x561cfb969290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd1fe59c700 1 -- 172.21.15.150:0/620647041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb959860 msgr2=0x561cfb962300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/620647041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb959860 0x561cfb962300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.364+0000 7fd1fe59c700 1 -- 172.21.15.150:0/620647041 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cfb83d510 con 0x561cfb95a730 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.365+0000 7fd1ff59e700 1 --2- 172.21.15.150:0/620647041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95b080 0x561cfb969290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.365+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/620647041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb968d50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80048a0 tx=0x7fd1e8099360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce493dd51bd342a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.365+0000 7fd1f5ffb700 1 -- 172.21.15.150:0/620647041 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8062f10 con 0x561cfb95a730 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.365+0000 7fd1f5ffb700 1 -- 172.21.15.150:0/620647041 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e8095a90 con 0x561cfb95a730 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.365+0000 7fd1f5ffb700 1 -- 172.21.15.150:0/620647041 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80105a0 con 0x561cfb95a730 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 -- 172.21.15.150:0/620647041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 msgr2=0x561cfb968d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 --2- 172.21.15.150:0/620647041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb968d50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80048a0 tx=0x7fd1e8099360 comp rx=0 tx=0).stop 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 -- 172.21.15.150:0/620647041 shutdown_connections 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 --2- 172.21.15.150:0/620647041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95b080 0x561cfb969290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 --2- 172.21.15.150:0/620647041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb959860 0x561cfb962300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 --2- 172.21.15.150:0/620647041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb95a730 0x561cfb968d50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 -- 172.21.15.150:0/620647041 >> 172.21.15.150:0/620647041 conn(0x561cfb8b9a10 msgr2=0x561cfb961bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 -- 172.21.15.150:0/620647041 shutdown_connections 2024-05-10T04:03:59.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 -- 172.21.15.150:0/620647041 wait complete. 2024-05-10T04:03:59.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 -- start start 2024-05-10T04:03:59.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 Processor -- start 2024-05-10T04:03:59.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb959860 0x561cfb9672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95a730 0x561cfb967830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb95b080 0x561cfb968710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb968c50 con 0x561cfb959860 2024-05-10T04:03:59.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb968dc0 con 0x561cfb95b080 2024-05-10T04:03:59.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.366+0000 7fd215375b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb968f80 con 0x561cfb95a730 2024-05-10T04:03:59.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd215375b80 10 client.0 _ll_register_callbacks cb 0x561cfb81cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:03:59.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1fed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb959860 0x561cfb9672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1fed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb959860 0x561cfb9672f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35320/0 (socket says 172.21.15.150:35320) 2024-05-10T04:03:59.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1fed9d700 1 -- 172.21.15.150:0/3974062518 learned_addr learned my addr 172.21.15.150:0/3974062518 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:03:59.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1ff59e700 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb95b080 0x561cfb968710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95a730 0x561cfb967830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1fed9d700 1 -- 172.21.15.150:0/3974062518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95a730 msgr2=0x561cfb967830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1fed9d700 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95a730 0x561cfb967830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1fed9d700 1 -- 172.21.15.150:0/3974062518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb95b080 msgr2=0x561cfb968710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:03:59.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1fed9d700 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb95b080 0x561cfb968710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1fed9d700 1 -- 172.21.15.150:0/3974062518 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cfb6a4b10 con 0x561cfb959860 2024-05-10T04:03:59.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95a730 0x561cfb967830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:03:59.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1fed9d700 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb959860 0x561cfb9672f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8005990 tx=0x7fd1f8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea6503ed726d3ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:03:59.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1e67fc700 1 -- 172.21.15.150:0/3974062518 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8005170 con 0x561cfb959860 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1e67fc700 1 -- 172.21.15.150:0/3974062518 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd1f80052d0 con 0x561cfb959860 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd215375b80 1 -- 172.21.15.150:0/3974062518 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1e80a1bd0 con 0x561cfb959860 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd1e67fc700 1 -- 172.21.15.150:0/3974062518 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1f8005490 con 0x561cfb959860 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd215375b80 10 client.15474 fetch_fsmap learned FSMap version 24 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.367+0000 7fd215375b80 1 -- 172.21.15.150:0/3974062518 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561cfb6fcc80 con 0x561cfb959860 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.368+0000 7fd1e67fc700 1 -- 172.21.15.150:0/3974062518 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e80a1bd0 con 0x561cfb959860 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.368+0000 7fd1e67fc700 1 -- 172.21.15.150:0/3974062518 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd1f800f580 con 0x561cfb959860 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.368+0000 7fd215375b80 10 client.15474 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.368+0000 7fd215375b80 10 client.15474 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.368+0000 7fd215375b80 20 client.15474 populate_metadata read hostname 'smithi150' 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.368+0000 7fd215375b80 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561cfb9c7d00 0x561cfb9c6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.368+0000 7fd215375b80 4 client.15474 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.368+0000 7fd215375b80 1 -- 172.21.15.150:0/3974062518 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561cfb86fe70 con 0x561cfb9c7d00 2024-05-10T04:03:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.368+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561cfb9c7d00 0x561cfb9c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.368+0000 7fd1e67fc700 10 client.15474 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.368+0000 7fd1fe59c700 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561cfb9c7d00 0x561cfb9c6260 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9094757dc30f25c server_cookie=329fe84576156bd1 in_seq=0 out_seq=0 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd1e67fc700 1 -- 172.21.15.150:0/3974062518 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x561cfb86fe70 con 0x561cfb9c7d00 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd1e67fc700 10 client.15474 handle_command_reply: tid=0 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd215375b80 1 client.15474 shutdown 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd215375b80 1 -- 172.21.15.150:0/3974062518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb959860 msgr2=0x561cfb9672f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd215375b80 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb959860 0x561cfb9672f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8005990 tx=0x7fd1f8005040 comp rx=0 tx=0).stop 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd215375b80 1 -- 172.21.15.150:0/3974062518 shutdown_connections 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd215375b80 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561cfb9c7d00 0x561cfb9c6260 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd215375b80 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cfb95b080 0x561cfb968710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd215375b80 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561cfb959860 0x561cfb9672f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd215375b80 1 --2- 172.21.15.150:0/3974062518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561cfb95a730 0x561cfb967830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd1e67fc700 0 client.15474 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:03:59.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd215375b80 1 -- 172.21.15.150:0/3974062518 >> 172.21.15.150:0/3974062518 conn(0x561cfb8b9a10 msgr2=0x561cfb956260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd215375b80 1 -- 172.21.15.150:0/3974062518 shutdown_connections 2024-05-10T04:03:59.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd215375b80 1 -- 172.21.15.150:0/3974062518 wait complete. 2024-05-10T04:03:59.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd215375b80 20 client.15474 trim_cache size 0 max 16384 2024-05-10T04:03:59.379 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:03:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "325b4c82-0e81-11ef-bc98-c7b262605968", 2024-05-10T04:03:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 0, 2024-05-10T04:03:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24557, 2024-05-10T04:03:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:03:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:03:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:03:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 87.761702569999997, 2024-05-10T04:03:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 24, 2024-05-10T04:03:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 87, 2024-05-10T04:03:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 85, 2024-05-10T04:03:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 93.285970864000006 2024-05-10T04:03:59.380 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:03:59.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd205031700 1 -- 172.21.15.150:0/1707556692 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd1ec0a1260 msgr2=0x7fd1ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd205031700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd1ec0a1260 0x7fd1ec0a3770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80098e0 tx=0x7fd1f801a040 comp rx=0 tx=0).stop 2024-05-10T04:03:59.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd205031700 1 -- 172.21.15.150:0/1707556692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd200101be0 msgr2=0x7fd200197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:03:59.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd205031700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd200101be0 0x7fd200197380 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800b530 tx=0x7fd1e800b840 comp rx=0 tx=0).stop 2024-05-10T04:03:59.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd205031700 1 -- 172.21.15.150:0/1707556692 shutdown_connections 2024-05-10T04:03:59.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd205031700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fd1ec0a1260 0x7fd1ec0a3770 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd205031700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd200102580 0x7fd20019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd205031700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd200100ac0 0x7fd200196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd205031700 1 --2- 172.21.15.150:0/1707556692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd200101be0 0x7fd200197380 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:03:59.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.369+0000 7fd205031700 1 -- 172.21.15.150:0/1707556692 >> 172.21.15.150:0/1707556692 conn(0x7fd2000fc0d0 msgr2=0x7fd200104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:03:59.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.370+0000 7fd205031700 1 -- 172.21.15.150:0/1707556692 shutdown_connections 2024-05-10T04:03:59.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:03:59.370+0000 7fd205031700 1 -- 172.21.15.150:0/1707556692 wait complete. 2024-05-10T04:03:59.384 INFO:teuthology.run_tasks:Running task sequential... 2024-05-10T04:03:59.397 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-10T04:03:59.397 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-10T04:03:59.398 INFO:teuthology.task.print:Enabling mgr modules 2024-05-10T04:03:59.398 INFO:teuthology.run_tasks:Running task kclient... 2024-05-10T04:03:59.444 INFO:tasks.kclient:Mounting kernel clients... 2024-05-10T04:03:59.444 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-10T04:03:59.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:03:59.444 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-05-10T04:03:59.462 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-10T04:03:59.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:03:59.463 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T04:03:59.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:59 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/62695132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:59.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:59 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3655322212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:59.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:03:59 smithi150 ceph-mon[42477]: pgmap v272: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:59.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:59 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/62695132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:59.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:59 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3655322212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:59.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:03:59 smithi181 ceph-mon[46602]: pgmap v272: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:59.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:59 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/62695132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:03:59.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:59 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3655322212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:03:59.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:03:59 smithi161 ceph-mon[46918]: pgmap v272: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:03:59.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:03:59.508 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-05-10T04:03:59.574 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-10T04:03:59.575 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T04:03:59.575 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T04:03:59.575 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-10T04:03:59.802 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:04:00.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.779+0000 7fbca77e9700 1 -- 172.21.15.150:0/2086618938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca010c270 msgr2=0x7fbca010e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:00.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.779+0000 7fbca77e9700 1 --2- 172.21.15.150:0/2086618938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca010c270 0x7fbca010e740 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800b0a0 tx=0x7fbc9800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:00.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.779+0000 7fbca77e9700 1 -- 172.21.15.150:0/2086618938 shutdown_connections 2024-05-10T04:04:00.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.779+0000 7fbca77e9700 1 --2- 172.21.15.150:0/2086618938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca010c270 0x7fbca010e740 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:00.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.779+0000 7fbca77e9700 1 --2- 172.21.15.150:0/2086618938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca0102e40 0x7fbca0103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:00.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.779+0000 7fbca77e9700 1 --2- 172.21.15.150:0/2086618938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca0103790 0x7fbca010bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:00.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.779+0000 7fbca77e9700 1 -- 172.21.15.150:0/2086618938 >> 172.21.15.150:0/2086618938 conn(0x7fbca00fc7a0 msgr2=0x7fbca00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:00.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.779+0000 7fbca77e9700 1 -- 172.21.15.150:0/2086618938 shutdown_connections 2024-05-10T04:04:00.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.779+0000 7fbca77e9700 1 -- 172.21.15.150:0/2086618938 wait complete. 2024-05-10T04:04:00.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.780+0000 7fbca77e9700 1 Processor -- start 2024-05-10T04:04:00.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.780+0000 7fbca77e9700 1 -- start start 2024-05-10T04:04:00.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.780+0000 7fbca77e9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca0102e40 0x7fbca019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:00.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.780+0000 7fbca77e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca0103790 0x7fbca019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:00.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.780+0000 7fbca77e9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca010c270 0x7fbca0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:00.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.780+0000 7fbca77e9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0198a50 con 0x7fbca0102e40 2024-05-10T04:04:00.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.780+0000 7fbca77e9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0198bc0 con 0x7fbca010c270 2024-05-10T04:04:00.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.780+0000 7fbca77e9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0198d80 con 0x7fbca0103790 2024-05-10T04:04:00.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.780+0000 7fbca5585700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca0102e40 0x7fbca019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:00.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.780+0000 7fbca5585700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca0102e40 0x7fbca019e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35334/0 (socket says 172.21.15.150:35334) 2024-05-10T04:04:00.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.780+0000 7fbca5585700 1 -- 172.21.15.150:0/4152312074 learned_addr learned my addr 172.21.15.150:0/4152312074 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:00.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.780+0000 7fbca5d86700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca010c270 0x7fbca0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:00.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.781+0000 7fbca4d84700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca0103790 0x7fbca019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:00.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.781+0000 7fbca5585700 1 -- 172.21.15.150:0/4152312074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca0103790 msgr2=0x7fbca019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:00.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.781+0000 7fbca5585700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca0103790 0x7fbca019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:00.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.781+0000 7fbca5585700 1 -- 172.21.15.150:0/4152312074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca010c270 msgr2=0x7fbca0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:00.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.781+0000 7fbca5585700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca010c270 0x7fbca0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:00.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.781+0000 7fbca5585700 1 -- 172.21.15.150:0/4152312074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc98009d00 con 0x7fbca0102e40 2024-05-10T04:04:00.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.781+0000 7fbca5585700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca0102e40 0x7fbca019e520 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000d980 tx=0x7fbc9000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:00.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.781+0000 7fbc967fc700 1 -- 172.21.15.150:0/4152312074 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90004020 con 0x7fbca0102e40 2024-05-10T04:04:00.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.781+0000 7fbca77e9700 1 -- 172.21.15.150:0/4152312074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca0199060 con 0x7fbca0102e40 2024-05-10T04:04:00.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.781+0000 7fbc967fc700 1 -- 172.21.15.150:0/4152312074 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc90009730 con 0x7fbca0102e40 2024-05-10T04:04:00.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.781+0000 7fbc967fc700 1 -- 172.21.15.150:0/4152312074 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90003a80 con 0x7fbca0102e40 2024-05-10T04:04:00.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.781+0000 7fbca77e9700 1 -- 172.21.15.150:0/4152312074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca01a54f0 con 0x7fbca0102e40 2024-05-10T04:04:00.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.782+0000 7fbca77e9700 1 -- 172.21.15.150:0/4152312074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca0107430 con 0x7fbca0102e40 2024-05-10T04:04:00.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.783+0000 7fbc967fc700 1 -- 172.21.15.150:0/4152312074 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fbc900098a0 con 0x7fbca0102e40 2024-05-10T04:04:00.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.784+0000 7fbc967fc700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc8c0a11b0 0x7fbc8c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:00.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.784+0000 7fbc967fc700 1 -- 172.21.15.150:0/4152312074 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbc900cce30 con 0x7fbca0102e40 2024-05-10T04:04:00.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.784+0000 7fbca4d84700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc8c0a11b0 0x7fbc8c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:00.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.785+0000 7fbca4d84700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc8c0a11b0 0x7fbc8c0a36c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c005fd0 tx=0x7fbc9c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:00.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:00.788+0000 7fbc967fc700 1 -- 172.21.15.150:0/4152312074 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbc900960f0 con 0x7fbca0102e40 2024-05-10T04:04:01.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.027+0000 7fbca77e9700 1 -- 172.21.15.150:0/4152312074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbca0199ba0 con 0x7fbca0102e40 2024-05-10T04:04:01.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.027+0000 7fbc967fc700 1 -- 172.21.15.150:0/4152312074 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+88 (secure 0 0 0) 0x7fbc90099710 con 0x7fbca0102e40 2024-05-10T04:04:01.030 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T04:04:01.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.030+0000 7fbca77e9700 1 -- 172.21.15.150:0/4152312074 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc8c0a11b0 msgr2=0x7fbc8c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:01.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.030+0000 7fbca77e9700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc8c0a11b0 0x7fbc8c0a36c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c005fd0 tx=0x7fbc9c005db0 comp rx=0 tx=0).stop 2024-05-10T04:04:01.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.030+0000 7fbca77e9700 1 -- 172.21.15.150:0/4152312074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca0102e40 msgr2=0x7fbca019e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:01.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.030+0000 7fbca77e9700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca0102e40 0x7fbca019e520 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000d980 tx=0x7fbc9000dc90 comp rx=0 tx=0).stop 2024-05-10T04:04:01.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.030+0000 7fbca77e9700 1 -- 172.21.15.150:0/4152312074 shutdown_connections 2024-05-10T04:04:01.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.030+0000 7fbca77e9700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc8c0a11b0 0x7fbc8c0a36c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:01.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.030+0000 7fbca77e9700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca010c270 0x7fbca0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:01.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.030+0000 7fbca77e9700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbca0102e40 0x7fbca019e520 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:01.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.030+0000 7fbca77e9700 1 --2- 172.21.15.150:0/4152312074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca0103790 0x7fbca019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:01.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.030+0000 7fbca77e9700 1 -- 172.21.15.150:0/4152312074 >> 172.21.15.150:0/4152312074 conn(0x7fbca00fc7a0 msgr2=0x7fbca00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:01.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.030+0000 7fbca77e9700 1 -- 172.21.15.150:0/4152312074 shutdown_connections 2024-05-10T04:04:01.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:01.030+0000 7fbca77e9700 1 -- 172.21.15.150:0/4152312074 wait complete. 2024-05-10T04:04:01.405 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:01 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4152312074' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T04:04:01.406 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T04:04:01.407 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T04:04:01.407 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-05-10T04:04:01.407 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T04:04:01.407 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T04:04:01.407 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T04:04:01.407 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T04:04:01.407 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T04:04:01.407 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T04:04:01.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:01.408 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-05-10T04:04:01.426 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T04:04:01.426 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T04:04:01.426 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-05-10T04:04:01.427 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-10T04:04:01.428 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 41808sec preferred_lft 41808sec 2024-05-10T04:04:01.428 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::d613:fbd0:302a:c5e5/64 scope link noprefixroute 2024-05-10T04:04:01.428 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-05-10T04:04:01.428 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T04:04:01.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:01.428 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:04:01.428 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:04:01.428 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add name ceph-brx type bridge 2024-05-10T04:04:01.428 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr flush dev ceph-brx 2024-05-10T04:04:01.428 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set ceph-brx up 2024-05-10T04:04:01.428 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T04:04:01.429 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:04:01.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:01 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4152312074' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T04:04:01.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:01 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4152312074' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T04:04:01.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:01.551 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T04:04:01.579 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-05-10T04:04:01.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:01.580 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-05-10T04:04:01.641 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-05-10T04:04:01.641 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T04:04:01.641 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T04:04:01.641 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T04:04:01.641 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T04:04:01.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:01.641 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:04:01.642 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:04:01.642 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T04:04:01.642 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T04:04:01.642 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T04:04:01.642 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:04:01.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:01.749 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-05-10T04:04:01.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:01.766 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2024-05-10T04:04:01.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:01.824 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:04:01.825 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:04:01.825 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T04:04:01.825 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-10T04:04:01.825 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:04:01.920 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-10T04:04:01.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:01.920 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:04:01.920 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:04:01.920 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-10T04:04:01.920 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T04:04:01.920 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-10T04:04:01.920 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-10T04:04:01.920 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-10T04:04:01.920 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:04:02.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:02.122 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:04:02.122 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:04:02.122 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.0 up 2024-05-10T04:04:02.122 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T04:04:02.122 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:04:02.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T04:04:02.220 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T04:04:02.237 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T04:04:02.237 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T04:04:02.237 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T04:04:02.295 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-10T04:04:02.295 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-10T04:04:02.295 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/c6d4350e-b7bb-4db1-9eac-321589f390c3 2024-05-10T04:04:02.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:02.295 DEBUG:teuthology.orchestra.run.smithi150:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/c6d4350e-b7bb-4db1-9eac-321589f390c3 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-10T04:04:02.359 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:02 smithi150 ceph-mon[42477]: pgmap v273: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T04:04:02.392 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-10T04:04:02.392 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-10T04:04:02.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:02 smithi181 ceph-mon[46602]: pgmap v273: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T04:04:02.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:02 smithi161 ceph-mon[46918]: pgmap v273: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T04:04:02.507 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-10T04:04:02.507 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-10T04:04:02.508 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.161:3300,172.21.15.181:3300" 2024-05-10T04:04:02.508 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: trying mount with new device syntax: 0@325b4c82-0e81-11ef-bc98-c7b262605968.cephfs=/volumes/qa/sv_1/c6d4350e-b7bb-4db1-9eac-321589f390c3 2024-05-10T04:04:02.508 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.150:3300/172.21.15.161:3300/172.21.15.181:3300" will pass to kernel 2024-05-10T04:04:02.508 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: v2 mount support check returned 2 2024-05-10T04:04:02.508 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-10T04:04:02.508 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.150:3300,172.21.15.161:3300,172.21.15.181:3300:/volumes/qa/sv_1/c6d4350e-b7bb-4db1-9eac-321589f390c3 2024-05-10T04:04:02.508 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mds_namespace=cephfs,fsid=325b4c82-0e81-11ef-bc98-c7b262605968" will pass to kernel 2024-05-10T04:04:02.511 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T04:04:02.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:02.511 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T04:04:02.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:02.583 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T04:04:02.688 INFO:teuthology.orchestra.run.smithi150.stdout:client25208 2024-05-10T04:04:02.688 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:04:02.688 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T04:04:02.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:02.707 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T04:04:02.807 INFO:teuthology.orchestra.run.smithi150.stdout:client25208 2024-05-10T04:04:02.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:02.807 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T04:04:02.857 INFO:teuthology.orchestra.run.smithi150.stdout:client25208 2024-05-10T04:04:02.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:04:02.858 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/325b4c82-0e81-11ef-bc98-c7b262605968.client25208/status') 2024-05-10T04:04:02.916 INFO:teuthology.orchestra.run.smithi150.stdout:instance: client.25208 (3)172.21.15.150:0/3171265293 2024-05-10T04:04:02.916 INFO:teuthology.orchestra.run.smithi150.stdout:blocklisted: false 2024-05-10T04:04:02.916 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records in 2024-05-10T04:04:02.916 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records out 2024-05-10T04:04:02.916 INFO:teuthology.orchestra.run.smithi150.stderr:72 bytes copied, 2.0731e-05 s, 3.5 MB/s 2024-05-10T04:04:02.917 INFO:teuthology.run_tasks:Running task exec... 2024-05-10T04:04:02.927 INFO:teuthology.task.exec:Executing custom commands... 2024-05-10T04:04:02.927 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:04:02.927 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-10T04:04:03.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.058+0000 7fb41f472700 1 -- 172.21.15.150:0/533573264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4181010a0 msgr2=0x7fb4181014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:03.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.058+0000 7fb41f472700 1 --2- 172.21.15.150:0/533573264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4181010a0 0x7fb4181014b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb408009990 tx=0x7fb408009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:03.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.058+0000 7fb41f472700 1 -- 172.21.15.150:0/533573264 shutdown_connections 2024-05-10T04:04:03.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.058+0000 7fb41f472700 1 --2- 172.21.15.150:0/533573264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4181028e0 0x7fb4181091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:03.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.058+0000 7fb41f472700 1 --2- 172.21.15.150:0/533573264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4181010a0 0x7fb4181014b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:03.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.058+0000 7fb41f472700 1 --2- 172.21.15.150:0/533573264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb418101f90 0x7fb4181023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:03.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.058+0000 7fb41f472700 1 -- 172.21.15.150:0/533573264 >> 172.21.15.150:0/533573264 conn(0x7fb4180fb840 msgr2=0x7fb4180fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:03.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.058+0000 7fb41f472700 1 -- 172.21.15.150:0/533573264 shutdown_connections 2024-05-10T04:04:03.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.058+0000 7fb41f472700 1 -- 172.21.15.150:0/533573264 wait complete. 2024-05-10T04:04:03.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.059+0000 7fb41f472700 1 Processor -- start 2024-05-10T04:04:03.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.059+0000 7fb41f472700 1 -- start start 2024-05-10T04:04:03.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41f472700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4181010a0 0x7fb418197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:03.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41f472700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb418101f90 0x7fb4181976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:03.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41f472700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4181028e0 0x7fb41819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:03.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41f472700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41819ec20 con 0x7fb418101f90 2024-05-10T04:04:03.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41f472700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41819ed60 con 0x7fb4181028e0 2024-05-10T04:04:03.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41f472700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41819eef0 con 0x7fb4181010a0 2024-05-10T04:04:03.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41ca0d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb418101f90 0x7fb4181976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:03.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41ca0d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb418101f90 0x7fb4181976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35362/0 (socket says 172.21.15.150:35362) 2024-05-10T04:04:03.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41ca0d700 1 -- 172.21.15.150:0/604192199 learned_addr learned my addr 172.21.15.150:0/604192199 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:03.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41d20e700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4181010a0 0x7fb418197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41da0f700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4181028e0 0x7fb41819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41ca0d700 1 -- 172.21.15.150:0/604192199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4181010a0 msgr2=0x7fb418197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41ca0d700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4181010a0 0x7fb418197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41ca0d700 1 -- 172.21.15.150:0/604192199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4181028e0 msgr2=0x7fb41819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41ca0d700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4181028e0 0x7fb41819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41ca0d700 1 -- 172.21.15.150:0/604192199 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4080095f0 con 0x7fb418101f90 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41da0f700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4181028e0 0x7fb41819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41ca0d700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb418101f90 0x7fb4181976c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb4140099a0 tx=0x7fb414009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb40e7fc700 1 -- 172.21.15.150:0/604192199 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb414012070 con 0x7fb418101f90 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb40e7fc700 1 -- 172.21.15.150:0/604192199 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb41400d8f0 con 0x7fb418101f90 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41f472700 1 -- 172.21.15.150:0/604192199 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4180ffd50 con 0x7fb418101f90 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb40e7fc700 1 -- 172.21.15.150:0/604192199 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41400c9c0 con 0x7fb418101f90 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.060+0000 7fb41f472700 1 -- 172.21.15.150:0/604192199 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb418100260 con 0x7fb418101f90 2024-05-10T04:04:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.061+0000 7fb41f472700 1 -- 172.21.15.150:0/604192199 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb418067970 con 0x7fb418101f90 2024-05-10T04:04:03.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.063+0000 7fb40e7fc700 1 -- 172.21.15.150:0/604192199 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb414007c00 con 0x7fb418101f90 2024-05-10T04:04:03.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.063+0000 7fb40e7fc700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb4040a11b0 0x7fb4040a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:03.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.064+0000 7fb41d20e700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb4040a11b0 0x7fb4040a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:03.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.064+0000 7fb40e7fc700 1 -- 172.21.15.150:0/604192199 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb4140cdd10 con 0x7fb418101f90 2024-05-10T04:04:03.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.064+0000 7fb41d20e700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb4040a11b0 0x7fb4040a36c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb4080037f0 tx=0x7fb40801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:03.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.067+0000 7fb40e7fc700 1 -- 172.21.15.150:0/604192199 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb414097030 con 0x7fb418101f90 2024-05-10T04:04:03.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:03.314+0000 7fb41f472700 1 -- 172.21.15.150:0/604192199 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7fb418068430 con 0x7fb418101f90 2024-05-10T04:04:04.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.169+0000 7fb40e7fc700 1 -- 172.21.15.150:0/604192199 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb41409a650 con 0x7fb418101f90 2024-05-10T04:04:04.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.172+0000 7fb41f472700 1 -- 172.21.15.150:0/604192199 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb4040a11b0 msgr2=0x7fb4040a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:04.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.172+0000 7fb41f472700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb4040a11b0 0x7fb4040a36c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb4080037f0 tx=0x7fb40801a040 comp rx=0 tx=0).stop 2024-05-10T04:04:04.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.172+0000 7fb41f472700 1 -- 172.21.15.150:0/604192199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb418101f90 msgr2=0x7fb4181976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:04.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.172+0000 7fb41f472700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb418101f90 0x7fb4181976c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb4140099a0 tx=0x7fb414009cb0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.173+0000 7fb41f472700 1 -- 172.21.15.150:0/604192199 shutdown_connections 2024-05-10T04:04:04.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.173+0000 7fb41f472700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb4040a11b0 0x7fb4040a36c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.173+0000 7fb41f472700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4181028e0 0x7fb41819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.173+0000 7fb41f472700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4181010a0 0x7fb418197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.173+0000 7fb41f472700 1 --2- 172.21.15.150:0/604192199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb418101f90 0x7fb4181976c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.173+0000 7fb41f472700 1 -- 172.21.15.150:0/604192199 >> 172.21.15.150:0/604192199 conn(0x7fb4180fb840 msgr2=0x7fb4181051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:04.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.173+0000 7fb41f472700 1 -- 172.21.15.150:0/604192199 shutdown_connections 2024-05-10T04:04:04.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.173+0000 7fb41f472700 1 -- 172.21.15.150:0/604192199 wait complete. 2024-05-10T04:04:04.191 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-10T04:04:04.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.330+0000 7faf3df58700 1 -- 172.21.15.150:0/1639080947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf38107970 msgr2=0x7faf38109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.330+0000 7faf3df58700 1 --2- 172.21.15.150:0/1639080947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf38107970 0x7faf38109d90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faf3000b0a0 tx=0x7faf3000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.330+0000 7faf3df58700 1 -- 172.21.15.150:0/1639080947 shutdown_connections 2024-05-10T04:04:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.330+0000 7faf3df58700 1 --2- 172.21.15.150:0/1639080947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf38107970 0x7faf38109d90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.330+0000 7faf3df58700 1 --2- 172.21.15.150:0/1639080947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf381026b0 0x7faf38104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.330+0000 7faf3df58700 1 --2- 172.21.15.150:0/1639080947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf38105010 0x7faf38107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.330+0000 7faf3df58700 1 -- 172.21.15.150:0/1639080947 >> 172.21.15.150:0/1639080947 conn(0x7faf380fc0f0 msgr2=0x7faf380fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.330+0000 7faf3df58700 1 -- 172.21.15.150:0/1639080947 shutdown_connections 2024-05-10T04:04:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.331+0000 7faf3df58700 1 -- 172.21.15.150:0/1639080947 wait complete. 2024-05-10T04:04:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.331+0000 7faf3df58700 1 Processor -- start 2024-05-10T04:04:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.331+0000 7faf3df58700 1 -- start start 2024-05-10T04:04:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.331+0000 7faf3df58700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf381026b0 0x7faf381980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.331+0000 7faf3df58700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf38105010 0x7faf38198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.331+0000 7faf3df58700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf38107970 0x7faf3819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.331+0000 7faf3df58700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3819fb60 con 0x7faf38107970 2024-05-10T04:04:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.331+0000 7faf3df58700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3819fca0 con 0x7faf381026b0 2024-05-10T04:04:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.331+0000 7faf3df58700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3819fe30 con 0x7faf38105010 2024-05-10T04:04:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.331+0000 7faf37fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf38107970 0x7faf3819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.331+0000 7faf37fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf38107970 0x7faf3819f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35372/0 (socket says 172.21.15.150:35372) 2024-05-10T04:04:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf37fff700 1 -- 172.21.15.150:0/3709982769 learned_addr learned my addr 172.21.15.150:0/3709982769 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf377fe700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf381026b0 0x7faf381980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf2ffff700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf38105010 0x7faf38198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf37fff700 1 -- 172.21.15.150:0/3709982769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf38105010 msgr2=0x7faf38198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf37fff700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf38105010 0x7faf38198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf37fff700 1 -- 172.21.15.150:0/3709982769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf381026b0 msgr2=0x7faf381980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf37fff700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf381026b0 0x7faf381980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf37fff700 1 -- 172.21.15.150:0/3709982769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf30009d00 con 0x7faf38107970 2024-05-10T04:04:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf377fe700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf381026b0 0x7faf381980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf2ffff700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf38105010 0x7faf38198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf37fff700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf38107970 0x7faf3819f590 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faf3000b070 tx=0x7faf3000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf357fa700 1 -- 172.21.15.150:0/3709982769 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3000e040 con 0x7faf38107970 2024-05-10T04:04:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf357fa700 1 -- 172.21.15.150:0/3709982769 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf30004c50 con 0x7faf38107970 2024-05-10T04:04:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf357fa700 1 -- 172.21.15.150:0/3709982769 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30005280 con 0x7faf38107970 2024-05-10T04:04:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf3df58700 1 -- 172.21.15.150:0/3709982769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf381a00b0 con 0x7faf38107970 2024-05-10T04:04:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.332+0000 7faf3df58700 1 -- 172.21.15.150:0/3709982769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf381a0530 con 0x7faf38107970 2024-05-10T04:04:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.333+0000 7faf3df58700 1 -- 172.21.15.150:0/3709982769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf380fdca0 con 0x7faf38107970 2024-05-10T04:04:04.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.335+0000 7faf357fa700 1 -- 172.21.15.150:0/3709982769 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7faf300053e0 con 0x7faf38107970 2024-05-10T04:04:04.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.335+0000 7faf357fa700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faf240a1230 0x7faf240a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:04.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.335+0000 7faf377fe700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faf240a1230 0x7faf240a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:04.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.335+0000 7faf357fa700 1 -- 172.21.15.150:0/3709982769 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7faf300ce5f0 con 0x7faf38107970 2024-05-10T04:04:04.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.336+0000 7faf377fe700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faf240a1230 0x7faf240a3740 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faf38069070 tx=0x7faf28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:04.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.339+0000 7faf357fa700 1 -- 172.21.15.150:0/3709982769 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faf300979a0 con 0x7faf38107970 2024-05-10T04:04:04.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:04 smithi150 ceph-mon[42477]: pgmap v274: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:04:04.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:04 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/604192199' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-10T04:04:04.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:04 smithi181 ceph-mon[46602]: pgmap v274: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:04:04.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:04 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/604192199' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-10T04:04:04.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:04 smithi161 ceph-mon[46918]: pgmap v274: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:04:04.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:04 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/604192199' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-10T04:04:04.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.569+0000 7faf3df58700 1 -- 172.21.15.150:0/3709982769 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7faf3802ce90 con 0x7faf240a1230 2024-05-10T04:04:04.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.574+0000 7faf357fa700 1 -- 172.21.15.150:0/3709982769 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7faf3802ce90 con 0x7faf240a1230 2024-05-10T04:04:04.576 INFO:teuthology.orchestra.run.smithi150.stdout:{} 2024-05-10T04:04:04.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.576+0000 7faf3df58700 1 -- 172.21.15.150:0/3709982769 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faf240a1230 msgr2=0x7faf240a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:04.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.576+0000 7faf3df58700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faf240a1230 0x7faf240a3740 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faf38069070 tx=0x7faf28009260 comp rx=0 tx=0).stop 2024-05-10T04:04:04.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.576+0000 7faf3df58700 1 -- 172.21.15.150:0/3709982769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf38107970 msgr2=0x7faf3819f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:04.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.576+0000 7faf3df58700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf38107970 0x7faf3819f590 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faf3000b070 tx=0x7faf3000bab0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.576+0000 7faf3df58700 1 -- 172.21.15.150:0/3709982769 shutdown_connections 2024-05-10T04:04:04.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.576+0000 7faf3df58700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faf240a1230 0x7faf240a3740 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.576+0000 7faf3df58700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf38107970 0x7faf3819f590 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.576+0000 7faf3df58700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf381026b0 0x7faf381980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.576+0000 7faf3df58700 1 --2- 172.21.15.150:0/3709982769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf38105010 0x7faf38198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.576+0000 7faf3df58700 1 -- 172.21.15.150:0/3709982769 >> 172.21.15.150:0/3709982769 conn(0x7faf380fc0f0 msgr2=0x7faf381034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:04.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.577+0000 7faf3df58700 1 -- 172.21.15.150:0/3709982769 shutdown_connections 2024-05-10T04:04:04.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.577+0000 7faf3df58700 1 -- 172.21.15.150:0/3709982769 wait complete. 2024-05-10T04:04:04.592 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-10T04:04:04.602 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-10T04:04:04.627 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:04:04.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.758+0000 7f0fe238a700 1 -- 172.21.15.150:0/2744325264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc101a50 msgr2=0x7f0fdc101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:04.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.758+0000 7f0fe238a700 1 --2- 172.21.15.150:0/2744325264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc101a50 0x7f0fdc101f20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009990 tx=0x7f0fcc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.758+0000 7f0fe238a700 1 -- 172.21.15.150:0/2744325264 shutdown_connections 2024-05-10T04:04:04.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.758+0000 7f0fe238a700 1 --2- 172.21.15.150:0/2744325264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fdc1025b0 0x7f0fdc109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.758+0000 7f0fe238a700 1 --2- 172.21.15.150:0/2744325264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fdc101070 0x7f0fdc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.758+0000 7f0fe238a700 1 --2- 172.21.15.150:0/2744325264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc101a50 0x7f0fdc101f20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.758+0000 7f0fe238a700 1 -- 172.21.15.150:0/2744325264 >> 172.21.15.150:0/2744325264 conn(0x7f0fdc0fb8b0 msgr2=0x7f0fdc0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:04.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.758+0000 7f0fe238a700 1 -- 172.21.15.150:0/2744325264 shutdown_connections 2024-05-10T04:04:04.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.758+0000 7f0fe238a700 1 -- 172.21.15.150:0/2744325264 wait complete. 2024-05-10T04:04:04.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe238a700 1 Processor -- start 2024-05-10T04:04:04.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe238a700 1 -- start start 2024-05-10T04:04:04.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe238a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fdc101070 0x7f0fdc100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:04.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe238a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fdc101a50 0x7f0fdc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe238a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc1025b0 0x7f0fdc19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe238a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc19fac0 con 0x7f0fdc1025b0 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe238a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc19fc30 con 0x7f0fdc101070 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe238a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc19fdf0 con 0x7f0fdc101a50 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe0927700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc1025b0 0x7f0fdc19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe0927700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc1025b0 0x7f0fdc19f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35392/0 (socket says 172.21.15.150:35392) 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe0927700 1 -- 172.21.15.150:0/4249032174 learned_addr learned my addr 172.21.15.150:0/4249032174 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fdbfff700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fdc101070 0x7f0fdc100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fdb7fe700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fdc101a50 0x7f0fdc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe0927700 1 -- 172.21.15.150:0/4249032174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fdc101a50 msgr2=0x7f0fdc19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe0927700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fdc101a50 0x7f0fdc19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe0927700 1 -- 172.21.15.150:0/4249032174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fdc101070 msgr2=0x7f0fdc100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe0927700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fdc101070 0x7f0fdc100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.759+0000 7f0fe0927700 1 -- 172.21.15.150:0/4249032174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc0095f0 con 0x7f0fdc1025b0 2024-05-10T04:04:04.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.760+0000 7f0fe0927700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc1025b0 0x7f0fdc19f370 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4007ee0 tx=0x7f0fd400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:04.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.760+0000 7f0fd97fa700 1 -- 172.21.15.150:0/4249032174 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4017070 con 0x7f0fdc1025b0 2024-05-10T04:04:04.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.760+0000 7f0fd97fa700 1 -- 172.21.15.150:0/4249032174 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fd400f040 con 0x7f0fdc1025b0 2024-05-10T04:04:04.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.760+0000 7f0fe238a700 1 -- 172.21.15.150:0/4249032174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fdc1a0070 con 0x7f0fdc1025b0 2024-05-10T04:04:04.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.760+0000 7f0fd97fa700 1 -- 172.21.15.150:0/4249032174 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4005740 con 0x7f0fdc1025b0 2024-05-10T04:04:04.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.760+0000 7f0fe238a700 1 -- 172.21.15.150:0/4249032174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc1a0530 con 0x7f0fdc1025b0 2024-05-10T04:04:04.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.761+0000 7f0fe238a700 1 -- 172.21.15.150:0/4249032174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fdc107ed0 con 0x7f0fdc1025b0 2024-05-10T04:04:04.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.762+0000 7f0fd97fa700 1 -- 172.21.15.150:0/4249032174 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0fd40058a0 con 0x7f0fdc1025b0 2024-05-10T04:04:04.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.762+0000 7f0fd97fa700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0fc40a11b0 0x7f0fc40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:04.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.763+0000 7f0fdbfff700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0fc40a11b0 0x7f0fc40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:04.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.763+0000 7f0fd97fa700 1 -- 172.21.15.150:0/4249032174 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0fd40cc5c0 con 0x7f0fdc1025b0 2024-05-10T04:04:04.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.763+0000 7f0fdbfff700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0fc40a11b0 0x7f0fc40a36c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0698a0 tx=0x7f0fd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:04.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.767+0000 7f0fd97fa700 1 -- 172.21.15.150:0/4249032174 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0fd40958e0 con 0x7f0fdc1025b0 2024-05-10T04:04:04.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.964+0000 7f0fe238a700 1 -- 172.21.15.150:0/4249032174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fdc1078b0 con 0x7f0fdc1025b0 2024-05-10T04:04:04.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.966+0000 7f0fd97fa700 1 -- 172.21.15.150:0/4249032174 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24554 (secure 0 0 0) 0x7f0fd4098f00 con 0x7f0fdc1025b0 2024-05-10T04:04:04.968 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:04:04.969 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":87,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:03:30.185138+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:04:04.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.968+0000 7f0fe238a700 1 -- 172.21.15.150:0/4249032174 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0fc40a11b0 msgr2=0x7f0fc40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:04.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.968+0000 7f0fe238a700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0fc40a11b0 0x7f0fc40a36c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0698a0 tx=0x7f0fd0009260 comp rx=0 tx=0).stop 2024-05-10T04:04:04.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.968+0000 7f0fe238a700 1 -- 172.21.15.150:0/4249032174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc1025b0 msgr2=0x7f0fdc19f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:04.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.968+0000 7f0fe238a700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc1025b0 0x7f0fdc19f370 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4007ee0 tx=0x7f0fd400d3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.968+0000 7f0fe238a700 1 -- 172.21.15.150:0/4249032174 shutdown_connections 2024-05-10T04:04:04.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.968+0000 7f0fe238a700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0fc40a11b0 0x7f0fc40a36c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.968+0000 7f0fe238a700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc1025b0 0x7f0fdc19f370 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.968+0000 7f0fe238a700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fdc101070 0x7f0fdc100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.968+0000 7f0fe238a700 1 --2- 172.21.15.150:0/4249032174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fdc101a50 0x7f0fdc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:04.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.968+0000 7f0fe238a700 1 -- 172.21.15.150:0/4249032174 >> 172.21.15.150:0/4249032174 conn(0x7f0fdc0fb8b0 msgr2=0x7f0fdc105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:04.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.968+0000 7f0fe238a700 1 -- 172.21.15.150:0/4249032174 shutdown_connections 2024-05-10T04:04:04.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:04.968+0000 7f0fe238a700 1 -- 172.21.15.150:0/4249032174 wait complete. 2024-05-10T04:04:04.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:02:30.076694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:02:30.760710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:02:33.850714+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:04:04.985 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:04:05.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.119+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/12271975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8cc100db0 msgr2=0x7fc8cc1011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:05.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.119+0000 7fc8d1c4e700 1 --2- 172.21.15.150:0/12271975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8cc100db0 0x7fc8cc1011c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009990 tx=0x7fc8c0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.119+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/12271975 shutdown_connections 2024-05-10T04:04:05.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.119+0000 7fc8d1c4e700 1 --2- 172.21.15.150:0/12271975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8cc101700 0x7fc8cc109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.119+0000 7fc8d1c4e700 1 --2- 172.21.15.150:0/12271975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8cc1020a0 0x7fc8cc1024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.119+0000 7fc8d1c4e700 1 --2- 172.21.15.150:0/12271975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8cc100db0 0x7fc8cc1011c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.119+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/12271975 >> 172.21.15.150:0/12271975 conn(0x7fc8cc0fb8f0 msgr2=0x7fc8cc0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:05.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.119+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/12271975 shutdown_connections 2024-05-10T04:04:05.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.119+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/12271975 wait complete. 2024-05-10T04:04:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.120+0000 7fc8d1c4e700 1 Processor -- start 2024-05-10T04:04:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.120+0000 7fc8d1c4e700 1 -- start start 2024-05-10T04:04:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.120+0000 7fc8d1c4e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8cc100db0 0x7fc8cc100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.120+0000 7fc8d1c4e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8cc101700 0x7fc8cc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.120+0000 7fc8d1c4e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8cc1020a0 0x7fc8cc19f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.120+0000 7fc8d1c4e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc19f9e0 con 0x7fc8cc101700 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.120+0000 7fc8d1c4e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc19fb50 con 0x7fc8cc1020a0 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.120+0000 7fc8d1c4e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc19fd10 con 0x7fc8cc100db0 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.120+0000 7fc8caffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8cc101700 0x7fc8cc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.120+0000 7fc8caffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8cc101700 0x7fc8cc19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35416/0 (socket says 172.21.15.150:35416) 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.120+0000 7fc8caffd700 1 -- 172.21.15.150:0/670414006 learned_addr learned my addr 172.21.15.150:0/670414006 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.120+0000 7fc8cbfff700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8cc1020a0 0x7fc8cc19f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.120+0000 7fc8cb7fe700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8cc100db0 0x7fc8cc100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.121+0000 7fc8caffd700 1 -- 172.21.15.150:0/670414006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8cc100db0 msgr2=0x7fc8cc100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.121+0000 7fc8caffd700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8cc100db0 0x7fc8cc100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.121+0000 7fc8caffd700 1 -- 172.21.15.150:0/670414006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8cc1020a0 msgr2=0x7fc8cc19f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.121+0000 7fc8caffd700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8cc1020a0 0x7fc8cc19f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.121+0000 7fc8caffd700 1 -- 172.21.15.150:0/670414006 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c00095f0 con 0x7fc8cc101700 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.121+0000 7fc8cb7fe700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8cc100db0 0x7fc8cc100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.121+0000 7fc8caffd700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8cc101700 0x7fc8cc19cc70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00095c0 tx=0x7fc8c000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.121+0000 7fc8c8ff9700 1 -- 172.21.15.150:0/670414006 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c001d070 con 0x7fc8cc101700 2024-05-10T04:04:05.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.121+0000 7fc8c8ff9700 1 -- 172.21.15.150:0/670414006 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8c0022470 con 0x7fc8cc101700 2024-05-10T04:04:05.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.121+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/670414006 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8cc19ff90 con 0x7fc8cc101700 2024-05-10T04:04:05.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.121+0000 7fc8c8ff9700 1 -- 172.21.15.150:0/670414006 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0017650 con 0x7fc8cc101700 2024-05-10T04:04:05.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.121+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/670414006 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc1a0300 con 0x7fc8cc101700 2024-05-10T04:04:05.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.122+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/670414006 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8cc107eb0 con 0x7fc8cc101700 2024-05-10T04:04:05.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.124+0000 7fc8c8ff9700 1 -- 172.21.15.150:0/670414006 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc8c00225e0 con 0x7fc8cc101700 2024-05-10T04:04:05.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.125+0000 7fc8c8ff9700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc8b40a1260 0x7fc8b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:05.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.125+0000 7fc8c8ff9700 1 -- 172.21.15.150:0/670414006 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc8c00ce4a0 con 0x7fc8cc101700 2024-05-10T04:04:05.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.125+0000 7fc8cb7fe700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc8b40a1260 0x7fc8b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:05.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.125+0000 7fc8cb7fe700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc8b40a1260 0x7fc8b40a3770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc005fd0 tx=0x7fc8bc005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:05.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.128+0000 7fc8c8ff9700 1 -- 172.21.15.150:0/670414006 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc8c0097880 con 0x7fc8cc101700 2024-05-10T04:04:05.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.328+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/670414006 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8cc10c4a0 con 0x7fc8cc101700 2024-05-10T04:04:05.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.329+0000 7fc8c8ff9700 1 -- 172.21.15.150:0/670414006 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8c009aea0 con 0x7fc8cc101700 2024-05-10T04:04:05.332 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:04:05.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.331+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/670414006 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc8b40a1260 msgr2=0x7fc8b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:05.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.331+0000 7fc8d1c4e700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc8b40a1260 0x7fc8b40a3770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc005fd0 tx=0x7fc8bc005e40 comp rx=0 tx=0).stop 2024-05-10T04:04:05.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.331+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/670414006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8cc101700 msgr2=0x7fc8cc19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:05.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.331+0000 7fc8d1c4e700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8cc101700 0x7fc8cc19cc70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00095c0 tx=0x7fc8c000f690 comp rx=0 tx=0).stop 2024-05-10T04:04:05.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.331+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/670414006 shutdown_connections 2024-05-10T04:04:05.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.331+0000 7fc8d1c4e700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc8b40a1260 0x7fc8b40a3770 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.331+0000 7fc8d1c4e700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8cc1020a0 0x7fc8cc19f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.331+0000 7fc8d1c4e700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8cc100db0 0x7fc8cc100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.331+0000 7fc8d1c4e700 1 --2- 172.21.15.150:0/670414006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8cc101700 0x7fc8cc19cc70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.331+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/670414006 >> 172.21.15.150:0/670414006 conn(0x7fc8cc0fb8f0 msgr2=0x7fc8cc104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:05.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.331+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/670414006 shutdown_connections 2024-05-10T04:04:05.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.331+0000 7fc8d1c4e700 1 -- 172.21.15.150:0/670414006 wait complete. 2024-05-10T04:04:05.345 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:05 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/604192199' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-10T04:04:05.346 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:05 smithi150 ceph-mon[42477]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T04:04:05.346 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:05 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4249032174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:04:05.347 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:04:05.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.480+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2105485496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a081025a0 msgr2=0x7f2a08108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:05.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.480+0000 7f2a0fbef700 1 --2- 172.21.15.150:0/2105485496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a081025a0 0x7f2a08108e50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000b0a0 tx=0x7f2a0000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.480+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2105485496 shutdown_connections 2024-05-10T04:04:05.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.480+0000 7f2a0fbef700 1 --2- 172.21.15.150:0/2105485496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a081025a0 0x7f2a08108e50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.480+0000 7f2a0fbef700 1 --2- 172.21.15.150:0/2105485496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a08100ae0 0x7f2a08100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.480+0000 7f2a0fbef700 1 --2- 172.21.15.150:0/2105485496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a08101c00 0x7f2a08102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.480+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2105485496 >> 172.21.15.150:0/2105485496 conn(0x7f2a080fc0f0 msgr2=0x7f2a080fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:05.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.480+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2105485496 shutdown_connections 2024-05-10T04:04:05.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.481+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2105485496 wait complete. 2024-05-10T04:04:05.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.481+0000 7f2a0fbef700 1 Processor -- start 2024-05-10T04:04:05.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.481+0000 7f2a0fbef700 1 -- start start 2024-05-10T04:04:05.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.481+0000 7f2a0fbef700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a08100ae0 0x7f2a0806a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:05.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.481+0000 7f2a0fbef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a08101c00 0x7f2a0819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:05.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.481+0000 7f2a0fbef700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a081025a0 0x7f2a0819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.481+0000 7f2a0fbef700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0819fb30 con 0x7f2a08100ae0 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.481+0000 7f2a0fbef700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0819fca0 con 0x7f2a081025a0 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.481+0000 7f2a0fbef700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0819fe60 con 0x7f2a08101c00 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.481+0000 7f2a0d98b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a08100ae0 0x7f2a0806a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.481+0000 7f2a0d98b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a08100ae0 0x7f2a0806a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35430/0 (socket says 172.21.15.150:35430) 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.481+0000 7f2a0d98b700 1 -- 172.21.15.150:0/2824442184 learned_addr learned my addr 172.21.15.150:0/2824442184 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f2a0e18c700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a081025a0 0x7f2a0819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f2a0d18a700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a08101c00 0x7f2a0819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f2a0d98b700 1 -- 172.21.15.150:0/2824442184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a08101c00 msgr2=0x7f2a0819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f2a0d98b700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a08101c00 0x7f2a0819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f2a0d98b700 1 -- 172.21.15.150:0/2824442184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a081025a0 msgr2=0x7f2a0819f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f2a0d98b700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a081025a0 0x7f2a0819f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f2a0d98b700 1 -- 172.21.15.150:0/2824442184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a00009d00 con 0x7f2a08100ae0 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f2a0d18a700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a08101c00 0x7f2a0819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:05.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f2a0d98b700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a08100ae0 0x7f2a0806a570 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400d950 tx=0x7f2a0400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:05.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f29faffd700 1 -- 172.21.15.150:0/2824442184 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04004220 con 0x7f2a08100ae0 2024-05-10T04:04:05.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f29faffd700 1 -- 172.21.15.150:0/2824442184 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a04004380 con 0x7f2a08100ae0 2024-05-10T04:04:05.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2824442184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a081a00e0 con 0x7f2a08100ae0 2024-05-10T04:04:05.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f29faffd700 1 -- 172.21.15.150:0/2824442184 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04003bf0 con 0x7f2a08100ae0 2024-05-10T04:04:05.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.482+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2824442184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a081a05f0 con 0x7f2a08100ae0 2024-05-10T04:04:05.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.483+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2824442184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a08107cf0 con 0x7f2a08100ae0 2024-05-10T04:04:05.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:05 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/604192199' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-10T04:04:05.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:05 smithi181 ceph-mon[46602]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T04:04:05.489 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:05 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4249032174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:04:05.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.484+0000 7f29faffd700 1 -- 172.21.15.150:0/2824442184 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f2a04003d50 con 0x7f2a08100ae0 2024-05-10T04:04:05.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.485+0000 7f29faffd700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f29f40a1260 0x7f29f40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:05.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.485+0000 7f2a0d18a700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f29f40a1260 0x7f29f40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:05.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.485+0000 7f29faffd700 1 -- 172.21.15.150:0/2824442184 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2a040cd070 con 0x7f2a08100ae0 2024-05-10T04:04:05.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.485+0000 7f2a0d18a700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f29f40a1260 0x7f29f40a3770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2a08069f10 tx=0x7f29fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:05.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.489+0000 7f29faffd700 1 -- 172.21.15.150:0/2824442184 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2a04096330 con 0x7f2a08100ae0 2024-05-10T04:04:05.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:05 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/604192199' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-10T04:04:05.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:05 smithi161 ceph-mon[46918]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T04:04:05.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:05 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4249032174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:04:05.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.687+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2824442184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a0810c2f0 con 0x7f2a08100ae0 2024-05-10T04:04:05.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.687+0000 7f29faffd700 1 -- 172.21.15.150:0/2824442184 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a040170a0 con 0x7f2a08100ae0 2024-05-10T04:04:05.690 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:04:05.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.690+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2824442184 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f29f40a1260 msgr2=0x7f29f40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:05.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.690+0000 7f2a0fbef700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f29f40a1260 0x7f29f40a3770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2a08069f10 tx=0x7f29fc009260 comp rx=0 tx=0).stop 2024-05-10T04:04:05.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.690+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2824442184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a08100ae0 msgr2=0x7f2a0806a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:05.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.690+0000 7f2a0fbef700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a08100ae0 0x7f2a0806a570 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400d950 tx=0x7f2a0400dc60 comp rx=0 tx=0).stop 2024-05-10T04:04:05.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.690+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2824442184 shutdown_connections 2024-05-10T04:04:05.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.690+0000 7f2a0fbef700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f29f40a1260 0x7f29f40a3770 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.690+0000 7f2a0fbef700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a081025a0 0x7f2a0819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.690+0000 7f2a0fbef700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a08100ae0 0x7f2a0806a570 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.690+0000 7f2a0fbef700 1 --2- 172.21.15.150:0/2824442184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a08101c00 0x7f2a0819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.690+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2824442184 >> 172.21.15.150:0/2824442184 conn(0x7f2a080fc0f0 msgr2=0x7f2a08104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:05.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.690+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2824442184 shutdown_connections 2024-05-10T04:04:05.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.690+0000 7f2a0fbef700 1 -- 172.21.15.150:0/2824442184 wait complete. 2024-05-10T04:04:05.705 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:04:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.839+0000 7fe252a1e700 1 -- 172.21.15.150:0/403403228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe24c0ffbe0 msgr2=0x7fe24c0ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.839+0000 7fe252a1e700 1 --2- 172.21.15.150:0/403403228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe24c0ffbe0 0x7fe24c0ffff0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe240009990 tx=0x7fe240009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.839+0000 7fe252a1e700 1 -- 172.21.15.150:0/403403228 shutdown_connections 2024-05-10T04:04:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.839+0000 7fe252a1e700 1 --2- 172.21.15.150:0/403403228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe24c100530 0x7fe24c10a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.839+0000 7fe252a1e700 1 --2- 172.21.15.150:0/403403228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe24c1031a0 0x7fe24c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.839+0000 7fe252a1e700 1 --2- 172.21.15.150:0/403403228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe24c0ffbe0 0x7fe24c0ffff0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.839+0000 7fe252a1e700 1 -- 172.21.15.150:0/403403228 >> 172.21.15.150:0/403403228 conn(0x7fe24c0fb8b0 msgr2=0x7fe24c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.839+0000 7fe252a1e700 1 -- 172.21.15.150:0/403403228 shutdown_connections 2024-05-10T04:04:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.839+0000 7fe252a1e700 1 -- 172.21.15.150:0/403403228 wait complete. 2024-05-10T04:04:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.840+0000 7fe252a1e700 1 Processor -- start 2024-05-10T04:04:05.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.840+0000 7fe252a1e700 1 -- start start 2024-05-10T04:04:05.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.840+0000 7fe252a1e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe24c0ffbe0 0x7fe24c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:05.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.840+0000 7fe252a1e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe24c100530 0x7fe24c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:05.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.840+0000 7fe252a1e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe24c1031a0 0x7fe24c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:05.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.840+0000 7fe252a1e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c19f9c0 con 0x7fe24c1031a0 2024-05-10T04:04:05.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.840+0000 7fe252a1e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c19fb30 con 0x7fe24c0ffbe0 2024-05-10T04:04:05.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.840+0000 7fe252a1e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c19fcf0 con 0x7fe24c100530 2024-05-10T04:04:05.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.840+0000 7fe250fbb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe24c1031a0 0x7fe24c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:05.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.840+0000 7fe250fbb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe24c1031a0 0x7fe24c19f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35440/0 (socket says 172.21.15.150:35440) 2024-05-10T04:04:05.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.840+0000 7fe250fbb700 1 -- 172.21.15.150:0/1743636886 learned_addr learned my addr 172.21.15.150:0/1743636886 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:05.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.840+0000 7fe24bfff700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe24c0ffbe0 0x7fe24c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.841+0000 7fe24b7fe700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe24c100530 0x7fe24c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.841+0000 7fe250fbb700 1 -- 172.21.15.150:0/1743636886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe24c100530 msgr2=0x7fe24c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.841+0000 7fe250fbb700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe24c100530 0x7fe24c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.841+0000 7fe250fbb700 1 -- 172.21.15.150:0/1743636886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe24c0ffbe0 msgr2=0x7fe24c197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.841+0000 7fe250fbb700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe24c0ffbe0 0x7fe24c197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.841+0000 7fe250fbb700 1 -- 172.21.15.150:0/1743636886 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400095f0 con 0x7fe24c1031a0 2024-05-10T04:04:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.841+0000 7fe250fbb700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe24c1031a0 0x7fe24c19f3b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe244007ee0 tx=0x7fe24400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.841+0000 7fe2497fa700 1 -- 172.21.15.150:0/1743636886 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe244017070 con 0x7fe24c1031a0 2024-05-10T04:04:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.841+0000 7fe2497fa700 1 -- 172.21.15.150:0/1743636886 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe24400f040 con 0x7fe24c1031a0 2024-05-10T04:04:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.841+0000 7fe252a1e700 1 -- 172.21.15.150:0/1743636886 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24c19ffd0 con 0x7fe24c1031a0 2024-05-10T04:04:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.841+0000 7fe2497fa700 1 -- 172.21.15.150:0/1743636886 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe244005740 con 0x7fe24c1031a0 2024-05-10T04:04:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.841+0000 7fe252a1e700 1 -- 172.21.15.150:0/1743636886 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24c1a0400 con 0x7fe24c1031a0 2024-05-10T04:04:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.842+0000 7fe252a1e700 1 -- 172.21.15.150:0/1743636886 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24c108ff0 con 0x7fe24c1031a0 2024-05-10T04:04:05.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.844+0000 7fe2497fa700 1 -- 172.21.15.150:0/1743636886 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe24400dc60 con 0x7fe24c1031a0 2024-05-10T04:04:05.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.844+0000 7fe2497fa700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe2340a1260 0x7fe2340a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:05.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.845+0000 7fe24bfff700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe2340a1260 0x7fe2340a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:05.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.845+0000 7fe2497fa700 1 -- 172.21.15.150:0/1743636886 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe2440ccc40 con 0x7fe24c1031a0 2024-05-10T04:04:05.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.845+0000 7fe24bfff700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe2340a1260 0x7fe2340a3770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0698c0 tx=0x7fe23c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:05.848+0000 7fe2497fa700 1 -- 172.21.15.150:0/1743636886 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe244095f60 con 0x7fe24c1031a0 2024-05-10T04:04:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.047+0000 7fe252a1e700 1 -- 172.21.15.150:0/1743636886 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe24c068730 con 0x7fe24c1031a0 2024-05-10T04:04:06.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.048+0000 7fe2497fa700 1 -- 172.21.15.150:0/1743636886 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe244099580 con 0x7fe24c1031a0 2024-05-10T04:04:06.050 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:04:06.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.050+0000 7fe252a1e700 1 -- 172.21.15.150:0/1743636886 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe2340a1260 msgr2=0x7fe2340a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.050+0000 7fe252a1e700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe2340a1260 0x7fe2340a3770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0698c0 tx=0x7fe23c00a380 comp rx=0 tx=0).stop 2024-05-10T04:04:06.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.050+0000 7fe252a1e700 1 -- 172.21.15.150:0/1743636886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe24c1031a0 msgr2=0x7fe24c19f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.050+0000 7fe252a1e700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe24c1031a0 0x7fe24c19f3b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe244007ee0 tx=0x7fe24400d3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.050+0000 7fe252a1e700 1 -- 172.21.15.150:0/1743636886 shutdown_connections 2024-05-10T04:04:06.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.050+0000 7fe252a1e700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe2340a1260 0x7fe2340a3770 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.050+0000 7fe252a1e700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe24c1031a0 0x7fe24c19f3b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.050+0000 7fe252a1e700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe24c0ffbe0 0x7fe24c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.050+0000 7fe252a1e700 1 --2- 172.21.15.150:0/1743636886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe24c100530 0x7fe24c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.050+0000 7fe252a1e700 1 -- 172.21.15.150:0/1743636886 >> 172.21.15.150:0/1743636886 conn(0x7fe24c0fb8b0 msgr2=0x7fe24c106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:06.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.050+0000 7fe252a1e700 1 -- 172.21.15.150:0/1743636886 shutdown_connections 2024-05-10T04:04:06.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.050+0000 7fe252a1e700 1 -- 172.21.15.150:0/1743636886 wait complete. 2024-05-10T04:04:06.065 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:04:06.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.201+0000 7fb8ad401700 1 -- 172.21.15.150:0/3441158473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a8100570 msgr2=0x7fb8a810a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.201+0000 7fb8ad401700 1 --2- 172.21.15.150:0/3441158473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a8100570 0x7fb8a810a190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000b0a0 tx=0x7fb8a000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.201+0000 7fb8ad401700 1 -- 172.21.15.150:0/3441158473 shutdown_connections 2024-05-10T04:04:06.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.201+0000 7fb8ad401700 1 --2- 172.21.15.150:0/3441158473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a8100570 0x7fb8a810a190 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.201+0000 7fb8ad401700 1 --2- 172.21.15.150:0/3441158473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8a81031e0 0x7fb8a81035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.201+0000 7fb8ad401700 1 --2- 172.21.15.150:0/3441158473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a80ffc20 0x7fb8a8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.201+0000 7fb8ad401700 1 -- 172.21.15.150:0/3441158473 >> 172.21.15.150:0/3441158473 conn(0x7fb8a80fb8f0 msgr2=0x7fb8a80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:06.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.201+0000 7fb8ad401700 1 -- 172.21.15.150:0/3441158473 shutdown_connections 2024-05-10T04:04:06.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.201+0000 7fb8ad401700 1 -- 172.21.15.150:0/3441158473 wait complete. 2024-05-10T04:04:06.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.202+0000 7fb8ad401700 1 Processor -- start 2024-05-10T04:04:06.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.202+0000 7fb8ad401700 1 -- start start 2024-05-10T04:04:06.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.202+0000 7fb8ad401700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a80ffc20 0x7fb8a81027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:06.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.202+0000 7fb8ad401700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a8100570 0x7fb8a8100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:06.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.202+0000 7fb8ad401700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8a81031e0 0x7fb8a8101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:06.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8ad401700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a81019f0 con 0x7fb8a80ffc20 2024-05-10T04:04:06.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8ad401700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a8101b60 con 0x7fb8a81031e0 2024-05-10T04:04:06.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8ad401700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a8101d20 con 0x7fb8a8100570 2024-05-10T04:04:06.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8a6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a80ffc20 0x7fb8a81027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:06.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8a6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a80ffc20 0x7fb8a81027b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35456/0 (socket says 172.21.15.150:35456) 2024-05-10T04:04:06.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8a6ffd700 1 -- 172.21.15.150:0/3014369128 learned_addr learned my addr 172.21.15.150:0/3014369128 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:06.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8a77fe700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8a81031e0 0x7fb8a8101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:06.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8a67fc700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a8100570 0x7fb8a8100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:06.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8a6ffd700 1 -- 172.21.15.150:0/3014369128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a8100570 msgr2=0x7fb8a8100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8a6ffd700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a8100570 0x7fb8a8100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8a6ffd700 1 -- 172.21.15.150:0/3014369128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8a81031e0 msgr2=0x7fb8a8101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8a6ffd700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8a81031e0 0x7fb8a8101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8a6ffd700 1 -- 172.21.15.150:0/3014369128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a0009d00 con 0x7fb8a80ffc20 2024-05-10T04:04:06.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8a67fc700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a8100570 0x7fb8a8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:06.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8a6ffd700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a80ffc20 0x7fb8a81027b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb89800d950 tx=0x7fb89800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:06.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb89ffff700 1 -- 172.21.15.150:0/3014369128 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898004220 con 0x7fb8a80ffc20 2024-05-10T04:04:06.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb89ffff700 1 -- 172.21.15.150:0/3014369128 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb898004380 con 0x7fb8a80ffc20 2024-05-10T04:04:06.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8ad401700 1 -- 172.21.15.150:0/3014369128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a81a2250 con 0x7fb8a80ffc20 2024-05-10T04:04:06.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb89ffff700 1 -- 172.21.15.150:0/3014369128 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898003bf0 con 0x7fb8a80ffc20 2024-05-10T04:04:06.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.203+0000 7fb8ad401700 1 -- 172.21.15.150:0/3014369128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a81a2760 con 0x7fb8a80ffc20 2024-05-10T04:04:06.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.204+0000 7fb8ad401700 1 -- 172.21.15.150:0/3014369128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a8109030 con 0x7fb8a80ffc20 2024-05-10T04:04:06.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.206+0000 7fb89ffff700 1 -- 172.21.15.150:0/3014369128 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb898003d50 con 0x7fb8a80ffc20 2024-05-10T04:04:06.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.206+0000 7fb89ffff700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb8940a1260 0x7fb8940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:06.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.206+0000 7fb89ffff700 1 -- 172.21.15.150:0/3014369128 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb8980ccf60 con 0x7fb8a80ffc20 2024-05-10T04:04:06.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.206+0000 7fb8a67fc700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb8940a1260 0x7fb8940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:06.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.207+0000 7fb8a67fc700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb8940a1260 0x7fb8940a3770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8102150 tx=0x7fb89000a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:06.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.210+0000 7fb89ffff700 1 -- 172.21.15.150:0/3014369128 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb898096280 con 0x7fb8a80ffc20 2024-05-10T04:04:06.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.405+0000 7fb8ad401700 1 -- 172.21.15.150:0/3014369128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb8a810d3d0 con 0x7fb8a80ffc20 2024-05-10T04:04:06.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.405+0000 7fb89ffff700 1 -- 172.21.15.150:0/3014369128 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb898017020 con 0x7fb8a80ffc20 2024-05-10T04:04:06.408 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:04:06.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.407+0000 7fb8ad401700 1 -- 172.21.15.150:0/3014369128 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb8940a1260 msgr2=0x7fb8940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.407+0000 7fb8ad401700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb8940a1260 0x7fb8940a3770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8102150 tx=0x7fb89000a680 comp rx=0 tx=0).stop 2024-05-10T04:04:06.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.408+0000 7fb8ad401700 1 -- 172.21.15.150:0/3014369128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a80ffc20 msgr2=0x7fb8a81027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.408+0000 7fb8ad401700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a80ffc20 0x7fb8a81027b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb89800d950 tx=0x7fb89800dc60 comp rx=0 tx=0).stop 2024-05-10T04:04:06.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.408+0000 7fb8ad401700 1 -- 172.21.15.150:0/3014369128 shutdown_connections 2024-05-10T04:04:06.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.408+0000 7fb8ad401700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb8940a1260 0x7fb8940a3770 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.408+0000 7fb8ad401700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8a81031e0 0x7fb8a8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.408+0000 7fb8ad401700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a80ffc20 0x7fb8a81027b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.408+0000 7fb8ad401700 1 --2- 172.21.15.150:0/3014369128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8a8100570 0x7fb8a8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.408+0000 7fb8ad401700 1 -- 172.21.15.150:0/3014369128 >> 172.21.15.150:0/3014369128 conn(0x7fb8a80fb8f0 msgr2=0x7fb8a8106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:06.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.408+0000 7fb8ad401700 1 -- 172.21.15.150:0/3014369128 shutdown_connections 2024-05-10T04:04:06.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.408+0000 7fb8ad401700 1 -- 172.21.15.150:0/3014369128 wait complete. 2024-05-10T04:04:06.422 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:06 smithi150 ceph-mon[42477]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:04:06.422 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:06 smithi150 ceph-mon[42477]: pgmap v275: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T04:04:06.422 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:06 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/670414006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:04:06.422 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:06 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2824442184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:04:06.422 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:06 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1743636886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:04:06.423 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:04:06.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:06 smithi181 ceph-mon[46602]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:04:06.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:06 smithi181 ceph-mon[46602]: pgmap v275: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T04:04:06.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:06 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/670414006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:04:06.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:06 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2824442184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:04:06.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:06 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1743636886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:04:06.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:06 smithi161 ceph-mon[46918]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:04:06.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:06 smithi161 ceph-mon[46918]: pgmap v275: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T04:04:06.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:06 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/670414006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:04:06.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:06 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2824442184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:04:06.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:06 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1743636886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:04:06.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.554+0000 7f3ef0305700 1 -- 172.21.15.150:0/2805261944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee8102580 msgr2=0x7f3ee8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.554+0000 7f3ef0305700 1 --2- 172.21.15.150:0/2805261944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee8102580 0x7f3ee8108e30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000b0a0 tx=0x7f3ee000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.554+0000 7f3ef0305700 1 -- 172.21.15.150:0/2805261944 shutdown_connections 2024-05-10T04:04:06.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.554+0000 7f3ef0305700 1 --2- 172.21.15.150:0/2805261944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee8102580 0x7f3ee8108e30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.554+0000 7f3ef0305700 1 --2- 172.21.15.150:0/2805261944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ee8100ac0 0x7f3ee8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.554+0000 7f3ef0305700 1 --2- 172.21.15.150:0/2805261944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee8101be0 0x7f3ee8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.554+0000 7f3ef0305700 1 -- 172.21.15.150:0/2805261944 >> 172.21.15.150:0/2805261944 conn(0x7f3ee80fc0d0 msgr2=0x7f3ee80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:06.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.554+0000 7f3ef0305700 1 -- 172.21.15.150:0/2805261944 shutdown_connections 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.554+0000 7f3ef0305700 1 -- 172.21.15.150:0/2805261944 wait complete. 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3ef0305700 1 Processor -- start 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3ef0305700 1 -- start start 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3ef0305700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee8100ac0 0x7f3ee8068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3ef0305700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee8101be0 0x7f3ee819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3ef0305700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ee8102580 0x7f3ee81a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3ef0305700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee81a1d40 con 0x7f3ee8101be0 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3ef0305700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee81a1eb0 con 0x7f3ee8102580 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3ef0305700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee81a2070 con 0x7f3ee8100ac0 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3eed8a0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee8101be0 0x7f3ee819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3eed8a0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee8101be0 0x7f3ee819f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35482/0 (socket says 172.21.15.150:35482) 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3eed8a0700 1 -- 172.21.15.150:0/3658250051 learned_addr learned my addr 172.21.15.150:0/3658250051 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3eee8a2700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ee8102580 0x7f3ee81a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3eee0a1700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee8100ac0 0x7f3ee8068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:06.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3eed8a0700 1 -- 172.21.15.150:0/3658250051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee8100ac0 msgr2=0x7f3ee8068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3eed8a0700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee8100ac0 0x7f3ee8068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3eed8a0700 1 -- 172.21.15.150:0/3658250051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ee8102580 msgr2=0x7f3ee81a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3eed8a0700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ee8102580 0x7f3ee81a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.555+0000 7f3eed8a0700 1 -- 172.21.15.150:0/3658250051 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee0009d00 con 0x7f3ee8101be0 2024-05-10T04:04:06.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.556+0000 7f3eed8a0700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee8101be0 0x7f3ee819f220 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400d980 tx=0x7f3ee400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:06.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.556+0000 7f3edf7fe700 1 -- 172.21.15.150:0/3658250051 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4004220 con 0x7f3ee8101be0 2024-05-10T04:04:06.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.556+0000 7f3ef0305700 1 -- 172.21.15.150:0/3658250051 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee81a2350 con 0x7f3ee8101be0 2024-05-10T04:04:06.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.556+0000 7f3edf7fe700 1 -- 172.21.15.150:0/3658250051 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee4007640 con 0x7f3ee8101be0 2024-05-10T04:04:06.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.556+0000 7f3edf7fe700 1 -- 172.21.15.150:0/3658250051 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4005690 con 0x7f3ee8101be0 2024-05-10T04:04:06.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.556+0000 7f3ef0305700 1 -- 172.21.15.150:0/3658250051 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee81a2860 con 0x7f3ee8101be0 2024-05-10T04:04:06.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.557+0000 7f3ef0305700 1 -- 172.21.15.150:0/3658250051 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee8107cd0 con 0x7f3ee8101be0 2024-05-10T04:04:06.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.558+0000 7f3edf7fe700 1 -- 172.21.15.150:0/3658250051 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3ee4004750 con 0x7f3ee8101be0 2024-05-10T04:04:06.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.558+0000 7f3edf7fe700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3ed40a11b0 0x7f3ed40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:06.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.559+0000 7f3eee0a1700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3ed40a11b0 0x7f3ed40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:06.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.559+0000 7f3edf7fe700 1 -- 172.21.15.150:0/3658250051 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3ee40cd040 con 0x7f3ee8101be0 2024-05-10T04:04:06.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.559+0000 7f3eee0a1700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3ed40a11b0 0x7f3ed40a36c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8101ae0 tx=0x7f3ed800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:06.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.562+0000 7f3edf7fe700 1 -- 172.21.15.150:0/3658250051 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3ee4096360 con 0x7f3ee8101be0 2024-05-10T04:04:06.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.761+0000 7f3ef0305700 1 -- 172.21.15.150:0/3658250051 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ee80694d0 con 0x7f3ee8101be0 2024-05-10T04:04:06.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.763+0000 7f3edf7fe700 1 -- 172.21.15.150:0/3658250051 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24554 (secure 0 0 0) 0x7f3ee4016020 con 0x7f3ee8101be0 2024-05-10T04:04:06.766 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:04:06.766 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":87,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:03:30.185138+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:04:06.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.765+0000 7f3ef0305700 1 -- 172.21.15.150:0/3658250051 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3ed40a11b0 msgr2=0x7f3ed40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.765+0000 7f3ef0305700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3ed40a11b0 0x7f3ed40a36c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8101ae0 tx=0x7f3ed800b410 comp rx=0 tx=0).stop 2024-05-10T04:04:06.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.765+0000 7f3ef0305700 1 -- 172.21.15.150:0/3658250051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee8101be0 msgr2=0x7f3ee819f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.765+0000 7f3ef0305700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee8101be0 0x7f3ee819f220 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400d980 tx=0x7f3ee400dc90 comp rx=0 tx=0).stop 2024-05-10T04:04:06.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.765+0000 7f3ef0305700 1 -- 172.21.15.150:0/3658250051 shutdown_connections 2024-05-10T04:04:06.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.765+0000 7f3ef0305700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3ed40a11b0 0x7f3ed40a36c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.765+0000 7f3ef0305700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ee8102580 0x7f3ee81a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.765+0000 7f3ef0305700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee8100ac0 0x7f3ee8068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.765+0000 7f3ef0305700 1 --2- 172.21.15.150:0/3658250051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee8101be0 0x7f3ee819f220 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.765+0000 7f3ef0305700 1 -- 172.21.15.150:0/3658250051 >> 172.21.15.150:0/3658250051 conn(0x7f3ee80fc0d0 msgr2=0x7f3ee8104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:06.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.765+0000 7f3ef0305700 1 -- 172.21.15.150:0/3658250051 shutdown_connections 2024-05-10T04:04:06.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.766+0000 7f3ef0305700 1 -- 172.21.15.150:0/3658250051 wait complete. 2024-05-10T04:04:06.782 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:02:30.076694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:02:30.760710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:02:33.850714+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:04:06.782 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:04:06.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.916+0000 7f2445f3a700 1 -- 172.21.15.150:0/3398225191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2440102590 msgr2=0x7f2440108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.916+0000 7f2445f3a700 1 --2- 172.21.15.150:0/3398225191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2440102590 0x7f2440108e40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f243800b0a0 tx=0x7f243800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.917+0000 7f2445f3a700 1 -- 172.21.15.150:0/3398225191 shutdown_connections 2024-05-10T04:04:06.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.917+0000 7f2445f3a700 1 --2- 172.21.15.150:0/3398225191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2440102590 0x7f2440108e40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.917+0000 7f2445f3a700 1 --2- 172.21.15.150:0/3398225191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2440100ad0 0x7f2440100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.917+0000 7f2445f3a700 1 --2- 172.21.15.150:0/3398225191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2440101bf0 0x7f2440102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.917+0000 7f2445f3a700 1 -- 172.21.15.150:0/3398225191 >> 172.21.15.150:0/3398225191 conn(0x7f24400fc0e0 msgr2=0x7f24400fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:06.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.917+0000 7f2445f3a700 1 -- 172.21.15.150:0/3398225191 shutdown_connections 2024-05-10T04:04:06.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.917+0000 7f2445f3a700 1 -- 172.21.15.150:0/3398225191 wait complete. 2024-05-10T04:04:06.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.917+0000 7f2445f3a700 1 Processor -- start 2024-05-10T04:04:06.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.917+0000 7f2445f3a700 1 -- start start 2024-05-10T04:04:06.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f2445f3a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2440100ad0 0x7f2440073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f2445f3a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2440101bf0 0x7f244019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f2445f3a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2440102590 0x7f244019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f2445f3a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019fb10 con 0x7f2440101bf0 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f2445f3a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019fc80 con 0x7f2440100ad0 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f2445f3a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019fe40 con 0x7f2440102590 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f243effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2440101bf0 0x7f244019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f243effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2440101bf0 0x7f244019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35490/0 (socket says 172.21.15.150:35490) 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f243effd700 1 -- 172.21.15.150:0/2153396739 learned_addr learned my addr 172.21.15.150:0/2153396739 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f243f7fe700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2440100ad0 0x7f2440073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f243ffff700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2440102590 0x7f244019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f243effd700 1 -- 172.21.15.150:0/2153396739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2440102590 msgr2=0x7f244019f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f243effd700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2440102590 0x7f244019f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f243effd700 1 -- 172.21.15.150:0/2153396739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2440100ad0 msgr2=0x7f2440073eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f243effd700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2440100ad0 0x7f2440073eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f243effd700 1 -- 172.21.15.150:0/2153396739 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2438009d00 con 0x7f2440101bf0 2024-05-10T04:04:06.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f243effd700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2440101bf0 0x7f244019ce00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f243000d950 tx=0x7f243000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:06.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f243cff9700 1 -- 172.21.15.150:0/2153396739 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243000f890 con 0x7f2440101bf0 2024-05-10T04:04:06.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f2445f3a700 1 -- 172.21.15.150:0/2153396739 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24401a0120 con 0x7f2440101bf0 2024-05-10T04:04:06.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.918+0000 7f243cff9700 1 -- 172.21.15.150:0/2153396739 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2430010460 con 0x7f2440101bf0 2024-05-10T04:04:06.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.919+0000 7f243cff9700 1 -- 172.21.15.150:0/2153396739 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243000e610 con 0x7f2440101bf0 2024-05-10T04:04:06.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.919+0000 7f2445f3a700 1 -- 172.21.15.150:0/2153396739 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24401a0680 con 0x7f2440101bf0 2024-05-10T04:04:06.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.919+0000 7f2445f3a700 1 -- 172.21.15.150:0/2153396739 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2440107ce0 con 0x7f2440101bf0 2024-05-10T04:04:06.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.921+0000 7f243cff9700 1 -- 172.21.15.150:0/2153396739 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f24300108d0 con 0x7f2440101bf0 2024-05-10T04:04:06.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.922+0000 7f243cff9700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f24280a1260 0x7f24280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:06.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.922+0000 7f243f7fe700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f24280a1260 0x7f24280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:06.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.922+0000 7f243cff9700 1 -- 172.21.15.150:0/2153396739 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f24300ccba0 con 0x7f2440101bf0 2024-05-10T04:04:06.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.923+0000 7f243f7fe700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f24280a1260 0x7f24280a3770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2440101af0 tx=0x7f243400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:06.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:06.926+0000 7f243cff9700 1 -- 172.21.15.150:0/2153396739 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2430095e60 con 0x7f2440101bf0 2024-05-10T04:04:07.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.126+0000 7f2445f3a700 1 -- 172.21.15.150:0/2153396739 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f244010c520 con 0x7f2440101bf0 2024-05-10T04:04:07.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.126+0000 7f243cff9700 1 -- 172.21.15.150:0/2153396739 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2430020020 con 0x7f2440101bf0 2024-05-10T04:04:07.128 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:04:07.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.128+0000 7f2445f3a700 1 -- 172.21.15.150:0/2153396739 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f24280a1260 msgr2=0x7f24280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:07.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.128+0000 7f2445f3a700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f24280a1260 0x7f24280a3770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2440101af0 tx=0x7f243400b410 comp rx=0 tx=0).stop 2024-05-10T04:04:07.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.128+0000 7f2445f3a700 1 -- 172.21.15.150:0/2153396739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2440101bf0 msgr2=0x7f244019ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:07.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.128+0000 7f2445f3a700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2440101bf0 0x7f244019ce00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f243000d950 tx=0x7f243000dc60 comp rx=0 tx=0).stop 2024-05-10T04:04:07.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.128+0000 7f2445f3a700 1 -- 172.21.15.150:0/2153396739 shutdown_connections 2024-05-10T04:04:07.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.128+0000 7f2445f3a700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f24280a1260 0x7f24280a3770 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.128+0000 7f2445f3a700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2440102590 0x7f244019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.128+0000 7f2445f3a700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2440100ad0 0x7f2440073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.128+0000 7f2445f3a700 1 --2- 172.21.15.150:0/2153396739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2440101bf0 0x7f244019ce00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.128+0000 7f2445f3a700 1 -- 172.21.15.150:0/2153396739 >> 172.21.15.150:0/2153396739 conn(0x7f24400fc0e0 msgr2=0x7f2440104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:07.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.128+0000 7f2445f3a700 1 -- 172.21.15.150:0/2153396739 shutdown_connections 2024-05-10T04:04:07.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.128+0000 7f2445f3a700 1 -- 172.21.15.150:0/2153396739 wait complete. 2024-05-10T04:04:07.144 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:04:07.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.280+0000 7f09fedac700 1 -- 172.21.15.150:0/1683971563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f8105030 msgr2=0x7f09f8107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:07.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.280+0000 7f09fedac700 1 --2- 172.21.15.150:0/1683971563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f8105030 0x7f09f8107450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009990 tx=0x7f09e4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.280+0000 7f09fedac700 1 -- 172.21.15.150:0/1683971563 shutdown_connections 2024-05-10T04:04:07.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.280+0000 7f09fedac700 1 --2- 172.21.15.150:0/1683971563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09f8107990 0x7f09f8109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.280+0000 7f09fedac700 1 --2- 172.21.15.150:0/1683971563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09f81026d0 0x7f09f8104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.280+0000 7f09fedac700 1 --2- 172.21.15.150:0/1683971563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f8105030 0x7f09f8107450 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.280+0000 7f09fedac700 1 -- 172.21.15.150:0/1683971563 >> 172.21.15.150:0/1683971563 conn(0x7f09f80fc0f0 msgr2=0x7f09f80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:07.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.280+0000 7f09fedac700 1 -- 172.21.15.150:0/1683971563 shutdown_connections 2024-05-10T04:04:07.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.280+0000 7f09fedac700 1 -- 172.21.15.150:0/1683971563 wait complete. 2024-05-10T04:04:07.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.280+0000 7f09fedac700 1 Processor -- start 2024-05-10T04:04:07.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fedac700 1 -- start start 2024-05-10T04:04:07.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fedac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09f81026d0 0x7f09f8101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:07.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fedac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f8105030 0x7f09f81005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:07.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fedac700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09f8107990 0x7f09f8100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:07.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fedac700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f8101200 con 0x7f09f8105030 2024-05-10T04:04:07.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fedac700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f8101370 con 0x7f09f8107990 2024-05-10T04:04:07.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fedac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f81a1f90 con 0x7f09f81026d0 2024-05-10T04:04:07.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09f7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f8105030 0x7f09f81005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:07.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fcb48700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09f81026d0 0x7f09f8101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:07.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09f7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f8105030 0x7f09f81005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35510/0 (socket says 172.21.15.150:35510) 2024-05-10T04:04:07.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fcb48700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09f81026d0 0x7f09f8101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:50678/0 (socket says 172.21.15.150:50678) 2024-05-10T04:04:07.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09f7fff700 1 -- 172.21.15.150:0/2799372190 learned_addr learned my addr 172.21.15.150:0/2799372190 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:07.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fd349700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09f8107990 0x7f09f8100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:07.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fcb48700 1 -- 172.21.15.150:0/2799372190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09f8107990 msgr2=0x7f09f8100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:07.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fcb48700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09f8107990 0x7f09f8100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fcb48700 1 -- 172.21.15.150:0/2799372190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f8105030 msgr2=0x7f09f81005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:07.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fcb48700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f8105030 0x7f09f81005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fcb48700 1 -- 172.21.15.150:0/2799372190 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e40095f0 con 0x7f09f81026d0 2024-05-10T04:04:07.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fd349700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09f8107990 0x7f09f8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:04:07.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09f7fff700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f8105030 0x7f09f81005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:04:07.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.281+0000 7f09fcb48700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09f81026d0 0x7f09f8101f40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00e9c0 tx=0x7f09ec00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:07.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.282+0000 7f09f5ffb700 1 -- 172.21.15.150:0/2799372190 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00c9c0 con 0x7f09f81026d0 2024-05-10T04:04:07.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.282+0000 7f09f5ffb700 1 -- 172.21.15.150:0/2799372190 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09ec010430 con 0x7f09f81026d0 2024-05-10T04:04:07.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.282+0000 7f09fedac700 1 -- 172.21.15.150:0/2799372190 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f81a21f0 con 0x7f09f81026d0 2024-05-10T04:04:07.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.282+0000 7f09f5ffb700 1 -- 172.21.15.150:0/2799372190 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec018680 con 0x7f09f81026d0 2024-05-10T04:04:07.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.282+0000 7f09fedac700 1 -- 172.21.15.150:0/2799372190 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f81a26d0 con 0x7f09f81026d0 2024-05-10T04:04:07.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.283+0000 7f09fedac700 1 -- 172.21.15.150:0/2799372190 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f80fdca0 con 0x7f09f81026d0 2024-05-10T04:04:07.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.284+0000 7f09f5ffb700 1 -- 172.21.15.150:0/2799372190 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f09ec00cb20 con 0x7f09f81026d0 2024-05-10T04:04:07.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.285+0000 7f09f5ffb700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f09e80a12e0 0x7f09e80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:07.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.285+0000 7f09f7fff700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f09e80a12e0 0x7f09e80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:07.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.285+0000 7f09f5ffb700 1 -- 172.21.15.150:0/2799372190 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f09ec014070 con 0x7f09f81026d0 2024-05-10T04:04:07.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.285+0000 7f09f7fff700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f09e80a12e0 0x7f09e80a37f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f09e4006010 tx=0x7f09e401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:07.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.289+0000 7f09f5ffb700 1 -- 172.21.15.150:0/2799372190 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f09ec096400 con 0x7f09f81026d0 2024-05-10T04:04:07.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:07 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3014369128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:04:07.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:07 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3658250051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:04:07.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:07 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2153396739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:04:07.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:07 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3014369128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:04:07.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:07 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3658250051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:04:07.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:07 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2153396739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:04:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.488+0000 7f09fedac700 1 -- 172.21.15.150:0/2799372190 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09f810d420 con 0x7f09f81026d0 2024-05-10T04:04:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.488+0000 7f09f5ffb700 1 -- 172.21.15.150:0/2799372190 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f09ec099a20 con 0x7f09f81026d0 2024-05-10T04:04:07.491 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:04:07.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:07 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3014369128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:04:07.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:07 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3658250051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:04:07.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:07 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2153396739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:04:07.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.491+0000 7f09fedac700 1 -- 172.21.15.150:0/2799372190 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f09e80a12e0 msgr2=0x7f09e80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:07.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.491+0000 7f09fedac700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f09e80a12e0 0x7f09e80a37f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f09e4006010 tx=0x7f09e401a040 comp rx=0 tx=0).stop 2024-05-10T04:04:07.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.491+0000 7f09fedac700 1 -- 172.21.15.150:0/2799372190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09f81026d0 msgr2=0x7f09f8101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:07.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.491+0000 7f09fedac700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09f81026d0 0x7f09f8101f40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00e9c0 tx=0x7f09ec00ecd0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.491+0000 7f09fedac700 1 -- 172.21.15.150:0/2799372190 shutdown_connections 2024-05-10T04:04:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.491+0000 7f09fedac700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f09e80a12e0 0x7f09e80a37f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.491+0000 7f09fedac700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09f8107990 0x7f09f8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.491+0000 7f09fedac700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09f81026d0 0x7f09f8101f40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.491+0000 7f09fedac700 1 --2- 172.21.15.150:0/2799372190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f8105030 0x7f09f81005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.491+0000 7f09fedac700 1 -- 172.21.15.150:0/2799372190 >> 172.21.15.150:0/2799372190 conn(0x7f09f80fc0f0 msgr2=0x7f09f8103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.491+0000 7f09fedac700 1 -- 172.21.15.150:0/2799372190 shutdown_connections 2024-05-10T04:04:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.491+0000 7f09fedac700 1 -- 172.21.15.150:0/2799372190 wait complete. 2024-05-10T04:04:07.506 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:04:07.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.643+0000 7ff6d3fff700 1 -- 172.21.15.150:0/467905275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4101840 msgr2=0x7ff6d4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:07.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.643+0000 7ff6d3fff700 1 --2- 172.21.15.150:0/467905275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4101840 0x7ff6d4101d10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80098f0 tx=0x7ff6c8009c00 comp rx=0 tx=0).stop 2024-05-10T04:04:07.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.643+0000 7ff6d3fff700 1 -- 172.21.15.150:0/467905275 shutdown_connections 2024-05-10T04:04:07.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.643+0000 7ff6d3fff700 1 --2- 172.21.15.150:0/467905275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d41023a0 0x7ff6d4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.643+0000 7ff6d3fff700 1 --2- 172.21.15.150:0/467905275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6d4100e40 0x7ff6d4101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.643+0000 7ff6d3fff700 1 --2- 172.21.15.150:0/467905275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4101840 0x7ff6d4101d10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.643+0000 7ff6d3fff700 1 -- 172.21.15.150:0/467905275 >> 172.21.15.150:0/467905275 conn(0x7ff6d40fb6c0 msgr2=0x7ff6d40fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:07.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.643+0000 7ff6d3fff700 1 -- 172.21.15.150:0/467905275 shutdown_connections 2024-05-10T04:04:07.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.643+0000 7ff6d3fff700 1 -- 172.21.15.150:0/467905275 wait complete. 2024-05-10T04:04:07.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.644+0000 7ff6d3fff700 1 Processor -- start 2024-05-10T04:04:07.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.644+0000 7ff6d3fff700 1 -- start start 2024-05-10T04:04:07.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.644+0000 7ff6d3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4100e40 0x7ff6d4104850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:07.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.644+0000 7ff6d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4101840 0x7ff6d4102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:07.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.644+0000 7ff6d3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6d41023a0 0x7ff6d4103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:07.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.644+0000 7ff6d3fff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d4103a90 con 0x7ff6d4100e40 2024-05-10T04:04:07.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.644+0000 7ff6d3fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d4103c00 con 0x7ff6d41023a0 2024-05-10T04:04:07.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.644+0000 7ff6d3fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d4103dc0 con 0x7ff6d4101840 2024-05-10T04:04:07.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.644+0000 7ff6d2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4100e40 0x7ff6d4104850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:07.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.644+0000 7ff6d2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4100e40 0x7ff6d4104850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35526/0 (socket says 172.21.15.150:35526) 2024-05-10T04:04:07.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.644+0000 7ff6d2ffd700 1 -- 172.21.15.150:0/3257898927 learned_addr learned my addr 172.21.15.150:0/3257898927 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:07.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.644+0000 7ff6d37fe700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6d41023a0 0x7ff6d4103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:07.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.644+0000 7ff6d27fc700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4101840 0x7ff6d4102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:07.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.645+0000 7ff6d2ffd700 1 -- 172.21.15.150:0/3257898927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4101840 msgr2=0x7ff6d4102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:07.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.645+0000 7ff6d2ffd700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4101840 0x7ff6d4102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.645+0000 7ff6d2ffd700 1 -- 172.21.15.150:0/3257898927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6d41023a0 msgr2=0x7ff6d4103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:07.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.645+0000 7ff6d2ffd700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6d41023a0 0x7ff6d4103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.645+0000 7ff6d2ffd700 1 -- 172.21.15.150:0/3257898927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c8009520 con 0x7ff6d4100e40 2024-05-10T04:04:07.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.645+0000 7ff6d37fe700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6d41023a0 0x7ff6d4103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:04:07.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.645+0000 7ff6d2ffd700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4100e40 0x7ff6d4104850 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400e9c0 tx=0x7ff6c400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:07.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.645+0000 7ff6d89fe700 1 -- 172.21.15.150:0/3257898927 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4004220 con 0x7ff6d4100e40 2024-05-10T04:04:07.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.645+0000 7ff6d89fe700 1 -- 172.21.15.150:0/3257898927 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c4004380 con 0x7ff6d4100e40 2024-05-10T04:04:07.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.645+0000 7ff6d3fff700 1 -- 172.21.15.150:0/3257898927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d41a20b0 con 0x7ff6d4100e40 2024-05-10T04:04:07.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.645+0000 7ff6d89fe700 1 -- 172.21.15.150:0/3257898927 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4003b20 con 0x7ff6d4100e40 2024-05-10T04:04:07.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.645+0000 7ff6d3fff700 1 -- 172.21.15.150:0/3257898927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d41a25c0 con 0x7ff6d4100e40 2024-05-10T04:04:07.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.646+0000 7ff6d3fff700 1 -- 172.21.15.150:0/3257898927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d4068560 con 0x7ff6d4100e40 2024-05-10T04:04:07.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.647+0000 7ff6d89fe700 1 -- 172.21.15.150:0/3257898927 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff6c4003cc0 con 0x7ff6d4100e40 2024-05-10T04:04:07.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.648+0000 7ff6d89fe700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff6bc0a11b0 0x7ff6bc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:07.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.648+0000 7ff6d27fc700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff6bc0a11b0 0x7ff6bc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:07.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.648+0000 7ff6d89fe700 1 -- 172.21.15.150:0/3257898927 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff6c4014070 con 0x7ff6d4100e40 2024-05-10T04:04:07.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.648+0000 7ff6d27fc700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff6bc0a11b0 0x7ff6bc0a36c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80098c0 tx=0x7ff6c8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:07.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.653+0000 7ff6d89fe700 1 -- 172.21.15.150:0/3257898927 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff6c40961f0 con 0x7ff6d4100e40 2024-05-10T04:04:07.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.850+0000 7ff6d3fff700 1 -- 172.21.15.150:0/3257898927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6d41a2870 con 0x7ff6d4100e40 2024-05-10T04:04:07.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.851+0000 7ff6d89fe700 1 -- 172.21.15.150:0/3257898927 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff6c4099810 con 0x7ff6d4100e40 2024-05-10T04:04:07.853 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:04:07.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.853+0000 7ff6d3fff700 1 -- 172.21.15.150:0/3257898927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff6bc0a11b0 msgr2=0x7ff6bc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:07.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.853+0000 7ff6d3fff700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff6bc0a11b0 0x7ff6bc0a36c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80098c0 tx=0x7ff6c8019040 comp rx=0 tx=0).stop 2024-05-10T04:04:07.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.853+0000 7ff6d3fff700 1 -- 172.21.15.150:0/3257898927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4100e40 msgr2=0x7ff6d4104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:07.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.853+0000 7ff6d3fff700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4100e40 0x7ff6d4104850 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400e9c0 tx=0x7ff6c400ecd0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.853+0000 7ff6d3fff700 1 -- 172.21.15.150:0/3257898927 shutdown_connections 2024-05-10T04:04:07.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.853+0000 7ff6d3fff700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff6bc0a11b0 0x7ff6bc0a36c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.853+0000 7ff6d3fff700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6d41023a0 0x7ff6d4103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.853+0000 7ff6d3fff700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4100e40 0x7ff6d4104850 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.853+0000 7ff6d3fff700 1 --2- 172.21.15.150:0/3257898927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4101840 0x7ff6d4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:07.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.853+0000 7ff6d3fff700 1 -- 172.21.15.150:0/3257898927 >> 172.21.15.150:0/3257898927 conn(0x7ff6d40fb6c0 msgr2=0x7ff6d40fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:07.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.853+0000 7ff6d3fff700 1 -- 172.21.15.150:0/3257898927 shutdown_connections 2024-05-10T04:04:07.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:07.853+0000 7ff6d3fff700 1 -- 172.21.15.150:0/3257898927 wait complete. 2024-05-10T04:04:07.868 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:04:08.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.002+0000 7fe921425700 1 -- 172.21.15.150:0/3134253551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91c101070 msgr2=0x7fe91c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.002+0000 7fe921425700 1 --2- 172.21.15.150:0/3134253551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91c101070 0x7fe91c101480 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe910009990 tx=0x7fe910009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.002+0000 7fe921425700 1 -- 172.21.15.150:0/3134253551 shutdown_connections 2024-05-10T04:04:08.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.002+0000 7fe921425700 1 --2- 172.21.15.150:0/3134253551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe91c1025b0 0x7fe91c109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.002+0000 7fe921425700 1 --2- 172.21.15.150:0/3134253551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91c101070 0x7fe91c101480 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.002+0000 7fe921425700 1 --2- 172.21.15.150:0/3134253551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c101a50 0x7fe91c101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.002+0000 7fe921425700 1 -- 172.21.15.150:0/3134253551 >> 172.21.15.150:0/3134253551 conn(0x7fe91c0fb8b0 msgr2=0x7fe91c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:08.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.002+0000 7fe921425700 1 -- 172.21.15.150:0/3134253551 shutdown_connections 2024-05-10T04:04:08.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.002+0000 7fe921425700 1 -- 172.21.15.150:0/3134253551 wait complete. 2024-05-10T04:04:08.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe921425700 1 Processor -- start 2024-05-10T04:04:08.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe921425700 1 -- start start 2024-05-10T04:04:08.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe921425700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91c101070 0x7fe91c100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:08.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe921425700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c101a50 0x7fe91c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:08.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe921425700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe91c1025b0 0x7fe91c19f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:08.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe921425700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c19f9d0 con 0x7fe91c101070 2024-05-10T04:04:08.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe921425700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c19fb40 con 0x7fe91c101a50 2024-05-10T04:04:08.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe921425700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c19fd00 con 0x7fe91c1025b0 2024-05-10T04:04:08.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe91affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91c101070 0x7fe91c100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:08.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe91affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91c101070 0x7fe91c100730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35542/0 (socket says 172.21.15.150:35542) 2024-05-10T04:04:08.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe91affd700 1 -- 172.21.15.150:0/4198426637 learned_addr learned my addr 172.21.15.150:0/4198426637 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:08.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe91a7fc700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c101a50 0x7fe91c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:08.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe91b7fe700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe91c1025b0 0x7fe91c19f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:08.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe91affd700 1 -- 172.21.15.150:0/4198426637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe91c1025b0 msgr2=0x7fe91c19f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe91affd700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe91c1025b0 0x7fe91c19f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe91affd700 1 -- 172.21.15.150:0/4198426637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c101a50 msgr2=0x7fe91c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.003+0000 7fe91affd700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c101a50 0x7fe91c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.004+0000 7fe91affd700 1 -- 172.21.15.150:0/4198426637 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9100095f0 con 0x7fe91c101070 2024-05-10T04:04:08.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.004+0000 7fe91b7fe700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe91c1025b0 0x7fe91c19f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:08.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.004+0000 7fe91affd700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91c101070 0x7fe91c100730 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe910000c00 tx=0x7fe910009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:08.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.004+0000 7fe903fff700 1 -- 172.21.15.150:0/4198426637 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91001d070 con 0x7fe91c101070 2024-05-10T04:04:08.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.004+0000 7fe903fff700 1 -- 172.21.15.150:0/4198426637 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe91000fbd0 con 0x7fe91c101070 2024-05-10T04:04:08.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.004+0000 7fe921425700 1 -- 172.21.15.150:0/4198426637 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91c19ff80 con 0x7fe91c101070 2024-05-10T04:04:08.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.004+0000 7fe903fff700 1 -- 172.21.15.150:0/4198426637 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910017650 con 0x7fe91c101070 2024-05-10T04:04:08.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.004+0000 7fe921425700 1 -- 172.21.15.150:0/4198426637 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91c1a02f0 con 0x7fe91c101070 2024-05-10T04:04:08.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.005+0000 7fe921425700 1 -- 172.21.15.150:0/4198426637 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91c107ed0 con 0x7fe91c101070 2024-05-10T04:04:08.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.006+0000 7fe903fff700 1 -- 172.21.15.150:0/4198426637 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe91000fd40 con 0x7fe91c101070 2024-05-10T04:04:08.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.007+0000 7fe903fff700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9040a1260 0x7fe9040a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:08.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.007+0000 7fe91a7fc700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9040a1260 0x7fe9040a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:08.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.007+0000 7fe903fff700 1 -- 172.21.15.150:0/4198426637 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe9100ce8c0 con 0x7fe91c101070 2024-05-10T04:04:08.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.007+0000 7fe91a7fc700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9040a1260 0x7fe9040a3770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe91c1000d0 tx=0x7fe90c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:08.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.010+0000 7fe903fff700 1 -- 172.21.15.150:0/4198426637 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe910097c70 con 0x7fe91c101070 2024-05-10T04:04:08.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.206+0000 7fe921425700 1 -- 172.21.15.150:0/4198426637 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe91c10c4b0 con 0x7fe91c101070 2024-05-10T04:04:08.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.207+0000 7fe903fff700 1 -- 172.21.15.150:0/4198426637 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe91009b290 con 0x7fe91c101070 2024-05-10T04:04:08.209 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:04:08.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.209+0000 7fe921425700 1 -- 172.21.15.150:0/4198426637 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9040a1260 msgr2=0x7fe9040a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.209+0000 7fe921425700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9040a1260 0x7fe9040a3770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe91c1000d0 tx=0x7fe90c009260 comp rx=0 tx=0).stop 2024-05-10T04:04:08.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.209+0000 7fe921425700 1 -- 172.21.15.150:0/4198426637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91c101070 msgr2=0x7fe91c100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.209+0000 7fe921425700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91c101070 0x7fe91c100730 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe910000c00 tx=0x7fe910009580 comp rx=0 tx=0).stop 2024-05-10T04:04:08.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.209+0000 7fe921425700 1 -- 172.21.15.150:0/4198426637 shutdown_connections 2024-05-10T04:04:08.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.209+0000 7fe921425700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9040a1260 0x7fe9040a3770 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.209+0000 7fe921425700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe91c1025b0 0x7fe91c19f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.209+0000 7fe921425700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91c101070 0x7fe91c100730 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.209+0000 7fe921425700 1 --2- 172.21.15.150:0/4198426637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c101a50 0x7fe91c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.209+0000 7fe921425700 1 -- 172.21.15.150:0/4198426637 >> 172.21.15.150:0/4198426637 conn(0x7fe91c0fb8b0 msgr2=0x7fe91c105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:08.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.209+0000 7fe921425700 1 -- 172.21.15.150:0/4198426637 shutdown_connections 2024-05-10T04:04:08.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.209+0000 7fe921425700 1 -- 172.21.15.150:0/4198426637 wait complete. 2024-05-10T04:04:08.224 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-10T04:04:08.225 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:04:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.355+0000 7f5023f86700 1 -- 172.21.15.150:0/2817159081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f501c102580 msgr2=0x7f501c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.355+0000 7f5023f86700 1 --2- 172.21.15.150:0/2817159081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f501c102580 0x7f501c108e30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f501400b0a0 tx=0x7f501400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.355+0000 7f5023f86700 1 -- 172.21.15.150:0/2817159081 shutdown_connections 2024-05-10T04:04:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.355+0000 7f5023f86700 1 --2- 172.21.15.150:0/2817159081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f501c102580 0x7f501c108e30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.355+0000 7f5023f86700 1 --2- 172.21.15.150:0/2817159081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f501c100ac0 0x7f501c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.355+0000 7f5023f86700 1 --2- 172.21.15.150:0/2817159081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f501c101be0 0x7f501c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.355+0000 7f5023f86700 1 -- 172.21.15.150:0/2817159081 >> 172.21.15.150:0/2817159081 conn(0x7f501c0fc0d0 msgr2=0x7f501c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.355+0000 7f5023f86700 1 -- 172.21.15.150:0/2817159081 shutdown_connections 2024-05-10T04:04:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.355+0000 7f5023f86700 1 -- 172.21.15.150:0/2817159081 wait complete. 2024-05-10T04:04:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.356+0000 7f5023f86700 1 Processor -- start 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.356+0000 7f5023f86700 1 -- start start 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.356+0000 7f5023f86700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f501c100ac0 0x7f501c104880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.356+0000 7f5023f86700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f501c101be0 0x7f501c102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.356+0000 7f5023f86700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f501c102580 0x7f501c103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.356+0000 7f5023f86700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c103b40 con 0x7f501c100ac0 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.356+0000 7f5023f86700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c103cb0 con 0x7f501c101be0 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.356+0000 7f5023f86700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c1a2160 con 0x7f501c102580 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.356+0000 7f5021d22700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f501c100ac0 0x7f501c104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.356+0000 7f5021d22700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f501c100ac0 0x7f501c104880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35552/0 (socket says 172.21.15.150:35552) 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.356+0000 7f5021d22700 1 -- 172.21.15.150:0/1207869510 learned_addr learned my addr 172.21.15.150:0/1207869510 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.356+0000 7f5021521700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f501c101be0 0x7f501c102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.356+0000 7f5022523700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f501c102580 0x7f501c103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.357+0000 7f5021d22700 1 -- 172.21.15.150:0/1207869510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f501c102580 msgr2=0x7f501c103460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.357+0000 7f5021d22700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f501c102580 0x7f501c103460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.357+0000 7f5021d22700 1 -- 172.21.15.150:0/1207869510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f501c101be0 msgr2=0x7f501c102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.357+0000 7f5021d22700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f501c101be0 0x7f501c102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.357+0000 7f5021d22700 1 -- 172.21.15.150:0/1207869510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5014009d00 con 0x7f501c100ac0 2024-05-10T04:04:08.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.357+0000 7f5021d22700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f501c100ac0 0x7f501c104880 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f501800d950 tx=0x7f501800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:08.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.357+0000 7f500effd700 1 -- 172.21.15.150:0/1207869510 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018004220 con 0x7f501c100ac0 2024-05-10T04:04:08.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.357+0000 7f500effd700 1 -- 172.21.15.150:0/1207869510 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5018004380 con 0x7f501c100ac0 2024-05-10T04:04:08.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.357+0000 7f5023f86700 1 -- 172.21.15.150:0/1207869510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501c1a23c0 con 0x7f501c100ac0 2024-05-10T04:04:08.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.357+0000 7f500effd700 1 -- 172.21.15.150:0/1207869510 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018003bf0 con 0x7f501c100ac0 2024-05-10T04:04:08.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.357+0000 7f5023f86700 1 -- 172.21.15.150:0/1207869510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501c1a28a0 con 0x7f501c100ac0 2024-05-10T04:04:08.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.358+0000 7f5023f86700 1 -- 172.21.15.150:0/1207869510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f501c0694d0 con 0x7f501c100ac0 2024-05-10T04:04:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.359+0000 7f500effd700 1 -- 172.21.15.150:0/1207869510 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5018003d50 con 0x7f501c100ac0 2024-05-10T04:04:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.360+0000 7f500effd700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f50080a11b0 0x7f50080a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.360+0000 7f5021521700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f50080a11b0 0x7f50080a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.360+0000 7f500effd700 1 -- 172.21.15.150:0/1207869510 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f50180cd040 con 0x7f501c100ac0 2024-05-10T04:04:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.360+0000 7f5021521700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f50080a11b0 0x7f50080a36c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f501c104220 tx=0x7f5010009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:08.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.363+0000 7f500effd700 1 -- 172.21.15.150:0/1207869510 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5018096360 con 0x7f501c100ac0 2024-05-10T04:04:08.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:08 smithi150 ceph-mon[42477]: pgmap v276: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T04:04:08.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:08 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2799372190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:04:08.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:08 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3257898927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:04:08.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:08 smithi181 ceph-mon[46602]: pgmap v276: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T04:04:08.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:08 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2799372190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:04:08.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:08 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3257898927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:04:08.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:08 smithi161 ceph-mon[46918]: pgmap v276: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T04:04:08.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:08 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2799372190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:04:08.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:08 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3257898927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:04:08.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.608+0000 7f5023f86700 1 -- 172.21.15.150:0/1207869510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f501c10c2d0 con 0x7f501c100ac0 2024-05-10T04:04:08.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.609+0000 7f500effd700 1 -- 172.21.15.150:0/1207869510 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10886 (secure 0 0 0) 0x7f5018099980 con 0x7f501c100ac0 2024-05-10T04:04:08.611 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:04:08.611 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:04:04.157819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:04:08.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.611+0000 7f5023f86700 1 -- 172.21.15.150:0/1207869510 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f50080a11b0 msgr2=0x7f50080a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.611+0000 7f5023f86700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f50080a11b0 0x7f50080a36c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f501c104220 tx=0x7f5010009260 comp rx=0 tx=0).stop 2024-05-10T04:04:08.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.611+0000 7f5023f86700 1 -- 172.21.15.150:0/1207869510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f501c100ac0 msgr2=0x7f501c104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.611+0000 7f5023f86700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f501c100ac0 0x7f501c104880 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f501800d950 tx=0x7f501800dc60 comp rx=0 tx=0).stop 2024-05-10T04:04:08.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.611+0000 7f5023f86700 1 -- 172.21.15.150:0/1207869510 shutdown_connections 2024-05-10T04:04:08.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.611+0000 7f5023f86700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f50080a11b0 0x7f50080a36c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.611+0000 7f5023f86700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f501c102580 0x7f501c103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.611+0000 7f5023f86700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f501c100ac0 0x7f501c104880 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.611+0000 7f5023f86700 1 --2- 172.21.15.150:0/1207869510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f501c101be0 0x7f501c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.611+0000 7f5023f86700 1 -- 172.21.15.150:0/1207869510 >> 172.21.15.150:0/1207869510 conn(0x7f501c0fc0d0 msgr2=0x7f501c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:08.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.611+0000 7f5023f86700 1 -- 172.21.15.150:0/1207869510 shutdown_connections 2024-05-10T04:04:08.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.611+0000 7f5023f86700 1 -- 172.21.15.150:0/1207869510 wait complete. 2024-05-10T04:04:08.615 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 25 2024-05-10T04:04:08.626 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-10T04:04:08.626 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-10T04:04:08.626 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T04:04:08.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.756+0000 7faf371d2700 1 -- 172.21.15.150:0/802891318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf30105010 msgr2=0x7faf30107430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.756+0000 7faf371d2700 1 --2- 172.21.15.150:0/802891318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf30105010 0x7faf30107430 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009990 tx=0x7faf1c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.756+0000 7faf371d2700 1 -- 172.21.15.150:0/802891318 shutdown_connections 2024-05-10T04:04:08.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.756+0000 7faf371d2700 1 --2- 172.21.15.150:0/802891318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf30107970 0x7faf30109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.756+0000 7faf371d2700 1 --2- 172.21.15.150:0/802891318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf301026b0 0x7faf30104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.756+0000 7faf371d2700 1 --2- 172.21.15.150:0/802891318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf30105010 0x7faf30107430 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.756+0000 7faf371d2700 1 -- 172.21.15.150:0/802891318 >> 172.21.15.150:0/802891318 conn(0x7faf300fc0d0 msgr2=0x7faf300fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:08.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.756+0000 7faf371d2700 1 -- 172.21.15.150:0/802891318 shutdown_connections 2024-05-10T04:04:08.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.756+0000 7faf371d2700 1 -- 172.21.15.150:0/802891318 wait complete. 2024-05-10T04:04:08.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.757+0000 7faf371d2700 1 Processor -- start 2024-05-10T04:04:08.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.757+0000 7faf371d2700 1 -- start start 2024-05-10T04:04:08.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.757+0000 7faf371d2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf301026b0 0x7faf3019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:08.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.757+0000 7faf371d2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf30105010 0x7faf3019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:08.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.757+0000 7faf371d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf30107970 0x7faf301a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:08.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.757+0000 7faf371d2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf301a1d20 con 0x7faf301026b0 2024-05-10T04:04:08.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.757+0000 7faf371d2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf301a1e90 con 0x7faf30105010 2024-05-10T04:04:08.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.757+0000 7faf371d2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf301a2050 con 0x7faf30107970 2024-05-10T04:04:08.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.757+0000 7faf34f6e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf301026b0 0x7faf3019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:08.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.757+0000 7faf34f6e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf301026b0 0x7faf3019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35564/0 (socket says 172.21.15.150:35564) 2024-05-10T04:04:08.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.757+0000 7faf34f6e700 1 -- 172.21.15.150:0/958873727 learned_addr learned my addr 172.21.15.150:0/958873727 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:08.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.757+0000 7faf2ffff700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf30105010 0x7faf3019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:08.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.757+0000 7faf3576f700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf30107970 0x7faf301a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:08.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.758+0000 7faf34f6e700 1 -- 172.21.15.150:0/958873727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf30107970 msgr2=0x7faf301a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.758+0000 7faf34f6e700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf30107970 0x7faf301a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.758+0000 7faf34f6e700 1 -- 172.21.15.150:0/958873727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf30105010 msgr2=0x7faf3019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.758+0000 7faf34f6e700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf30105010 0x7faf3019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.758+0000 7faf34f6e700 1 -- 172.21.15.150:0/958873727 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1c0095f0 con 0x7faf301026b0 2024-05-10T04:04:08.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.758+0000 7faf34f6e700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf301026b0 0x7faf3019a240 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7faf2400d980 tx=0x7faf2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:08.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.758+0000 7faf2dffb700 1 -- 172.21.15.150:0/958873727 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2400f890 con 0x7faf301026b0 2024-05-10T04:04:08.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.758+0000 7faf371d2700 1 -- 172.21.15.150:0/958873727 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf301a2330 con 0x7faf301026b0 2024-05-10T04:04:08.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.758+0000 7faf2dffb700 1 -- 172.21.15.150:0/958873727 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf24010460 con 0x7faf301026b0 2024-05-10T04:04:08.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.758+0000 7faf2dffb700 1 -- 172.21.15.150:0/958873727 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2400e610 con 0x7faf301026b0 2024-05-10T04:04:08.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.758+0000 7faf371d2700 1 -- 172.21.15.150:0/958873727 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf301a2840 con 0x7faf301026b0 2024-05-10T04:04:08.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.759+0000 7faf371d2700 1 -- 172.21.15.150:0/958873727 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf300694d0 con 0x7faf301026b0 2024-05-10T04:04:08.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.760+0000 7faf2dffb700 1 -- 172.21.15.150:0/958873727 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7faf240105d0 con 0x7faf301026b0 2024-05-10T04:04:08.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.761+0000 7faf2dffb700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faf200a1230 0x7faf200a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:08.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.761+0000 7faf2dffb700 1 -- 172.21.15.150:0/958873727 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7faf240cc7e0 con 0x7faf301026b0 2024-05-10T04:04:08.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.761+0000 7faf2ffff700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faf200a1230 0x7faf200a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:08.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.761+0000 7faf2ffff700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faf200a1230 0x7faf200a3740 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00b2e0 tx=0x7faf1c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:08.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.766+0000 7faf2dffb700 1 -- 172.21.15.150:0/958873727 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faf24095aa0 con 0x7faf301026b0 2024-05-10T04:04:08.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.960+0000 7faf371d2700 1 -- 172.21.15.150:0/958873727 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf30100c30 con 0x7faf200a1230 2024-05-10T04:04:08.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.972+0000 7faf2dffb700 1 -- 172.21.15.150:0/958873727 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427881 (secure 0 0 0) 0x7faf30100c30 con 0x7faf200a1230 2024-05-10T04:04:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:04:08.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.977+0000 7faf371d2700 1 -- 172.21.15.150:0/958873727 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faf200a1230 msgr2=0x7faf200a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.977+0000 7faf371d2700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faf200a1230 0x7faf200a3740 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00b2e0 tx=0x7faf1c01a040 comp rx=0 tx=0).stop 2024-05-10T04:04:08.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.977+0000 7faf371d2700 1 -- 172.21.15.150:0/958873727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf301026b0 msgr2=0x7faf3019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:08.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.977+0000 7faf371d2700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf301026b0 0x7faf3019a240 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7faf2400d980 tx=0x7faf2400dc90 comp rx=0 tx=0).stop 2024-05-10T04:04:08.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.977+0000 7faf371d2700 1 -- 172.21.15.150:0/958873727 shutdown_connections 2024-05-10T04:04:08.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.977+0000 7faf371d2700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faf200a1230 0x7faf200a3740 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.977+0000 7faf371d2700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf30107970 0x7faf301a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.977+0000 7faf371d2700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf301026b0 0x7faf3019a240 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.977+0000 7faf371d2700 1 --2- 172.21.15.150:0/958873727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf30105010 0x7faf3019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:08.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.977+0000 7faf371d2700 1 -- 172.21.15.150:0/958873727 >> 172.21.15.150:0/958873727 conn(0x7faf300fc0d0 msgr2=0x7faf30100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:08.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.977+0000 7faf371d2700 1 -- 172.21.15.150:0/958873727 shutdown_connections 2024-05-10T04:04:08.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:08.977+0000 7faf371d2700 1 -- 172.21.15.150:0/958873727 wait complete. 2024-05-10T04:04:08.985 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:04:08.992 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-05-10T04:04:07.106414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657261,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":562,"num_read_kb":398,"num_write":341,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180965,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372712,"kb_used_data":11612,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569112,"statfs":{"total":1151940427776,"available":1151558770688,"internally_reserved":0,"allocated":11890688,"data_stored":7719107,"data_compressed":322563,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":628,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":32,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003943"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195663+0000","last_change":"2024-05-10T04:02:31.642089+0000","last_active":"2024-05-10T04:03:29.195663+0000","last_peered":"2024-05-10T04:03:29.195663+0000","last_clean":"2024-05-10T04:03:29.195663+0000","last_became_active":"2024-05-10T04:02:31.641080+0000","last_became_peered":"2024-05-10T04:02:31.641080+0000","last_unstale":"2024-05-10T04:03:29.195663+0000","last_undegraded":"2024-05-10T04:03:29.195663+0000","last_fullsized":"2024-05-10T04:03:29.195663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:29.693139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197452+0000","last_change":"2024-05-10T04:02:31.633056+0000","last_active":"2024-05-10T04:03:29.197452+0000","last_peered":"2024-05-10T04:03:29.197452+0000","last_clean":"2024-05-10T04:03:29.197452+0000","last_became_active":"2024-05-10T04:02:31.632725+0000","last_became_peered":"2024-05-10T04:02:31.632725+0000","last_unstale":"2024-05-10T04:03:29.197452+0000","last_undegraded":"2024-05-10T04:03:29.197452+0000","last_fullsized":"2024-05-10T04:03:29.197452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:08:40.365433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197250+0000","last_change":"2024-05-10T04:02:31.625219+0000","last_active":"2024-05-10T04:03:29.197250+0000","last_peered":"2024-05-10T04:03:29.197250+0000","last_clean":"2024-05-10T04:03:29.197250+0000","last_became_active":"2024-05-10T04:02:31.624889+0000","last_became_peered":"2024-05-10T04:02:31.624889+0000","last_unstale":"2024-05-10T04:03:29.197250+0000","last_undegraded":"2024-05-10T04:03:29.197250+0000","last_fullsized":"2024-05-10T04:03:29.197250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:10:59.081031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.199344+0000","last_change":"2024-05-10T04:03:30.198971+0000","last_active":"2024-05-10T04:03:30.199344+0000","last_peered":"2024-05-10T04:03:30.199344+0000","last_clean":"2024-05-10T04:03:30.199344+0000","last_became_active":"2024-05-10T04:03:30.198664+0000","last_became_peered":"2024-05-10T04:03:30.198664+0000","last_unstale":"2024-05-10T04:03:30.199344+0000","last_undegraded":"2024-05-10T04:03:30.199344+0000","last_fullsized":"2024-05-10T04:03:30.199344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:22.655875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195511+0000","last_change":"2024-05-10T04:02:31.630688+0000","last_active":"2024-05-10T04:03:29.195511+0000","last_peered":"2024-05-10T04:03:29.195511+0000","last_clean":"2024-05-10T04:03:29.195511+0000","last_became_active":"2024-05-10T04:02:31.629252+0000","last_became_peered":"2024-05-10T04:02:31.629252+0000","last_unstale":"2024-05-10T04:03:29.195511+0000","last_undegraded":"2024-05-10T04:03:29.195511+0000","last_fullsized":"2024-05-10T04:03:29.195511+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:37.647516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198848+0000","last_change":"2024-05-10T04:02:31.642586+0000","last_active":"2024-05-10T04:03:29.198848+0000","last_peered":"2024-05-10T04:03:29.198848+0000","last_clean":"2024-05-10T04:03:29.198848+0000","last_became_active":"2024-05-10T04:02:31.642244+0000","last_became_peered":"2024-05-10T04:02:31.642244+0000","last_unstale":"2024-05-10T04:03:29.198848+0000","last_undegraded":"2024-05-10T04:03:29.198848+0000","last_fullsized":"2024-05-10T04:03:29.198848+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:28.020151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191697+0000","last_change":"2024-05-10T04:02:31.636706+0000","last_active":"2024-05-10T04:03:29.191697+0000","last_peered":"2024-05-10T04:03:29.191697+0000","last_clean":"2024-05-10T04:03:29.191697+0000","last_became_active":"2024-05-10T04:02:31.636303+0000","last_became_peered":"2024-05-10T04:02:31.636303+0000","last_unstale":"2024-05-10T04:03:29.191697+0000","last_undegraded":"2024-05-10T04:03:29.191697+0000","last_fullsized":"2024-05-10T04:03:29.191697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:16.941400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198297+0000","last_change":"2024-05-10T04:02:31.640032+0000","last_active":"2024-05-10T04:03:29.198297+0000","last_peered":"2024-05-10T04:03:29.198297+0000","last_clean":"2024-05-10T04:03:29.198297+0000","last_became_active":"2024-05-10T04:02:31.639733+0000","last_became_peered":"2024-05-10T04:02:31.639733+0000","last_unstale":"2024-05-10T04:03:29.198297+0000","last_undegraded":"2024-05-10T04:03:29.198297+0000","last_fullsized":"2024-05-10T04:03:29.198297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:02:36.963414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200412+0000","last_change":"2024-05-10T04:02:31.638940+0000","last_active":"2024-05-10T04:03:29.200412+0000","last_peered":"2024-05-10T04:03:29.200412+0000","last_clean":"2024-05-10T04:03:29.200412+0000","last_became_active":"2024-05-10T04:02:31.638617+0000","last_became_peered":"2024-05-10T04:02:31.638617+0000","last_unstale":"2024-05-10T04:03:29.200412+0000","last_undegraded":"2024-05-10T04:03:29.200412+0000","last_fullsized":"2024-05-10T04:03:29.200412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:10:36.348117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199245+0000","last_change":"2024-05-10T04:02:31.635702+0000","last_active":"2024-05-10T04:03:29.199245+0000","last_peered":"2024-05-10T04:03:29.199245+0000","last_clean":"2024-05-10T04:03:29.199245+0000","last_became_active":"2024-05-10T04:02:31.635330+0000","last_became_peered":"2024-05-10T04:02:31.635330+0000","last_unstale":"2024-05-10T04:03:29.199245+0000","last_undegraded":"2024-05-10T04:03:29.199245+0000","last_fullsized":"2024-05-10T04:03:29.199245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:21:15.160340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192388+0000","last_change":"2024-05-10T04:02:31.628252+0000","last_active":"2024-05-10T04:03:29.192388+0000","last_peered":"2024-05-10T04:03:29.192388+0000","last_clean":"2024-05-10T04:03:29.192388+0000","last_became_active":"2024-05-10T04:02:31.627394+0000","last_became_peered":"2024-05-10T04:02:31.627394+0000","last_unstale":"2024-05-10T04:03:29.192388+0000","last_undegraded":"2024-05-10T04:03:29.192388+0000","last_fullsized":"2024-05-10T04:03:29.192388+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:54.569786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196477+0000","last_change":"2024-05-10T04:02:31.641788+0000","last_active":"2024-05-10T04:03:29.196477+0000","last_peered":"2024-05-10T04:03:29.196477+0000","last_clean":"2024-05-10T04:03:29.196477+0000","last_became_active":"2024-05-10T04:02:31.641328+0000","last_became_peered":"2024-05-10T04:02:31.641328+0000","last_unstale":"2024-05-10T04:03:29.196477+0000","last_undegraded":"2024-05-10T04:03:29.196477+0000","last_fullsized":"2024-05-10T04:03:29.196477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:10:55.920245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191930+0000","last_change":"2024-05-10T04:02:31.631919+0000","last_active":"2024-05-10T04:03:29.191930+0000","last_peered":"2024-05-10T04:03:29.191930+0000","last_clean":"2024-05-10T04:03:29.191930+0000","last_became_active":"2024-05-10T04:02:31.631592+0000","last_became_peered":"2024-05-10T04:02:31.631592+0000","last_unstale":"2024-05-10T04:03:29.191930+0000","last_undegraded":"2024-05-10T04:03:29.191930+0000","last_fullsized":"2024-05-10T04:03:29.191930+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:59:03.792090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196091+0000","last_change":"2024-05-10T04:02:31.633288+0000","last_active":"2024-05-10T04:03:29.196091+0000","last_peered":"2024-05-10T04:03:29.196091+0000","last_clean":"2024-05-10T04:03:29.196091+0000","last_became_active":"2024-05-10T04:02:31.632582+0000","last_became_peered":"2024-05-10T04:02:31.632582+0000","last_unstale":"2024-05-10T04:03:29.196091+0000","last_undegraded":"2024-05-10T04:03:29.196091+0000","last_fullsized":"2024-05-10T04:03:29.196091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:27.991439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196261+0000","last_change":"2024-05-10T04:02:31.635833+0000","last_active":"2024-05-10T04:03:29.196261+0000","last_peered":"2024-05-10T04:03:29.196261+0000","last_clean":"2024-05-10T04:03:29.196261+0000","last_became_active":"2024-05-10T04:02:31.635520+0000","last_became_peered":"2024-05-10T04:02:31.635520+0000","last_unstale":"2024-05-10T04:03:29.196261+0000","last_undegraded":"2024-05-10T04:03:29.196261+0000","last_fullsized":"2024-05-10T04:03:29.196261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:40:06.312819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199745+0000","last_change":"2024-05-10T04:02:31.631734+0000","last_active":"2024-05-10T04:03:29.199745+0000","last_peered":"2024-05-10T04:03:29.199745+0000","last_clean":"2024-05-10T04:03:29.199745+0000","last_became_active":"2024-05-10T04:02:31.631375+0000","last_became_peered":"2024-05-10T04:02:31.631375+0000","last_unstale":"2024-05-10T04:03:29.199745+0000","last_undegraded":"2024-05-10T04:03:29.199745+0000","last_fullsized":"2024-05-10T04:03:29.199745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:25:13.815406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195179+0000","last_change":"2024-05-10T04:02:31.642292+0000","last_active":"2024-05-10T04:03:29.195179+0000","last_peered":"2024-05-10T04:03:29.195179+0000","last_clean":"2024-05-10T04:03:29.195179+0000","last_became_active":"2024-05-10T04:02:31.641570+0000","last_became_peered":"2024-05-10T04:02:31.641570+0000","last_unstale":"2024-05-10T04:03:29.195179+0000","last_undegraded":"2024-05-10T04:03:29.195179+0000","last_fullsized":"2024-05-10T04:03:29.195179+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:16:33.704809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195044+0000","last_change":"2024-05-10T04:02:31.625695+0000","last_active":"2024-05-10T04:03:29.195044+0000","last_peered":"2024-05-10T04:03:29.195044+0000","last_clean":"2024-05-10T04:03:29.195044+0000","last_became_active":"2024-05-10T04:02:31.625393+0000","last_became_peered":"2024-05-10T04:02:31.625393+0000","last_unstale":"2024-05-10T04:03:29.195044+0000","last_undegraded":"2024-05-10T04:03:29.195044+0000","last_fullsized":"2024-05-10T04:03:29.195044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:53.955921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188569+0000","last_change":"2024-05-10T04:02:31.629152+0000","last_active":"2024-05-10T04:03:29.188569+0000","last_peered":"2024-05-10T04:03:29.188569+0000","last_clean":"2024-05-10T04:03:29.188569+0000","last_became_active":"2024-05-10T04:02:31.628778+0000","last_became_peered":"2024-05-10T04:02:31.628778+0000","last_unstale":"2024-05-10T04:03:29.188569+0000","last_undegraded":"2024-05-10T04:03:29.188569+0000","last_fullsized":"2024-05-10T04:03:29.188569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:39.772500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188510+0000","last_change":"2024-05-10T04:02:35.650536+0000","last_active":"2024-05-10T04:03:29.188510+0000","last_peered":"2024-05-10T04:03:29.188510+0000","last_clean":"2024-05-10T04:03:29.188510+0000","last_became_active":"2024-05-10T04:02:35.650102+0000","last_became_peered":"2024-05-10T04:02:35.650102+0000","last_unstale":"2024-05-10T04:03:29.188510+0000","last_undegraded":"2024-05-10T04:03:29.188510+0000","last_fullsized":"2024-05-10T04:03:29.188510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:19:35.605844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192581+0000","last_change":"2024-05-10T04:02:31.186846+0000","last_active":"2024-05-10T04:03:29.192581+0000","last_peered":"2024-05-10T04:03:29.192581+0000","last_clean":"2024-05-10T04:03:29.192581+0000","last_became_active":"2024-05-10T04:02:31.186497+0000","last_became_peered":"2024-05-10T04:02:31.186497+0000","last_unstale":"2024-05-10T04:03:29.192581+0000","last_undegraded":"2024-05-10T04:03:29.192581+0000","last_fullsized":"2024-05-10T04:03:29.192581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:15:40.832455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.252299+0000","last_change":"2024-05-10T04:03:30.251863+0000","last_active":"2024-05-10T04:03:30.252299+0000","last_peered":"2024-05-10T04:03:30.252299+0000","last_clean":"2024-05-10T04:03:30.252299+0000","last_became_active":"2024-05-10T04:03:30.251466+0000","last_became_peered":"2024-05-10T04:03:30.251466+0000","last_unstale":"2024-05-10T04:03:30.252299+0000","last_undegraded":"2024-05-10T04:03:30.252299+0000","last_fullsized":"2024-05-10T04:03:30.252299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:56:12.810512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194320+0000","last_change":"2024-05-10T04:02:35.651622+0000","last_active":"2024-05-10T04:03:29.194320+0000","last_peered":"2024-05-10T04:03:29.194320+0000","last_clean":"2024-05-10T04:03:29.194320+0000","last_became_active":"2024-05-10T04:02:35.651148+0000","last_became_peered":"2024-05-10T04:02:35.651148+0000","last_unstale":"2024-05-10T04:03:29.194320+0000","last_undegraded":"2024-05-10T04:03:29.194320+0000","last_fullsized":"2024-05-10T04:03:29.194320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:40:09.019286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199659+0000","last_change":"2024-05-10T04:02:31.187785+0000","last_active":"2024-05-10T04:03:29.199659+0000","last_peered":"2024-05-10T04:03:29.199659+0000","last_clean":"2024-05-10T04:03:29.199659+0000","last_became_active":"2024-05-10T04:02:31.187499+0000","last_became_peered":"2024-05-10T04:02:31.187499+0000","last_unstale":"2024-05-10T04:03:29.199659+0000","last_undegraded":"2024-05-10T04:03:29.199659+0000","last_fullsized":"2024-05-10T04:03:29.199659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:05:29.015767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197457+0000","last_change":"2024-05-10T04:02:31.624433+0000","last_active":"2024-05-10T04:03:29.197457+0000","last_peered":"2024-05-10T04:03:29.197457+0000","last_clean":"2024-05-10T04:03:29.197457+0000","last_became_active":"2024-05-10T04:02:31.624115+0000","last_became_peered":"2024-05-10T04:02:31.624115+0000","last_unstale":"2024-05-10T04:03:29.197457+0000","last_undegraded":"2024-05-10T04:03:29.197457+0000","last_fullsized":"2024-05-10T04:03:29.197457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:54.584681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.252518+0000","last_change":"2024-05-10T04:03:30.252518+0000","last_active":"2024-05-10T04:03:30.252518+0000","last_peered":"2024-05-10T04:03:30.252518+0000","last_clean":"2024-05-10T04:03:30.252518+0000","last_became_active":"2024-05-10T04:03:30.251919+0000","last_became_peered":"2024-05-10T04:03:30.251919+0000","last_unstale":"2024-05-10T04:03:30.252518+0000","last_undegraded":"2024-05-10T04:03:30.252518+0000","last_fullsized":"2024-05-10T04:03:30.252518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:05:29.247977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.206678+0000","last_change":"2024-05-10T04:03:30.206145+0000","last_active":"2024-05-10T04:03:30.206678+0000","last_peered":"2024-05-10T04:03:30.206678+0000","last_clean":"2024-05-10T04:03:30.206678+0000","last_became_active":"2024-05-10T04:03:30.200138+0000","last_became_peered":"2024-05-10T04:03:30.200138+0000","last_unstale":"2024-05-10T04:03:30.206678+0000","last_undegraded":"2024-05-10T04:03:30.206678+0000","last_fullsized":"2024-05-10T04:03:30.206678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:49.885012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200208+0000","last_change":"2024-05-10T04:02:31.628124+0000","last_active":"2024-05-10T04:03:29.200208+0000","last_peered":"2024-05-10T04:03:29.200208+0000","last_clean":"2024-05-10T04:03:29.200208+0000","last_became_active":"2024-05-10T04:02:31.627789+0000","last_became_peered":"2024-05-10T04:02:31.627789+0000","last_unstale":"2024-05-10T04:03:29.200208+0000","last_undegraded":"2024-05-10T04:03:29.200208+0000","last_fullsized":"2024-05-10T04:03:29.200208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:40:11.240726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194237+0000","last_change":"2024-05-10T04:02:35.649536+0000","last_active":"2024-05-10T04:03:29.194237+0000","last_peered":"2024-05-10T04:03:29.194237+0000","last_clean":"2024-05-10T04:03:29.194237+0000","last_became_active":"2024-05-10T04:02:35.649130+0000","last_became_peered":"2024-05-10T04:02:35.649130+0000","last_unstale":"2024-05-10T04:03:29.194237+0000","last_undegraded":"2024-05-10T04:03:29.194237+0000","last_fullsized":"2024-05-10T04:03:29.194237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:03.870071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196512+0000","last_change":"2024-05-10T04:02:31.192627+0000","last_active":"2024-05-10T04:03:29.196512+0000","last_peered":"2024-05-10T04:03:29.196512+0000","last_clean":"2024-05-10T04:03:29.196512+0000","last_became_active":"2024-05-10T04:02:31.191433+0000","last_became_peered":"2024-05-10T04:02:31.191433+0000","last_unstale":"2024-05-10T04:03:29.196512+0000","last_undegraded":"2024-05-10T04:03:29.196512+0000","last_fullsized":"2024-05-10T04:03:29.196512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:29.235483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193100+0000","last_change":"2024-05-10T04:02:31.641192+0000","last_active":"2024-05-10T04:03:29.193100+0000","last_peered":"2024-05-10T04:03:29.193100+0000","last_clean":"2024-05-10T04:03:29.193100+0000","last_became_active":"2024-05-10T04:02:31.640540+0000","last_became_peered":"2024-05-10T04:02:31.640540+0000","last_unstale":"2024-05-10T04:03:29.193100+0000","last_undegraded":"2024-05-10T04:03:29.193100+0000","last_fullsized":"2024-05-10T04:03:29.193100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:36.734231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.200392+0000","last_change":"2024-05-10T04:03:30.200392+0000","last_active":"2024-05-10T04:03:30.200392+0000","last_peered":"2024-05-10T04:03:30.200392+0000","last_clean":"2024-05-10T04:03:30.200392+0000","last_became_active":"2024-05-10T04:03:30.199878+0000","last_became_peered":"2024-05-10T04:03:30.199878+0000","last_unstale":"2024-05-10T04:03:30.200392+0000","last_undegraded":"2024-05-10T04:03:30.200392+0000","last_fullsized":"2024-05-10T04:03:30.200392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:33:33.143992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:04.298481+0000","last_change":"2024-05-10T04:02:31.198037+0000","last_active":"2024-05-10T04:04:04.298481+0000","last_peered":"2024-05-10T04:04:04.298481+0000","last_clean":"2024-05-10T04:04:04.298481+0000","last_became_active":"2024-05-10T04:02:31.197679+0000","last_became_peered":"2024-05-10T04:02:31.197679+0000","last_unstale":"2024-05-10T04:04:04.298481+0000","last_undegraded":"2024-05-10T04:04:04.298481+0000","last_fullsized":"2024-05-10T04:04:04.298481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:19.059645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196289+0000","last_change":"2024-05-10T04:02:31.630020+0000","last_active":"2024-05-10T04:03:29.196289+0000","last_peered":"2024-05-10T04:03:29.196289+0000","last_clean":"2024-05-10T04:03:29.196289+0000","last_became_active":"2024-05-10T04:02:31.629738+0000","last_became_peered":"2024-05-10T04:02:31.629738+0000","last_unstale":"2024-05-10T04:03:29.196289+0000","last_undegraded":"2024-05-10T04:03:29.196289+0000","last_fullsized":"2024-05-10T04:03:29.196289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:58:42.317143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188383+0000","last_change":"2024-05-10T04:02:35.647242+0000","last_active":"2024-05-10T04:03:29.188383+0000","last_peered":"2024-05-10T04:03:29.188383+0000","last_clean":"2024-05-10T04:03:29.188383+0000","last_became_active":"2024-05-10T04:02:35.646774+0000","last_became_peered":"2024-05-10T04:02:35.646774+0000","last_unstale":"2024-05-10T04:03:29.188383+0000","last_undegraded":"2024-05-10T04:03:29.188383+0000","last_fullsized":"2024-05-10T04:03:29.188383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:20:05.602820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198723+0000","last_change":"2024-05-10T04:02:31.191981+0000","last_active":"2024-05-10T04:03:29.198723+0000","last_peered":"2024-05-10T04:03:29.198723+0000","last_clean":"2024-05-10T04:03:29.198723+0000","last_became_active":"2024-05-10T04:02:31.191632+0000","last_became_peered":"2024-05-10T04:02:31.191632+0000","last_unstale":"2024-05-10T04:03:29.198723+0000","last_undegraded":"2024-05-10T04:03:29.198723+0000","last_fullsized":"2024-05-10T04:03:29.198723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:44:52.744661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194359+0000","last_change":"2024-05-10T04:02:31.643240+0000","last_active":"2024-05-10T04:03:29.194359+0000","last_peered":"2024-05-10T04:03:29.194359+0000","last_clean":"2024-05-10T04:03:29.194359+0000","last_became_active":"2024-05-10T04:02:31.642891+0000","last_became_peered":"2024-05-10T04:02:31.642891+0000","last_unstale":"2024-05-10T04:03:29.194359+0000","last_undegraded":"2024-05-10T04:03:29.194359+0000","last_fullsized":"2024-05-10T04:03:29.194359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:41:42.901192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195070+0000","last_change":"2024-05-10T04:02:35.652370+0000","last_active":"2024-05-10T04:03:29.195070+0000","last_peered":"2024-05-10T04:03:29.195070+0000","last_clean":"2024-05-10T04:03:29.195070+0000","last_became_active":"2024-05-10T04:02:35.651803+0000","last_became_peered":"2024-05-10T04:02:35.651803+0000","last_unstale":"2024-05-10T04:03:29.195070+0000","last_undegraded":"2024-05-10T04:03:29.195070+0000","last_fullsized":"2024-05-10T04:03:29.195070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:33:36.049311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192316+0000","last_change":"2024-05-10T04:02:31.189053+0000","last_active":"2024-05-10T04:03:29.192316+0000","last_peered":"2024-05-10T04:03:29.192316+0000","last_clean":"2024-05-10T04:03:29.192316+0000","last_became_active":"2024-05-10T04:02:31.188614+0000","last_became_peered":"2024-05-10T04:02:31.188614+0000","last_unstale":"2024-05-10T04:03:29.192316+0000","last_undegraded":"2024-05-10T04:03:29.192316+0000","last_fullsized":"2024-05-10T04:03:29.192316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:59:37.750507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198409+0000","last_change":"2024-05-10T04:02:31.630379+0000","last_active":"2024-05-10T04:03:29.198409+0000","last_peered":"2024-05-10T04:03:29.198409+0000","last_clean":"2024-05-10T04:03:29.198409+0000","last_became_active":"2024-05-10T04:02:31.630094+0000","last_became_peered":"2024-05-10T04:02:31.630094+0000","last_unstale":"2024-05-10T04:03:29.198409+0000","last_undegraded":"2024-05-10T04:03:29.198409+0000","last_fullsized":"2024-05-10T04:03:29.198409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:05.699923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197360+0000","last_change":"2024-05-10T04:02:35.650770+0000","last_active":"2024-05-10T04:03:29.197360+0000","last_peered":"2024-05-10T04:03:29.197360+0000","last_clean":"2024-05-10T04:03:29.197360+0000","last_became_active":"2024-05-10T04:02:35.650351+0000","last_became_peered":"2024-05-10T04:02:35.650351+0000","last_unstale":"2024-05-10T04:03:29.197360+0000","last_undegraded":"2024-05-10T04:03:29.197360+0000","last_fullsized":"2024-05-10T04:03:29.197360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:41:06.728126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192986+0000","last_change":"2024-05-10T04:02:31.189557+0000","last_active":"2024-05-10T04:03:29.192986+0000","last_peered":"2024-05-10T04:03:29.192986+0000","last_clean":"2024-05-10T04:03:29.192986+0000","last_became_active":"2024-05-10T04:02:31.189206+0000","last_became_peered":"2024-05-10T04:02:31.189206+0000","last_unstale":"2024-05-10T04:03:29.192986+0000","last_undegraded":"2024-05-10T04:03:29.192986+0000","last_fullsized":"2024-05-10T04:03:29.192986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:11:08.225965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192249+0000","last_change":"2024-05-10T04:02:31.633561+0000","last_active":"2024-05-10T04:03:29.192249+0000","last_peered":"2024-05-10T04:03:29.192249+0000","last_clean":"2024-05-10T04:03:29.192249+0000","last_became_active":"2024-05-10T04:02:31.633224+0000","last_became_peered":"2024-05-10T04:02:31.633224+0000","last_unstale":"2024-05-10T04:03:29.192249+0000","last_undegraded":"2024-05-10T04:03:29.192249+0000","last_fullsized":"2024-05-10T04:03:29.192249+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:32.901146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198675+0000","last_change":"2024-05-10T04:02:35.649327+0000","last_active":"2024-05-10T04:03:29.198675+0000","last_peered":"2024-05-10T04:03:29.198675+0000","last_clean":"2024-05-10T04:03:29.198675+0000","last_became_active":"2024-05-10T04:02:35.648982+0000","last_became_peered":"2024-05-10T04:02:35.648982+0000","last_unstale":"2024-05-10T04:03:29.198675+0000","last_undegraded":"2024-05-10T04:03:29.198675+0000","last_fullsized":"2024-05-10T04:03:29.198675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:25.405748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":87,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:03.296425+0000","last_change":"2024-05-10T04:02:31.186447+0000","last_active":"2024-05-10T04:04:03.296425+0000","last_peered":"2024-05-10T04:04:03.296425+0000","last_clean":"2024-05-10T04:04:03.296425+0000","last_became_active":"2024-05-10T04:02:31.186073+0000","last_became_peered":"2024-05-10T04:02:31.186073+0000","last_unstale":"2024-05-10T04:04:03.296425+0000","last_undegraded":"2024-05-10T04:04:03.296425+0000","last_fullsized":"2024-05-10T04:04:03.296425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:53.381147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200415+0000","last_change":"2024-05-10T04:02:31.633772+0000","last_active":"2024-05-10T04:03:29.200415+0000","last_peered":"2024-05-10T04:03:29.200415+0000","last_clean":"2024-05-10T04:03:29.200415+0000","last_became_active":"2024-05-10T04:02:31.629592+0000","last_became_peered":"2024-05-10T04:02:31.629592+0000","last_unstale":"2024-05-10T04:03:29.200415+0000","last_undegraded":"2024-05-10T04:03:29.200415+0000","last_fullsized":"2024-05-10T04:03:29.200415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:38:37.342351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191885+0000","last_change":"2024-05-10T04:02:35.646960+0000","last_active":"2024-05-10T04:03:29.191885+0000","last_peered":"2024-05-10T04:03:29.191885+0000","last_clean":"2024-05-10T04:03:29.191885+0000","last_became_active":"2024-05-10T04:02:35.646598+0000","last_became_peered":"2024-05-10T04:02:35.646598+0000","last_unstale":"2024-05-10T04:03:29.191885+0000","last_undegraded":"2024-05-10T04:03:29.191885+0000","last_fullsized":"2024-05-10T04:03:29.191885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:40:10.478745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192245+0000","last_change":"2024-05-10T04:02:31.192846+0000","last_active":"2024-05-10T04:03:29.192245+0000","last_peered":"2024-05-10T04:03:29.192245+0000","last_clean":"2024-05-10T04:03:29.192245+0000","last_became_active":"2024-05-10T04:02:31.192514+0000","last_became_peered":"2024-05-10T04:02:31.192514+0000","last_unstale":"2024-05-10T04:03:29.192245+0000","last_undegraded":"2024-05-10T04:03:29.192245+0000","last_fullsized":"2024-05-10T04:03:29.192245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:50:32.649440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197690+0000","last_change":"2024-05-10T04:02:31.641578+0000","last_active":"2024-05-10T04:03:29.197690+0000","last_peered":"2024-05-10T04:03:29.197690+0000","last_clean":"2024-05-10T04:03:29.197690+0000","last_became_active":"2024-05-10T04:02:31.641268+0000","last_became_peered":"2024-05-10T04:02:31.641268+0000","last_unstale":"2024-05-10T04:03:29.197690+0000","last_undegraded":"2024-05-10T04:03:29.197690+0000","last_fullsized":"2024-05-10T04:03:29.197690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:31:00.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195504+0000","last_change":"2024-05-10T04:02:35.646754+0000","last_active":"2024-05-10T04:03:29.195504+0000","last_peered":"2024-05-10T04:03:29.195504+0000","last_clean":"2024-05-10T04:03:29.195504+0000","last_became_active":"2024-05-10T04:02:35.646342+0000","last_became_peered":"2024-05-10T04:02:35.646342+0000","last_unstale":"2024-05-10T04:03:29.195504+0000","last_undegraded":"2024-05-10T04:03:29.195504+0000","last_fullsized":"2024-05-10T04:03:29.195504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:28:33.373312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192990+0000","last_change":"2024-05-10T04:02:31.186427+0000","last_active":"2024-05-10T04:03:29.192990+0000","last_peered":"2024-05-10T04:03:29.192990+0000","last_clean":"2024-05-10T04:03:29.192990+0000","last_became_active":"2024-05-10T04:02:31.185937+0000","last_became_peered":"2024-05-10T04:02:31.185937+0000","last_unstale":"2024-05-10T04:03:29.192990+0000","last_undegraded":"2024-05-10T04:03:29.192990+0000","last_fullsized":"2024-05-10T04:03:29.192990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.818126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200293+0000","last_change":"2024-05-10T04:02:31.645582+0000","last_active":"2024-05-10T04:03:29.200293+0000","last_peered":"2024-05-10T04:03:29.200293+0000","last_clean":"2024-05-10T04:03:29.200293+0000","last_became_active":"2024-05-10T04:02:31.645259+0000","last_became_peered":"2024-05-10T04:02:31.645259+0000","last_unstale":"2024-05-10T04:03:29.200293+0000","last_undegraded":"2024-05-10T04:03:29.200293+0000","last_fullsized":"2024-05-10T04:03:29.200293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:17:31.818116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192983+0000","last_change":"2024-05-10T04:02:35.647341+0000","last_active":"2024-05-10T04:03:29.192983+0000","last_peered":"2024-05-10T04:03:29.192983+0000","last_clean":"2024-05-10T04:03:29.192983+0000","last_became_active":"2024-05-10T04:02:35.646806+0000","last_became_peered":"2024-05-10T04:02:35.646806+0000","last_unstale":"2024-05-10T04:03:29.192983+0000","last_undegraded":"2024-05-10T04:03:29.192983+0000","last_fullsized":"2024-05-10T04:03:29.192983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:01:51.252683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":193,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:04.298928+0000","last_change":"2024-05-10T04:02:31.195377+0000","last_active":"2024-05-10T04:04:04.298928+0000","last_peered":"2024-05-10T04:04:04.298928+0000","last_clean":"2024-05-10T04:04:04.298928+0000","last_became_active":"2024-05-10T04:02:31.194965+0000","last_became_peered":"2024-05-10T04:02:31.194965+0000","last_unstale":"2024-05-10T04:04:04.298928+0000","last_undegraded":"2024-05-10T04:04:04.298928+0000","last_fullsized":"2024-05-10T04:04:04.298928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:11:09.237525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62045,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":144,"num_read_kb":133,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198701+0000","last_change":"2024-05-10T04:02:31.629600+0000","last_active":"2024-05-10T04:03:29.198701+0000","last_peered":"2024-05-10T04:03:29.198701+0000","last_clean":"2024-05-10T04:03:29.198701+0000","last_became_active":"2024-05-10T04:02:31.629264+0000","last_became_peered":"2024-05-10T04:02:31.629264+0000","last_unstale":"2024-05-10T04:03:29.198701+0000","last_undegraded":"2024-05-10T04:03:29.198701+0000","last_fullsized":"2024-05-10T04:03:29.198701+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:45:08.867783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193519+0000","last_change":"2024-05-10T04:02:35.649468+0000","last_active":"2024-05-10T04:03:29.193519+0000","last_peered":"2024-05-10T04:03:29.193519+0000","last_clean":"2024-05-10T04:03:29.193519+0000","last_became_active":"2024-05-10T04:02:35.649053+0000","last_became_peered":"2024-05-10T04:02:35.649053+0000","last_unstale":"2024-05-10T04:03:29.193519+0000","last_undegraded":"2024-05-10T04:03:29.193519+0000","last_fullsized":"2024-05-10T04:03:29.193519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:01:05.761287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188394+0000","last_change":"2024-05-10T04:02:31.195169+0000","last_active":"2024-05-10T04:03:29.188394+0000","last_peered":"2024-05-10T04:03:29.188394+0000","last_clean":"2024-05-10T04:03:29.188394+0000","last_became_active":"2024-05-10T04:02:31.194720+0000","last_became_peered":"2024-05-10T04:02:31.194720+0000","last_unstale":"2024-05-10T04:03:29.188394+0000","last_undegraded":"2024-05-10T04:03:29.188394+0000","last_fullsized":"2024-05-10T04:03:29.188394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:42.727841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191480+0000","last_change":"2024-05-10T04:02:31.631739+0000","last_active":"2024-05-10T04:03:29.191480+0000","last_peered":"2024-05-10T04:03:29.191480+0000","last_clean":"2024-05-10T04:03:29.191480+0000","last_became_active":"2024-05-10T04:02:31.631401+0000","last_became_peered":"2024-05-10T04:02:31.631401+0000","last_unstale":"2024-05-10T04:03:29.191480+0000","last_undegraded":"2024-05-10T04:03:29.191480+0000","last_fullsized":"2024-05-10T04:03:29.191480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:06.958328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196041+0000","last_change":"2024-05-10T04:02:35.651484+0000","last_active":"2024-05-10T04:03:29.196041+0000","last_peered":"2024-05-10T04:03:29.196041+0000","last_clean":"2024-05-10T04:03:29.196041+0000","last_became_active":"2024-05-10T04:02:35.651069+0000","last_became_peered":"2024-05-10T04:02:35.651069+0000","last_unstale":"2024-05-10T04:03:29.196041+0000","last_undegraded":"2024-05-10T04:03:29.196041+0000","last_fullsized":"2024-05-10T04:03:29.196041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:26:45.229101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192929+0000","last_change":"2024-05-10T04:02:31.186314+0000","last_active":"2024-05-10T04:03:29.192929+0000","last_peered":"2024-05-10T04:03:29.192929+0000","last_clean":"2024-05-10T04:03:29.192929+0000","last_became_active":"2024-05-10T04:02:31.185861+0000","last_became_peered":"2024-05-10T04:02:31.185861+0000","last_unstale":"2024-05-10T04:03:29.192929+0000","last_undegraded":"2024-05-10T04:03:29.192929+0000","last_fullsized":"2024-05-10T04:03:29.192929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:07.386978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191954+0000","last_change":"2024-05-10T04:02:31.637151+0000","last_active":"2024-05-10T04:03:29.191954+0000","last_peered":"2024-05-10T04:03:29.191954+0000","last_clean":"2024-05-10T04:03:29.191954+0000","last_became_active":"2024-05-10T04:02:31.636745+0000","last_became_peered":"2024-05-10T04:02:31.636745+0000","last_unstale":"2024-05-10T04:03:29.191954+0000","last_undegraded":"2024-05-10T04:03:29.191954+0000","last_fullsized":"2024-05-10T04:03:29.191954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:59.804391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192369+0000","last_change":"2024-05-10T04:02:35.656108+0000","last_active":"2024-05-10T04:03:29.192369+0000","last_peered":"2024-05-10T04:03:29.192369+0000","last_clean":"2024-05-10T04:03:29.192369+0000","last_became_active":"2024-05-10T04:02:35.655664+0000","last_became_peered":"2024-05-10T04:02:35.655664+0000","last_unstale":"2024-05-10T04:03:29.192369+0000","last_undegraded":"2024-05-10T04:03:29.192369+0000","last_fullsized":"2024-05-10T04:03:29.192369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:10.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:03.297197+0000","last_change":"2024-05-10T04:02:31.189432+0000","last_active":"2024-05-10T04:04:03.297197+0000","last_peered":"2024-05-10T04:04:03.297197+0000","last_clean":"2024-05-10T04:04:03.297197+0000","last_became_active":"2024-05-10T04:02:31.189062+0000","last_became_peered":"2024-05-10T04:02:31.189062+0000","last_unstale":"2024-05-10T04:04:03.297197+0000","last_undegraded":"2024-05-10T04:04:03.297197+0000","last_fullsized":"2024-05-10T04:04:03.297197+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:28.452100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191766+0000","last_change":"2024-05-10T04:02:31.637517+0000","last_active":"2024-05-10T04:03:29.191766+0000","last_peered":"2024-05-10T04:03:29.191766+0000","last_clean":"2024-05-10T04:03:29.191766+0000","last_became_active":"2024-05-10T04:02:31.636518+0000","last_became_peered":"2024-05-10T04:02:31.636518+0000","last_unstale":"2024-05-10T04:03:29.191766+0000","last_undegraded":"2024-05-10T04:03:29.191766+0000","last_fullsized":"2024-05-10T04:03:29.191766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:07:09.732117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192266+0000","last_change":"2024-05-10T04:02:35.649693+0000","last_active":"2024-05-10T04:03:29.192266+0000","last_peered":"2024-05-10T04:03:29.192266+0000","last_clean":"2024-05-10T04:03:29.192266+0000","last_became_active":"2024-05-10T04:02:35.649175+0000","last_became_peered":"2024-05-10T04:02:35.649175+0000","last_unstale":"2024-05-10T04:03:29.192266+0000","last_undegraded":"2024-05-10T04:03:29.192266+0000","last_fullsized":"2024-05-10T04:03:29.192266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:03:52.340444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.211403+0000","last_change":"2024-05-10T04:03:30.211403+0000","last_active":"2024-05-10T04:03:30.211403+0000","last_peered":"2024-05-10T04:03:30.211403+0000","last_clean":"2024-05-10T04:03:30.211403+0000","last_became_active":"2024-05-10T04:03:30.198739+0000","last_became_peered":"2024-05-10T04:03:30.198739+0000","last_unstale":"2024-05-10T04:03:30.211403+0000","last_undegraded":"2024-05-10T04:03:30.211403+0000","last_fullsized":"2024-05-10T04:03:30.211403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:52.430884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194588+0000","last_change":"2024-05-10T04:02:31.641426+0000","last_active":"2024-05-10T04:03:29.194588+0000","last_peered":"2024-05-10T04:03:29.194588+0000","last_clean":"2024-05-10T04:03:29.194588+0000","last_became_active":"2024-05-10T04:02:31.641054+0000","last_became_peered":"2024-05-10T04:02:31.641054+0000","last_unstale":"2024-05-10T04:03:29.194588+0000","last_undegraded":"2024-05-10T04:03:29.194588+0000","last_fullsized":"2024-05-10T04:03:29.194588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:12:48.472674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194503+0000","last_change":"2024-05-10T04:02:35.653258+0000","last_active":"2024-05-10T04:03:29.194503+0000","last_peered":"2024-05-10T04:03:29.194503+0000","last_clean":"2024-05-10T04:03:29.194503+0000","last_became_active":"2024-05-10T04:02:35.652859+0000","last_became_peered":"2024-05-10T04:02:35.652859+0000","last_unstale":"2024-05-10T04:03:29.194503+0000","last_undegraded":"2024-05-10T04:03:29.194503+0000","last_fullsized":"2024-05-10T04:03:29.194503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:15:57.814963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198019+0000","last_change":"2024-05-10T04:02:31.192231+0000","last_active":"2024-05-10T04:03:29.198019+0000","last_peered":"2024-05-10T04:03:29.198019+0000","last_clean":"2024-05-10T04:03:29.198019+0000","last_became_active":"2024-05-10T04:02:31.191902+0000","last_became_peered":"2024-05-10T04:02:31.191902+0000","last_unstale":"2024-05-10T04:03:29.198019+0000","last_undegraded":"2024-05-10T04:03:29.198019+0000","last_fullsized":"2024-05-10T04:03:29.198019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:18.637112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195378+0000","last_change":"2024-05-10T04:02:31.635514+0000","last_active":"2024-05-10T04:03:29.195378+0000","last_peered":"2024-05-10T04:03:29.195378+0000","last_clean":"2024-05-10T04:03:29.195378+0000","last_became_active":"2024-05-10T04:02:31.635171+0000","last_became_peered":"2024-05-10T04:02:31.635171+0000","last_unstale":"2024-05-10T04:03:29.195378+0000","last_undegraded":"2024-05-10T04:03:29.195378+0000","last_fullsized":"2024-05-10T04:03:29.195378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:48.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.475636+0000","last_change":"2024-05-10T04:03:30.475636+0000","last_active":"2024-05-10T04:03:30.475636+0000","last_peered":"2024-05-10T04:03:30.475636+0000","last_clean":"2024-05-10T04:03:30.475636+0000","last_became_active":"2024-05-10T04:03:30.475203+0000","last_became_peered":"2024-05-10T04:03:30.475203+0000","last_unstale":"2024-05-10T04:03:30.475636+0000","last_undegraded":"2024-05-10T04:03:30.475636+0000","last_fullsized":"2024-05-10T04:03:30.475636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:56:07.461905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194614+0000","last_change":"2024-05-10T04:02:31.190173+0000","last_active":"2024-05-10T04:03:29.194614+0000","last_peered":"2024-05-10T04:03:29.194614+0000","last_clean":"2024-05-10T04:03:29.194614+0000","last_became_active":"2024-05-10T04:02:31.189840+0000","last_became_peered":"2024-05-10T04:02:31.189840+0000","last_unstale":"2024-05-10T04:03:29.194614+0000","last_undegraded":"2024-05-10T04:03:29.194614+0000","last_fullsized":"2024-05-10T04:03:29.194614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:31:49.382482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199097+0000","last_change":"2024-05-10T04:02:31.629497+0000","last_active":"2024-05-10T04:03:29.199097+0000","last_peered":"2024-05-10T04:03:29.199097+0000","last_clean":"2024-05-10T04:03:29.199097+0000","last_became_active":"2024-05-10T04:02:31.629182+0000","last_became_peered":"2024-05-10T04:02:31.629182+0000","last_unstale":"2024-05-10T04:03:29.199097+0000","last_undegraded":"2024-05-10T04:03:29.199097+0000","last_fullsized":"2024-05-10T04:03:29.199097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:11:33.478737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192656+0000","last_change":"2024-05-10T04:02:35.647249+0000","last_active":"2024-05-10T04:03:29.192656+0000","last_peered":"2024-05-10T04:03:29.192656+0000","last_clean":"2024-05-10T04:03:29.192656+0000","last_became_active":"2024-05-10T04:02:35.646716+0000","last_became_peered":"2024-05-10T04:02:35.646716+0000","last_unstale":"2024-05-10T04:03:29.192656+0000","last_undegraded":"2024-05-10T04:03:29.192656+0000","last_fullsized":"2024-05-10T04:03:29.192656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:44.651836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191715+0000","last_change":"2024-05-10T04:02:31.191241+0000","last_active":"2024-05-10T04:03:29.191715+0000","last_peered":"2024-05-10T04:03:29.191715+0000","last_clean":"2024-05-10T04:03:29.191715+0000","last_became_active":"2024-05-10T04:02:31.190888+0000","last_became_peered":"2024-05-10T04:02:31.190888+0000","last_unstale":"2024-05-10T04:03:29.191715+0000","last_undegraded":"2024-05-10T04:03:29.191715+0000","last_fullsized":"2024-05-10T04:03:29.191715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:55.256119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198888+0000","last_change":"2024-05-10T04:02:31.638394+0000","last_active":"2024-05-10T04:03:29.198888+0000","last_peered":"2024-05-10T04:03:29.198888+0000","last_clean":"2024-05-10T04:03:29.198888+0000","last_became_active":"2024-05-10T04:02:31.638025+0000","last_became_peered":"2024-05-10T04:02:31.638025+0000","last_unstale":"2024-05-10T04:03:29.198888+0000","last_undegraded":"2024-05-10T04:03:29.198888+0000","last_fullsized":"2024-05-10T04:03:29.198888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:13.088393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192346+0000","last_change":"2024-05-10T04:02:35.899898+0000","last_active":"2024-05-10T04:03:29.192346+0000","last_peered":"2024-05-10T04:03:29.192346+0000","last_clean":"2024-05-10T04:03:29.192346+0000","last_became_active":"2024-05-10T04:02:35.899485+0000","last_became_peered":"2024-05-10T04:02:35.899485+0000","last_unstale":"2024-05-10T04:03:29.192346+0000","last_undegraded":"2024-05-10T04:03:29.192346+0000","last_fullsized":"2024-05-10T04:03:29.192346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:26:50.909122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193219+0000","last_change":"2024-05-10T04:02:31.640923+0000","last_active":"2024-05-10T04:03:29.193219+0000","last_peered":"2024-05-10T04:03:29.193219+0000","last_clean":"2024-05-10T04:03:29.193219+0000","last_became_active":"2024-05-10T04:02:31.640490+0000","last_became_peered":"2024-05-10T04:02:31.640490+0000","last_unstale":"2024-05-10T04:03:29.193219+0000","last_undegraded":"2024-05-10T04:03:29.193219+0000","last_fullsized":"2024-05-10T04:03:29.193219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:31:58.585789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:02.499077+0000","last_change":"2024-05-10T04:02:31.191326+0000","last_active":"2024-05-10T04:04:02.499077+0000","last_peered":"2024-05-10T04:04:02.499077+0000","last_clean":"2024-05-10T04:04:02.499077+0000","last_became_active":"2024-05-10T04:02:31.190974+0000","last_became_peered":"2024-05-10T04:02:31.190974+0000","last_unstale":"2024-05-10T04:04:02.499077+0000","last_undegraded":"2024-05-10T04:04:02.499077+0000","last_fullsized":"2024-05-10T04:04:02.499077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:43.670619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192264+0000","last_change":"2024-05-10T04:02:35.647526+0000","last_active":"2024-05-10T04:03:29.192264+0000","last_peered":"2024-05-10T04:03:29.192264+0000","last_clean":"2024-05-10T04:03:29.192264+0000","last_became_active":"2024-05-10T04:02:35.647018+0000","last_became_peered":"2024-05-10T04:02:35.647018+0000","last_unstale":"2024-05-10T04:03:29.192264+0000","last_undegraded":"2024-05-10T04:03:29.192264+0000","last_fullsized":"2024-05-10T04:03:29.192264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:17.173497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199043+0000","last_change":"2024-05-10T04:02:31.632263+0000","last_active":"2024-05-10T04:03:29.199043+0000","last_peered":"2024-05-10T04:03:29.199043+0000","last_clean":"2024-05-10T04:03:29.199043+0000","last_became_active":"2024-05-10T04:02:31.631785+0000","last_became_peered":"2024-05-10T04:02:31.631785+0000","last_unstale":"2024-05-10T04:03:29.199043+0000","last_undegraded":"2024-05-10T04:03:29.199043+0000","last_fullsized":"2024-05-10T04:03:29.199043+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:30.585862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195400+0000","last_change":"2024-05-10T04:02:31.182262+0000","last_active":"2024-05-10T04:03:29.195400+0000","last_peered":"2024-05-10T04:03:29.195400+0000","last_clean":"2024-05-10T04:03:29.195400+0000","last_became_active":"2024-05-10T04:02:31.181802+0000","last_became_peered":"2024-05-10T04:02:31.181802+0000","last_unstale":"2024-05-10T04:03:29.195400+0000","last_undegraded":"2024-05-10T04:03:29.195400+0000","last_fullsized":"2024-05-10T04:03:29.195400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:45:40.696571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":327,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191496+0000","last_change":"2024-05-10T04:01:41.866365+0000","last_active":"2024-05-10T04:03:29.191496+0000","last_peered":"2024-05-10T04:03:29.191496+0000","last_clean":"2024-05-10T04:03:29.191496+0000","last_became_active":"2024-05-10T04:01:41.865825+0000","last_became_peered":"2024-05-10T04:01:41.865825+0000","last_unstale":"2024-05-10T04:03:29.191496+0000","last_undegraded":"2024-05-10T04:03:29.191496+0000","last_fullsized":"2024-05-10T04:03:29.191496+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_clean_scrub_stamp":"2024-05-10T03:59:19.853473+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:28:29.598339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193669+0000","last_change":"2024-05-10T04:02:35.900841+0000","last_active":"2024-05-10T04:03:29.193669+0000","last_peered":"2024-05-10T04:03:29.193669+0000","last_clean":"2024-05-10T04:03:29.193669+0000","last_became_active":"2024-05-10T04:02:35.900323+0000","last_became_peered":"2024-05-10T04:02:35.900323+0000","last_unstale":"2024-05-10T04:03:29.193669+0000","last_undegraded":"2024-05-10T04:03:29.193669+0000","last_fullsized":"2024-05-10T04:03:29.193669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:00.554804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.196577+0000","last_change":"2024-05-10T04:03:30.196577+0000","last_active":"2024-05-10T04:03:30.196577+0000","last_peered":"2024-05-10T04:03:30.196577+0000","last_clean":"2024-05-10T04:03:30.196577+0000","last_became_active":"2024-05-10T04:03:30.196249+0000","last_became_peered":"2024-05-10T04:03:30.196249+0000","last_unstale":"2024-05-10T04:03:30.196577+0000","last_undegraded":"2024-05-10T04:03:30.196577+0000","last_fullsized":"2024-05-10T04:03:30.196577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:48:38.347871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191634+0000","last_change":"2024-05-10T04:02:31.187834+0000","last_active":"2024-05-10T04:03:29.191634+0000","last_peered":"2024-05-10T04:03:29.191634+0000","last_clean":"2024-05-10T04:03:29.191634+0000","last_became_active":"2024-05-10T04:02:31.187440+0000","last_became_peered":"2024-05-10T04:02:31.187440+0000","last_unstale":"2024-05-10T04:03:29.191634+0000","last_undegraded":"2024-05-10T04:03:29.191634+0000","last_fullsized":"2024-05-10T04:03:29.191634+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:50:37.678245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194942+0000","last_change":"2024-05-10T04:02:35.649664+0000","last_active":"2024-05-10T04:03:29.194942+0000","last_peered":"2024-05-10T04:03:29.194942+0000","last_clean":"2024-05-10T04:03:29.194942+0000","last_became_active":"2024-05-10T04:02:35.649076+0000","last_became_peered":"2024-05-10T04:02:35.649076+0000","last_unstale":"2024-05-10T04:03:29.194942+0000","last_undegraded":"2024-05-10T04:03:29.194942+0000","last_fullsized":"2024-05-10T04:03:29.194942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:14:42.864845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198666+0000","last_change":"2024-05-10T04:02:31.192126+0000","last_active":"2024-05-10T04:03:29.198666+0000","last_peered":"2024-05-10T04:03:29.198666+0000","last_clean":"2024-05-10T04:03:29.198666+0000","last_became_active":"2024-05-10T04:02:31.191771+0000","last_became_peered":"2024-05-10T04:02:31.191771+0000","last_unstale":"2024-05-10T04:03:29.198666+0000","last_undegraded":"2024-05-10T04:03:29.198666+0000","last_fullsized":"2024-05-10T04:03:29.198666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:43.550535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192799+0000","last_change":"2024-05-10T04:02:31.620788+0000","last_active":"2024-05-10T04:03:29.192799+0000","last_peered":"2024-05-10T04:03:29.192799+0000","last_clean":"2024-05-10T04:03:29.192799+0000","last_became_active":"2024-05-10T04:02:31.620456+0000","last_became_peered":"2024-05-10T04:02:31.620456+0000","last_unstale":"2024-05-10T04:03:29.192799+0000","last_undegraded":"2024-05-10T04:03:29.192799+0000","last_fullsized":"2024-05-10T04:03:29.192799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:02.956030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198321+0000","last_change":"2024-05-10T04:02:35.899935+0000","last_active":"2024-05-10T04:03:29.198321+0000","last_peered":"2024-05-10T04:03:29.198321+0000","last_clean":"2024-05-10T04:03:29.198321+0000","last_became_active":"2024-05-10T04:02:35.899488+0000","last_became_peered":"2024-05-10T04:02:35.899488+0000","last_unstale":"2024-05-10T04:03:29.198321+0000","last_undegraded":"2024-05-10T04:03:29.198321+0000","last_fullsized":"2024-05-10T04:03:29.198321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:28.600463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195179+0000","last_change":"2024-05-10T04:02:31.188605+0000","last_active":"2024-05-10T04:03:29.195179+0000","last_peered":"2024-05-10T04:03:29.195179+0000","last_clean":"2024-05-10T04:03:29.195179+0000","last_became_active":"2024-05-10T04:02:31.188283+0000","last_became_peered":"2024-05-10T04:02:31.188283+0000","last_unstale":"2024-05-10T04:03:29.195179+0000","last_undegraded":"2024-05-10T04:03:29.195179+0000","last_fullsized":"2024-05-10T04:03:29.195179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:40:03.940269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199712+0000","last_change":"2024-05-10T04:02:31.629476+0000","last_active":"2024-05-10T04:03:29.199712+0000","last_peered":"2024-05-10T04:03:29.199712+0000","last_clean":"2024-05-10T04:03:29.199712+0000","last_became_active":"2024-05-10T04:02:31.629104+0000","last_became_peered":"2024-05-10T04:02:31.629104+0000","last_unstale":"2024-05-10T04:03:29.199712+0000","last_undegraded":"2024-05-10T04:03:29.199712+0000","last_fullsized":"2024-05-10T04:03:29.199712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:57.873755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188055+0000","last_change":"2024-05-10T04:02:35.644094+0000","last_active":"2024-05-10T04:03:29.188055+0000","last_peered":"2024-05-10T04:03:29.188055+0000","last_clean":"2024-05-10T04:03:29.188055+0000","last_became_active":"2024-05-10T04:02:35.643382+0000","last_became_peered":"2024-05-10T04:02:35.643382+0000","last_unstale":"2024-05-10T04:03:29.188055+0000","last_undegraded":"2024-05-10T04:03:29.188055+0000","last_fullsized":"2024-05-10T04:03:29.188055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:33.250052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195430+0000","last_change":"2024-05-10T04:02:31.189302+0000","last_active":"2024-05-10T04:03:29.195430+0000","last_peered":"2024-05-10T04:03:29.195430+0000","last_clean":"2024-05-10T04:03:29.195430+0000","last_became_active":"2024-05-10T04:02:31.188323+0000","last_became_peered":"2024-05-10T04:02:31.188323+0000","last_unstale":"2024-05-10T04:03:29.195430+0000","last_undegraded":"2024-05-10T04:03:29.195430+0000","last_fullsized":"2024-05-10T04:03:29.195430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:11:02.922884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197540+0000","last_change":"2024-05-10T04:02:31.635864+0000","last_active":"2024-05-10T04:03:29.197540+0000","last_peered":"2024-05-10T04:03:29.197540+0000","last_clean":"2024-05-10T04:03:29.197540+0000","last_became_active":"2024-05-10T04:02:31.635512+0000","last_became_peered":"2024-05-10T04:02:31.635512+0000","last_unstale":"2024-05-10T04:03:29.197540+0000","last_undegraded":"2024-05-10T04:03:29.197540+0000","last_fullsized":"2024-05-10T04:03:29.197540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:26:10.957848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194647+0000","last_change":"2024-05-10T04:02:35.900489+0000","last_active":"2024-05-10T04:03:29.194647+0000","last_peered":"2024-05-10T04:03:29.194647+0000","last_clean":"2024-05-10T04:03:29.194647+0000","last_became_active":"2024-05-10T04:02:35.899908+0000","last_became_peered":"2024-05-10T04:02:35.899908+0000","last_unstale":"2024-05-10T04:03:29.194647+0000","last_undegraded":"2024-05-10T04:03:29.194647+0000","last_fullsized":"2024-05-10T04:03:29.194647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:08.386128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199847+0000","last_change":"2024-05-10T04:02:31.193956+0000","last_active":"2024-05-10T04:03:29.199847+0000","last_peered":"2024-05-10T04:03:29.199847+0000","last_clean":"2024-05-10T04:03:29.199847+0000","last_became_active":"2024-05-10T04:02:31.193574+0000","last_became_peered":"2024-05-10T04:02:31.193574+0000","last_unstale":"2024-05-10T04:03:29.199847+0000","last_undegraded":"2024-05-10T04:03:29.199847+0000","last_fullsized":"2024-05-10T04:03:29.199847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:46.369152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195487+0000","last_change":"2024-05-10T04:02:31.633263+0000","last_active":"2024-05-10T04:03:29.195487+0000","last_peered":"2024-05-10T04:03:29.195487+0000","last_clean":"2024-05-10T04:03:29.195487+0000","last_became_active":"2024-05-10T04:02:31.632903+0000","last_became_peered":"2024-05-10T04:02:31.632903+0000","last_unstale":"2024-05-10T04:03:29.195487+0000","last_undegraded":"2024-05-10T04:03:29.195487+0000","last_fullsized":"2024-05-10T04:03:29.195487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:28.047473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192521+0000","last_change":"2024-05-10T04:02:35.656003+0000","last_active":"2024-05-10T04:03:29.192521+0000","last_peered":"2024-05-10T04:03:29.192521+0000","last_clean":"2024-05-10T04:03:29.192521+0000","last_became_active":"2024-05-10T04:02:35.655518+0000","last_became_peered":"2024-05-10T04:02:35.655518+0000","last_unstale":"2024-05-10T04:03:29.192521+0000","last_undegraded":"2024-05-10T04:03:29.192521+0000","last_fullsized":"2024-05-10T04:03:29.192521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:49.696699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198559+0000","last_change":"2024-05-10T04:02:31.195297+0000","last_active":"2024-05-10T04:03:29.198559+0000","last_peered":"2024-05-10T04:03:29.198559+0000","last_clean":"2024-05-10T04:03:29.198559+0000","last_became_active":"2024-05-10T04:02:31.193960+0000","last_became_peered":"2024-05-10T04:02:31.193960+0000","last_unstale":"2024-05-10T04:03:29.198559+0000","last_undegraded":"2024-05-10T04:03:29.198559+0000","last_fullsized":"2024-05-10T04:03:29.198559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:04:10.314700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193216+0000","last_change":"2024-05-10T04:02:31.629247+0000","last_active":"2024-05-10T04:03:29.193216+0000","last_peered":"2024-05-10T04:03:29.193216+0000","last_clean":"2024-05-10T04:03:29.193216+0000","last_became_active":"2024-05-10T04:02:31.628838+0000","last_became_peered":"2024-05-10T04:02:31.628838+0000","last_unstale":"2024-05-10T04:03:29.193216+0000","last_undegraded":"2024-05-10T04:03:29.193216+0000","last_fullsized":"2024-05-10T04:03:29.193216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:46:39.483264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.252785+0000","last_change":"2024-05-10T04:03:30.252785+0000","last_active":"2024-05-10T04:03:30.252785+0000","last_peered":"2024-05-10T04:03:30.252785+0000","last_clean":"2024-05-10T04:03:30.252785+0000","last_became_active":"2024-05-10T04:03:30.248696+0000","last_became_peered":"2024-05-10T04:03:30.248696+0000","last_unstale":"2024-05-10T04:03:30.252785+0000","last_undegraded":"2024-05-10T04:03:30.252785+0000","last_fullsized":"2024-05-10T04:03:30.252785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:55:15.159906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.202901+0000","last_change":"2024-05-10T04:03:30.202456+0000","last_active":"2024-05-10T04:03:30.202901+0000","last_peered":"2024-05-10T04:03:30.202901+0000","last_clean":"2024-05-10T04:03:30.202901+0000","last_became_active":"2024-05-10T04:03:30.197484+0000","last_became_peered":"2024-05-10T04:03:30.197484+0000","last_unstale":"2024-05-10T04:03:30.202901+0000","last_undegraded":"2024-05-10T04:03:30.202901+0000","last_fullsized":"2024-05-10T04:03:30.202901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:15:07.899758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198950+0000","last_change":"2024-05-10T04:02:31.628029+0000","last_active":"2024-05-10T04:03:29.198950+0000","last_peered":"2024-05-10T04:03:29.198950+0000","last_clean":"2024-05-10T04:03:29.198950+0000","last_became_active":"2024-05-10T04:02:31.627667+0000","last_became_peered":"2024-05-10T04:02:31.627667+0000","last_unstale":"2024-05-10T04:03:29.198950+0000","last_undegraded":"2024-05-10T04:03:29.198950+0000","last_fullsized":"2024-05-10T04:03:29.198950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:39:10.878894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194397+0000","last_change":"2024-05-10T04:02:35.900237+0000","last_active":"2024-05-10T04:03:29.194397+0000","last_peered":"2024-05-10T04:03:29.194397+0000","last_clean":"2024-05-10T04:03:29.194397+0000","last_became_active":"2024-05-10T04:02:35.899781+0000","last_became_peered":"2024-05-10T04:02:35.899781+0000","last_unstale":"2024-05-10T04:03:29.194397+0000","last_undegraded":"2024-05-10T04:03:29.194397+0000","last_fullsized":"2024-05-10T04:03:29.194397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:52:46.132557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196543+0000","last_change":"2024-05-10T04:02:31.196920+0000","last_active":"2024-05-10T04:03:29.196543+0000","last_peered":"2024-05-10T04:03:29.196543+0000","last_clean":"2024-05-10T04:03:29.196543+0000","last_became_active":"2024-05-10T04:02:31.196191+0000","last_became_peered":"2024-05-10T04:02:31.196191+0000","last_unstale":"2024-05-10T04:03:29.196543+0000","last_undegraded":"2024-05-10T04:03:29.196543+0000","last_fullsized":"2024-05-10T04:03:29.196543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:45:17.980149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191841+0000","last_change":"2024-05-10T04:02:31.623678+0000","last_active":"2024-05-10T04:03:29.191841+0000","last_peered":"2024-05-10T04:03:29.191841+0000","last_clean":"2024-05-10T04:03:29.191841+0000","last_became_active":"2024-05-10T04:02:31.623090+0000","last_became_peered":"2024-05-10T04:02:31.623090+0000","last_unstale":"2024-05-10T04:03:29.191841+0000","last_undegraded":"2024-05-10T04:03:29.191841+0000","last_fullsized":"2024-05-10T04:03:29.191841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:34.321959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199997+0000","last_change":"2024-05-10T04:02:35.648833+0000","last_active":"2024-05-10T04:03:29.199997+0000","last_peered":"2024-05-10T04:03:29.199997+0000","last_clean":"2024-05-10T04:03:29.199997+0000","last_became_active":"2024-05-10T04:02:35.648391+0000","last_became_peered":"2024-05-10T04:02:35.648391+0000","last_unstale":"2024-05-10T04:03:29.199997+0000","last_undegraded":"2024-05-10T04:03:29.199997+0000","last_fullsized":"2024-05-10T04:03:29.199997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:12.773284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.189688+0000","last_change":"2024-05-10T04:02:31.192900+0000","last_active":"2024-05-10T04:03:29.189688+0000","last_peered":"2024-05-10T04:03:29.189688+0000","last_clean":"2024-05-10T04:03:29.189688+0000","last_became_active":"2024-05-10T04:02:31.192481+0000","last_became_peered":"2024-05-10T04:02:31.192481+0000","last_unstale":"2024-05-10T04:03:29.189688+0000","last_undegraded":"2024-05-10T04:03:29.189688+0000","last_fullsized":"2024-05-10T04:03:29.189688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:17.202383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192511+0000","last_change":"2024-05-10T04:02:31.628490+0000","last_active":"2024-05-10T04:03:29.192511+0000","last_peered":"2024-05-10T04:03:29.192511+0000","last_clean":"2024-05-10T04:03:29.192511+0000","last_became_active":"2024-05-10T04:02:31.627889+0000","last_became_peered":"2024-05-10T04:02:31.627889+0000","last_unstale":"2024-05-10T04:03:29.192511+0000","last_undegraded":"2024-05-10T04:03:29.192511+0000","last_fullsized":"2024-05-10T04:03:29.192511+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:32:55.366759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196442+0000","last_change":"2024-05-10T04:02:35.653530+0000","last_active":"2024-05-10T04:03:29.196442+0000","last_peered":"2024-05-10T04:03:29.196442+0000","last_clean":"2024-05-10T04:03:29.196442+0000","last_became_active":"2024-05-10T04:02:35.653095+0000","last_became_peered":"2024-05-10T04:02:35.653095+0000","last_unstale":"2024-05-10T04:03:29.196442+0000","last_undegraded":"2024-05-10T04:03:29.196442+0000","last_fullsized":"2024-05-10T04:03:29.196442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:09.921713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193200+0000","last_change":"2024-05-10T04:02:31.189434+0000","last_active":"2024-05-10T04:03:29.193200+0000","last_peered":"2024-05-10T04:03:29.193200+0000","last_clean":"2024-05-10T04:03:29.193200+0000","last_became_active":"2024-05-10T04:02:31.188843+0000","last_became_peered":"2024-05-10T04:02:31.188843+0000","last_unstale":"2024-05-10T04:03:29.193200+0000","last_undegraded":"2024-05-10T04:03:29.193200+0000","last_fullsized":"2024-05-10T04:03:29.193200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:31.508794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191797+0000","last_change":"2024-05-10T04:02:31.636775+0000","last_active":"2024-05-10T04:03:29.191797+0000","last_peered":"2024-05-10T04:03:29.191797+0000","last_clean":"2024-05-10T04:03:29.191797+0000","last_became_active":"2024-05-10T04:02:31.635946+0000","last_became_peered":"2024-05-10T04:02:31.635946+0000","last_unstale":"2024-05-10T04:03:29.191797+0000","last_undegraded":"2024-05-10T04:03:29.191797+0000","last_fullsized":"2024-05-10T04:03:29.191797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:48.830097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192629+0000","last_change":"2024-05-10T04:02:35.646991+0000","last_active":"2024-05-10T04:03:29.192629+0000","last_peered":"2024-05-10T04:03:29.192629+0000","last_clean":"2024-05-10T04:03:29.192629+0000","last_became_active":"2024-05-10T04:02:35.646585+0000","last_became_peered":"2024-05-10T04:02:35.646585+0000","last_unstale":"2024-05-10T04:03:29.192629+0000","last_undegraded":"2024-05-10T04:03:29.192629+0000","last_fullsized":"2024-05-10T04:03:29.192629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:32:50.329746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192244+0000","last_change":"2024-05-10T04:02:31.192103+0000","last_active":"2024-05-10T04:03:29.192244+0000","last_peered":"2024-05-10T04:03:29.192244+0000","last_clean":"2024-05-10T04:03:29.192244+0000","last_became_active":"2024-05-10T04:02:31.191378+0000","last_became_peered":"2024-05-10T04:02:31.191378+0000","last_unstale":"2024-05-10T04:03:29.192244+0000","last_undegraded":"2024-05-10T04:03:29.192244+0000","last_fullsized":"2024-05-10T04:03:29.192244+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:55.298158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192829+0000","last_change":"2024-05-10T04:02:31.628587+0000","last_active":"2024-05-10T04:03:29.192829+0000","last_peered":"2024-05-10T04:03:29.192829+0000","last_clean":"2024-05-10T04:03:29.192829+0000","last_became_active":"2024-05-10T04:02:31.627296+0000","last_became_peered":"2024-05-10T04:02:31.627296+0000","last_unstale":"2024-05-10T04:03:29.192829+0000","last_undegraded":"2024-05-10T04:03:29.192829+0000","last_fullsized":"2024-05-10T04:03:29.192829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:04:09.783266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193327+0000","last_change":"2024-05-10T04:02:31.634668+0000","last_active":"2024-05-10T04:03:29.193327+0000","last_peered":"2024-05-10T04:03:29.193327+0000","last_clean":"2024-05-10T04:03:29.193327+0000","last_became_active":"2024-05-10T04:02:31.634321+0000","last_became_peered":"2024-05-10T04:02:31.634321+0000","last_unstale":"2024-05-10T04:03:29.193327+0000","last_undegraded":"2024-05-10T04:03:29.193327+0000","last_fullsized":"2024-05-10T04:03:29.193327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:16:44.833804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188283+0000","last_change":"2024-05-10T04:02:31.646002+0000","last_active":"2024-05-10T04:03:29.188283+0000","last_peered":"2024-05-10T04:03:29.188283+0000","last_clean":"2024-05-10T04:03:29.188283+0000","last_became_active":"2024-05-10T04:02:31.645673+0000","last_became_peered":"2024-05-10T04:02:31.645673+0000","last_unstale":"2024-05-10T04:03:29.188283+0000","last_undegraded":"2024-05-10T04:03:29.188283+0000","last_fullsized":"2024-05-10T04:03:29.188283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:26:26.814523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188124+0000","last_change":"2024-05-10T04:02:31.630606+0000","last_active":"2024-05-10T04:03:29.188124+0000","last_peered":"2024-05-10T04:03:29.188124+0000","last_clean":"2024-05-10T04:03:29.188124+0000","last_became_active":"2024-05-10T04:02:31.630313+0000","last_became_peered":"2024-05-10T04:02:31.630313+0000","last_unstale":"2024-05-10T04:03:29.188124+0000","last_undegraded":"2024-05-10T04:03:29.188124+0000","last_fullsized":"2024-05-10T04:03:29.188124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:39.876016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196756+0000","last_change":"2024-05-10T04:02:31.636239+0000","last_active":"2024-05-10T04:03:29.196756+0000","last_peered":"2024-05-10T04:03:29.196756+0000","last_clean":"2024-05-10T04:03:29.196756+0000","last_became_active":"2024-05-10T04:02:31.635377+0000","last_became_peered":"2024-05-10T04:02:31.635377+0000","last_unstale":"2024-05-10T04:03:29.196756+0000","last_undegraded":"2024-05-10T04:03:29.196756+0000","last_fullsized":"2024-05-10T04:03:29.196756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:02.612285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192883+0000","last_change":"2024-05-10T04:02:31.643231+0000","last_active":"2024-05-10T04:03:29.192883+0000","last_peered":"2024-05-10T04:03:29.192883+0000","last_clean":"2024-05-10T04:03:29.192883+0000","last_became_active":"2024-05-10T04:02:31.642797+0000","last_became_peered":"2024-05-10T04:02:31.642797+0000","last_unstale":"2024-05-10T04:03:29.192883+0000","last_undegraded":"2024-05-10T04:03:29.192883+0000","last_fullsized":"2024-05-10T04:03:29.192883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:01:57.365339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196100+0000","last_change":"2024-05-10T04:02:31.633928+0000","last_active":"2024-05-10T04:03:29.196100+0000","last_peered":"2024-05-10T04:03:29.196100+0000","last_clean":"2024-05-10T04:03:29.196100+0000","last_became_active":"2024-05-10T04:02:31.633423+0000","last_became_peered":"2024-05-10T04:02:31.633423+0000","last_unstale":"2024-05-10T04:03:29.196100+0000","last_undegraded":"2024-05-10T04:03:29.196100+0000","last_fullsized":"2024-05-10T04:03:29.196100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:07.011354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.189591+0000","last_change":"2024-05-10T04:02:31.627151+0000","last_active":"2024-05-10T04:03:29.189591+0000","last_peered":"2024-05-10T04:03:29.189591+0000","last_clean":"2024-05-10T04:03:29.189591+0000","last_became_active":"2024-05-10T04:02:31.626780+0000","last_became_peered":"2024-05-10T04:02:31.626780+0000","last_unstale":"2024-05-10T04:03:29.189591+0000","last_undegraded":"2024-05-10T04:03:29.189591+0000","last_fullsized":"2024-05-10T04:03:29.189591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:32.070530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199643+0000","last_change":"2024-05-10T04:02:31.640669+0000","last_active":"2024-05-10T04:03:29.199643+0000","last_peered":"2024-05-10T04:03:29.199643+0000","last_clean":"2024-05-10T04:03:29.199643+0000","last_became_active":"2024-05-10T04:02:31.640410+0000","last_became_peered":"2024-05-10T04:02:31.640410+0000","last_unstale":"2024-05-10T04:03:29.199643+0000","last_undegraded":"2024-05-10T04:03:29.199643+0000","last_fullsized":"2024-05-10T04:03:29.199643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:02.235391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193090+0000","last_change":"2024-05-10T04:02:31.633624+0000","last_active":"2024-05-10T04:03:29.193090+0000","last_peered":"2024-05-10T04:03:29.193090+0000","last_clean":"2024-05-10T04:03:29.193090+0000","last_became_active":"2024-05-10T04:02:31.633241+0000","last_became_peered":"2024-05-10T04:02:31.633241+0000","last_unstale":"2024-05-10T04:03:29.193090+0000","last_undegraded":"2024-05-10T04:03:29.193090+0000","last_fullsized":"2024-05-10T04:03:29.193090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:58.750912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198385+0000","last_change":"2024-05-10T04:02:31.628632+0000","last_active":"2024-05-10T04:03:29.198385+0000","last_peered":"2024-05-10T04:03:29.198385+0000","last_clean":"2024-05-10T04:03:29.198385+0000","last_became_active":"2024-05-10T04:02:31.628349+0000","last_became_peered":"2024-05-10T04:02:31.628349+0000","last_unstale":"2024-05-10T04:03:29.198385+0000","last_undegraded":"2024-05-10T04:03:29.198385+0000","last_fullsized":"2024-05-10T04:03:29.198385+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:32.575168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200128+0000","last_change":"2024-05-10T04:02:31.631328+0000","last_active":"2024-05-10T04:03:29.200128+0000","last_peered":"2024-05-10T04:03:29.200128+0000","last_clean":"2024-05-10T04:03:29.200128+0000","last_became_active":"2024-05-10T04:02:31.630957+0000","last_became_peered":"2024-05-10T04:02:31.630957+0000","last_unstale":"2024-05-10T04:03:29.200128+0000","last_undegraded":"2024-05-10T04:03:29.200128+0000","last_fullsized":"2024-05-10T04:03:29.200128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:40.400911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191789+0000","last_change":"2024-05-10T04:02:31.617972+0000","last_active":"2024-05-10T04:03:29.191789+0000","last_peered":"2024-05-10T04:03:29.191789+0000","last_clean":"2024-05-10T04:03:29.191789+0000","last_became_active":"2024-05-10T04:02:31.617663+0000","last_became_peered":"2024-05-10T04:02:31.617663+0000","last_unstale":"2024-05-10T04:03:29.191789+0000","last_undegraded":"2024-05-10T04:03:29.191789+0000","last_fullsized":"2024-05-10T04:03:29.191789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:50.121955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195144+0000","last_change":"2024-05-10T04:02:31.644256+0000","last_active":"2024-05-10T04:03:29.195144+0000","last_peered":"2024-05-10T04:03:29.195144+0000","last_clean":"2024-05-10T04:03:29.195144+0000","last_became_active":"2024-05-10T04:02:31.643819+0000","last_became_peered":"2024-05-10T04:02:31.643819+0000","last_unstale":"2024-05-10T04:03:29.195144+0000","last_undegraded":"2024-05-10T04:03:29.195144+0000","last_fullsized":"2024-05-10T04:03:29.195144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:54.932547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279970,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66659,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":436,"num_read_kb":289,"num_write":158,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":204291,"data_compressed":4764,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514525,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":950272,"data_stored":480641,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90900000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73299999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.59199999999999997}]},{"osd":1,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.622,"15min":0.626},"min":{"1min":0.48599999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.42399999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.625,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86199999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63,"15min":0.623},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72099999999999997,"15min":0.71499999999999997},"min":{"1min":0.61199999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.83499999999999996,"15min":0.82599999999999996},"min":{"1min":0.68300000000000005,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1379999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77000000000000002,"15min":0.76800000000000002},"min":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.86599999999999999}]},{"osd":8,"last update":"Fri May 10 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.82199999999999995}]},{"osd":9,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83899999999999997,"15min":0.84099999999999997},"min":{"1min":0.69099999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0469999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.90400000000000003,"15min":0.89900000000000002},"min":{"1min":0.76900000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1599999999999999,"5min":1.282,"15min":1.282},"last":0.92300000000000004}]},{"osd":10,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83099999999999996,"15min":0.82899999999999996},"min":{"1min":0.67600000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0309999999999999,"5min":1.319,"15min":1.319},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.84499999999999997,"15min":0.84599999999999997},"min":{"1min":0.68999999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0880000000000001,"5min":1.304,"15min":1.304},"last":0.73999999999999999}]}]},{"osd":10,"up_from":68,"seq":292057776161,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1089536,"data_stored":1134428,"data_compressed":54778,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.63600000000000001}]},{"osd":1,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.32200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.47799999999999998,"15min":0.48099999999999998},"min":{"1min":0.29899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67300000000000004,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.69899999999999995}]},{"osd":3,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.74099999999999999,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.40200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68799999999999994,"5min":0.872,"15min":0.872},"last":0.80000000000000004}]},{"osd":4,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.75700000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.42799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88800000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95599999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91400000000000003,"5min":1.175,"15min":1.175},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70099999999999996,"15min":0.70799999999999996},"min":{"1min":0.57699999999999996,"5min":0.504,"15min":0.504},"max":{"1min":0.81499999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.86899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":8,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90000000000000002,"5min":1.131,"15min":1.131},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.92900000000000005,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.85399999999999998}]},{"osd":9,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.76100000000000001},"min":{"1min":0.58399999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.88100000000000001,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.53100000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.79900000000000004,"5min":1.151,"15min":1.151},"last":0.81899999999999995}]},{"osd":11,"last update":"Fri May 10 04:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.871,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.78700000000000003,"15min":0.79400000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90900000000000003,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.96899999999999997}]}]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":966656,"data_stored":480877,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.435,"15min":0.443},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.46500000000000002,"15min":0.47599999999999998},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.89100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.38800000000000001}]},{"osd":1,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80700000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.61499999999999999,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71399999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.755},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.63300000000000001}]},{"osd":3,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.34799999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.52100000000000002,"15min":0.54300000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.58799999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48799999999999999}]},{"osd":4,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.67100000000000004,"15min":0.68400000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.751,"5min":1.149,"15min":1.149},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.1699999999999999}]},{"osd":7,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.74299999999999999,"15min":0.76300000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2470000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.752,"15min":0.78000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88100000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.65900000000000003}]},{"osd":8,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.53000000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.81100000000000005}]},{"osd":10,"last update":"Fri May 10 04:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93999999999999995,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.753,"15min":0.76800000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97299999999999998,"5min":1.306,"15min":1.306},"last":0.69499999999999995}]},{"osd":11,"last update":"Fri May 10 04:03:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79600000000000004}]}]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":966656,"data_stored":480797,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":0.83499999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54300000000000004,"15min":0.54800000000000004},"min":{"1min":0.40699999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.65000000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90000000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82099999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48399999999999999,"15min":0.47299999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.47599999999999998},"min":{"1min":0.32700000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.495},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.71499999999999997}]},{"osd":4,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.66200000000000003,"15min":0.64600000000000002},"min":{"1min":0.47199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96699999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78000000000000003}]},{"osd":5,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.41299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.435},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.68200000000000005}]},{"osd":6,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.627},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90000000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.49199999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.55300000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.72999999999999998}]},{"osd":9,"last update":"Fri May 10 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.69899999999999995,"15min":0.68700000000000006},"min":{"1min":0.58199999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.76500000000000001,"15min":0.747},"min":{"1min":0.63600000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.81000000000000005}]},{"osd":10,"last update":"Fri May 10 04:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.71999999999999997,"15min":0.69499999999999995},"min":{"1min":0.69999999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.747},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.77400000000000002,"15min":0.74399999999999999},"min":{"1min":0.67400000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.83299999999999996}]},{"osd":11,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.878},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.85599999999999998}]}]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1015808,"data_stored":1077319,"data_compressed":53190,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.623,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.45700000000000002},"min":{"1min":0.36599999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.626,"5min":0.752,"15min":0.752},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85499999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.56399999999999995,"5min":0.376,"15min":0.376},"max":{"1min":0.83499999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66900000000000004}]},{"osd":3,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78900000000000003,"5min":1.143,"15min":1.143},"last":0.628},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76000000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.74299999999999999}]},{"osd":4,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.42599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77300000000000002,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.36699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73899999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.54300000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88600000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.67900000000000005},"min":{"1min":0.61399999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97299999999999998,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69499999999999995,"15min":0.69399999999999995},"min":{"1min":0.51100000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.90600000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.69199999999999995,"15min":0.70299999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85099999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.68500000000000005}]},{"osd":8,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.204,"15min":1.286},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88900000000000001,"5min":15.521000000000001,"15min":15.521000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":1.2829999999999999,"15min":1.3660000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0049999999999999,"5min":16.306999999999999,"15min":16.306999999999999},"last":0.89300000000000002}]},{"osd":9,"last update":"Fri May 10 04:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0509999999999999,"5min":1.167,"15min":1.167},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.753,"15min":0.751},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.83099999999999996}]},{"osd":10,"last update":"Fri May 10 04:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.622,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97099999999999997,"5min":1.232,"15min":1.232},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.66300000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93899999999999995,"5min":1.111,"15min":1.111},"last":0.874}]},{"osd":11,"last update":"Fri May 10 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.94299999999999995}]}]},{"osd":6,"up_from":44,"seq":188978561070,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":946176,"data_stored":476053,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.40200000000000002,"15min":0.39100000000000001},"min":{"1min":0.311,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.61599999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.371},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.438,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.5739999999999998,"5min":3.5739999999999998,"15min":3.5739999999999998},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.57299999999999995},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.84499999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.48299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.49299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.002,"5min":1.087,"15min":1.087},"last":0.443}]},{"osd":3,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48499999999999999,"15min":0.45600000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.50700000000000001,"15min":0.47699999999999998},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.1819999999999999,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":0.53300000000000003}]},{"osd":4,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89400000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94399999999999995,"5min":1.018,"15min":1.018},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.52200000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98199999999999998,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri May 10 04:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63500000000000001,"15min":0.63200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86899999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.629}]},{"osd":8,"last update":"Fri May 10 04:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.55000000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.503},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.57899999999999996}]},{"osd":10,"last update":"Fri May 10 04:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.75,"15min":0.76300000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.70699999999999996}]},{"osd":11,"last update":"Fri May 10 04:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1,"5min":1,"15min":1},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.73299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1015808,"data_stored":542322,"data_compressed":19169,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80700000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56100000000000005,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72699999999999998}]},{"osd":2,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.58199999999999996,"15min":0.53700000000000003},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.56299999999999994,"15min":0.52300000000000002},"min":{"1min":0.41599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri May 10 04:03:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56999999999999995,"15min":0.52000000000000002},"min":{"1min":0.46800000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81100000000000005,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57699999999999996,"15min":0.54100000000000004},"min":{"1min":0.442,"5min":0.373,"15min":0.373},"max":{"1min":0.88400000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri May 10 04:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.496,"15min":0.46200000000000002},"min":{"1min":0.36699999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.371},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.59099999999999997,"15min":0.53100000000000003},"min":{"1min":0.47099999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri May 10 04:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.56499999999999995,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri May 10 04:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.47199999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94599999999999995,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.621},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.66900000000000004,"15min":0.627},"min":{"1min":0.45500000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.069,"5min":1.119,"15min":1.119},"last":0.64300000000000002}]},{"osd":7,"last update":"Fri May 10 04:03:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.61499999999999999,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92900000000000005,"5min":1.075,"15min":1.075},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.68100000000000005,"15min":0.63500000000000001},"min":{"1min":0.47399999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.115,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.44400000000000001}]},{"osd":8,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69999999999999996,"15min":0.68799999999999994},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.55100000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83199999999999996}]},{"osd":9,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.81999999999999995}]},{"osd":10,"last update":"Fri May 10 04:03:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.80500000000000005,"15min":0.80100000000000005},"min":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0329999999999999,"5min":1.046,"15min":1.046},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.83899999999999997,"15min":0.82899999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0940000000000001,"5min":1.143,"15min":1.143},"last":0.755}]},{"osd":11,"last update":"Fri May 10 04:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.80000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":477441,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68100000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.71999999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.71999999999999997}]},{"osd":2,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.62,"15min":0.55000000000000004},"min":{"1min":0.443,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.9009999999999998,"5min":4.9009999999999998,"15min":4.9009999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82499999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42499999999999999}]},{"osd":3,"last update":"Fri May 10 04:03:33 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.44},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.38300000000000001}]},{"osd":4,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.61599999999999999,"15min":0.56599999999999995},"min":{"1min":0.42499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.8369999999999997,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.47299999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.76800000000000002}]},{"osd":5,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79700000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.64800000000000002,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri May 10 04:03:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.59899999999999998,"15min":0.57999999999999996},"min":{"1min":0.32400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":4.7439999999999998,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri May 10 04:03:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.78200000000000003}]},{"osd":8,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.67000000000000004,"15min":0.67800000000000005},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85099999999999998,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86299999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56599999999999995}]},{"osd":9,"last update":"Fri May 10 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.72099999999999997,"15min":0.69299999999999995},"min":{"1min":0.38500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.7999999999999998,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.47499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.64700000000000002}]},{"osd":10,"last update":"Fri May 10 04:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68300000000000005,"15min":0.67500000000000004},"min":{"1min":0.53500000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.77800000000000002,"15min":0.746},"min":{"1min":0.57999999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":4.96,"5min":4.96,"15min":4.96},"last":0.747}]},{"osd":11,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.499,"5min":0.495,"15min":0.495},"max":{"1min":0.876,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.63700000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":1024000,"data_stored":539591,"data_compressed":19169,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.54200000000000004,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56799999999999995,"15min":0.53500000000000003},"min":{"1min":0.41799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.629},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.59799999999999998,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.78500000000000003}]},{"osd":3,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.57899999999999996},"min":{"1min":0.48299999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri May 10 04:03:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri May 10 04:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.51700000000000002,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79000000000000004,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.40799999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri May 10 04:03:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.48899999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.496},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.66800000000000004,"15min":0.63900000000000001},"min":{"1min":0.52600000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri May 10 04:03:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.89300000000000002}]},{"osd":8,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.74299999999999999,"15min":0.72299999999999998},"min":{"1min":0.66800000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0189999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.87},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.59199999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.91900000000000004,"5min":1.016,"15min":1.016},"last":0.70399999999999996}]},{"osd":10,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.753,"15min":0.73699999999999999},"min":{"1min":0.66000000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.79200000000000004,"15min":0.77800000000000002},"min":{"1min":0.73399999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.80400000000000005}]},{"osd":11,"last update":"Fri May 10 04:03:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.85299999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":476247,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36199999999999999,"15min":0.35599999999999998},"min":{"1min":0.29199999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.503,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.377,"15min":0.36399999999999999},"min":{"1min":0.32600000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.41899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.747,"15min":0.747},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.53600000000000003,"15min":0.50800000000000001},"min":{"1min":0.47599999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77300000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80300000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri May 10 04:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.48899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.69899999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.52500000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78700000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri May 10 04:03:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80100000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.59199999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78800000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.502}]},{"osd":6,"last update":"Fri May 10 04:03:33 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.68000000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.53900000000000003,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.41299999999999998}]},{"osd":7,"last update":"Fri May 10 04:03:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.435,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73299999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.52700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.71999999999999997}]},{"osd":8,"last update":"Fri May 10 04:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.439}]},{"osd":9,"last update":"Fri May 10 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69599999999999995,"15min":0.71199999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89900000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83999999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.443}]},{"osd":10,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70299999999999996,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83799999999999997,"5min":1.228,"15min":1.228},"last":0.625},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.499,"5min":0.433,"15min":0.433},"max":{"1min":0.82099999999999995,"5min":1.25,"15min":1.25},"last":0.66100000000000003}]},{"osd":11,"last update":"Fri May 10 04:03:33 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.69999999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":481474,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.33900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50900000000000001,"15min":0.502},"min":{"1min":0.371,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.79000000000000004}]},{"osd":1,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41699999999999998,"15min":0.39300000000000002},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53900000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.39100000000000001},"min":{"1min":0.35099999999999998,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.53500000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51300000000000001}]},{"osd":2,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.64400000000000002,"15min":0.57699999999999996},"min":{"1min":0.59399999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.91000000000000003,"5min":1.046,"15min":1.046},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.73999999999999999}]},{"osd":3,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79800000000000004,"5min":0.873,"15min":0.873},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.59299999999999997,"15min":0.54700000000000004},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.6050000000000004,"5min":4.6050000000000004,"15min":4.6050000000000004},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri May 10 04:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.46400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.92300000000000004,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81699999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.92200000000000004}]},{"osd":6,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64000000000000001,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.89700000000000002,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.89200000000000002}]},{"osd":7,"last update":"Fri May 10 04:03:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72999999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.44900000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":1.01,"15min":1.01},"last":0.97199999999999998}]},{"osd":8,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68100000000000005,"15min":0.68500000000000005},"min":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.79900000000000004,"5min":0.879,"15min":0.879},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.56899999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.95599999999999996}]},{"osd":9,"last update":"Fri May 10 04:03:26 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.80000000000000004,"15min":0.76700000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.6449999999999996,"5min":5.6449999999999996,"15min":5.6449999999999996},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.82899999999999996},"min":{"1min":0.54500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":5.6100000000000003,"5min":5.6100000000000003,"15min":5.6100000000000003},"last":1.006}]},{"osd":10,"last update":"Fri May 10 04:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.52300000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.93300000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.878},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83899999999999997,"15min":0.84499999999999997},"min":{"1min":0.56799999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.997,"5min":1.161,"15min":1.161},"last":1.038}]},{"osd":11,"last update":"Fri May 10 04:03:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":1.0660000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790002,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":1019904,"data_stored":1071917,"data_compressed":53190,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52400000000000002,"15min":0.498},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77800000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.433,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74299999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57499999999999996,"15min":0.55100000000000005},"min":{"1min":0.45400000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.505},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.47999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.40400000000000003},"min":{"1min":0.35799999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.44,"15min":0.42199999999999999},"min":{"1min":0.37,"5min":0.249,"15min":0.249},"max":{"1min":0.66100000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80500000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.74299999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61099999999999999,"15min":0.59199999999999997},"min":{"1min":0.48199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.874,"15min":0.874},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66300000000000003,"15min":0.65500000000000003},"min":{"1min":0.51700000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85999999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78000000000000003}]},{"osd":7,"last update":"Fri May 10 04:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.70799999999999996,"15min":0.68500000000000005},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.68899999999999995,"15min":0.65600000000000003},"min":{"1min":0.56100000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.81000000000000005}]},{"osd":8,"last update":"Fri May 10 04:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91900000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.46000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.74199999999999999}]},{"osd":9,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.55000000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.88400000000000001}]},{"osd":10,"last update":"Fri May 10 04:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77100000000000002,"15min":0.76900000000000002},"min":{"1min":0.56299999999999994,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80300000000000005,"15min":0.80200000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.85099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":66359,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63628,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64282,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:04:09.000 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T04:04:09.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.162+0000 7fdd101b9700 1 -- 172.21.15.150:0/3340602167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd081026b0 msgr2=0x7fdd08104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:09.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.162+0000 7fdd101b9700 1 --2- 172.21.15.150:0/3340602167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd081026b0 0x7fdd08104ad0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009990 tx=0x7fdcf8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.162+0000 7fdd101b9700 1 -- 172.21.15.150:0/3340602167 shutdown_connections 2024-05-10T04:04:09.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.162+0000 7fdd101b9700 1 --2- 172.21.15.150:0/3340602167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08107970 0x7fdd08109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.162+0000 7fdd101b9700 1 --2- 172.21.15.150:0/3340602167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd081026b0 0x7fdd08104ad0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.162+0000 7fdd101b9700 1 --2- 172.21.15.150:0/3340602167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd08105010 0x7fdd08107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.162+0000 7fdd101b9700 1 -- 172.21.15.150:0/3340602167 >> 172.21.15.150:0/3340602167 conn(0x7fdd080fc0d0 msgr2=0x7fdd080fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:09.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.162+0000 7fdd101b9700 1 -- 172.21.15.150:0/3340602167 shutdown_connections 2024-05-10T04:04:09.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.162+0000 7fdd101b9700 1 -- 172.21.15.150:0/3340602167 wait complete. 2024-05-10T04:04:09.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd101b9700 1 Processor -- start 2024-05-10T04:04:09.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd101b9700 1 -- start start 2024-05-10T04:04:09.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd101b9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd081026b0 0x7fdd08197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:09.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd101b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08105010 0x7fdd08198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:09.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd101b9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd08107970 0x7fdd0819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:09.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd101b9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0819fad0 con 0x7fdd081026b0 2024-05-10T04:04:09.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd101b9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0819fc40 con 0x7fdd08107970 2024-05-10T04:04:09.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd101b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0819fe00 con 0x7fdd08105010 2024-05-10T04:04:09.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd0df55700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd081026b0 0x7fdd08197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:09.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd0df55700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd081026b0 0x7fdd08197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35586/0 (socket says 172.21.15.150:35586) 2024-05-10T04:04:09.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd0df55700 1 -- 172.21.15.150:0/2979378796 learned_addr learned my addr 172.21.15.150:0/2979378796 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:09.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd0d754700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08105010 0x7fdd08198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd0e756700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd08107970 0x7fdd0819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd0df55700 1 -- 172.21.15.150:0/2979378796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08105010 msgr2=0x7fdd08198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd0df55700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08105010 0x7fdd08198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd0df55700 1 -- 172.21.15.150:0/2979378796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd08107970 msgr2=0x7fdd0819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd0df55700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd08107970 0x7fdd0819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.163+0000 7fdd0df55700 1 -- 172.21.15.150:0/2979378796 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf80095f0 con 0x7fdd081026b0 2024-05-10T04:04:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.164+0000 7fdd0df55700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd081026b0 0x7fdd08197ff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8006010 tx=0x7fdcf800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.164+0000 7fdcfeffd700 1 -- 172.21.15.150:0/2979378796 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf801d070 con 0x7fdd081026b0 2024-05-10T04:04:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.164+0000 7fdcfeffd700 1 -- 172.21.15.150:0/2979378796 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcf8022470 con 0x7fdd081026b0 2024-05-10T04:04:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.164+0000 7fdd101b9700 1 -- 172.21.15.150:0/2979378796 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd081a0080 con 0x7fdd081026b0 2024-05-10T04:04:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.164+0000 7fdcfeffd700 1 -- 172.21.15.150:0/2979378796 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8017650 con 0x7fdd081026b0 2024-05-10T04:04:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.164+0000 7fdd101b9700 1 -- 172.21.15.150:0/2979378796 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd081a0500 con 0x7fdd081026b0 2024-05-10T04:04:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.165+0000 7fdd101b9700 1 -- 172.21.15.150:0/2979378796 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd080fdc80 con 0x7fdd081026b0 2024-05-10T04:04:09.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.166+0000 7fdcfeffd700 1 -- 172.21.15.150:0/2979378796 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fdcf80225e0 con 0x7fdd081026b0 2024-05-10T04:04:09.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.167+0000 7fdcfeffd700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdcf40a1230 0x7fdcf40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:09.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.167+0000 7fdd0d754700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdcf40a1230 0x7fdcf40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:09.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.167+0000 7fdcfeffd700 1 -- 172.21.15.150:0/2979378796 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdcf80ce2b0 con 0x7fdd081026b0 2024-05-10T04:04:09.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.167+0000 7fdd0d754700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdcf40a1230 0x7fdcf40a3740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdd08199090 tx=0x7fdd0400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:09.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.170+0000 7fdcfeffd700 1 -- 172.21.15.150:0/2979378796 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdcf8097790 con 0x7fdd081026b0 2024-05-10T04:04:09.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.367+0000 7fdd101b9700 1 -- 172.21.15.150:0/2979378796 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd08061650 con 0x7fdcf40a1230 2024-05-10T04:04:09.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.379+0000 7fdcfeffd700 1 -- 172.21.15.150:0/2979378796 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427782 (secure 0 0 0) 0x7fdd08061650 con 0x7fdcf40a1230 2024-05-10T04:04:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:04:09.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.383+0000 7fdd101b9700 1 -- 172.21.15.150:0/2979378796 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdcf40a1230 msgr2=0x7fdcf40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:09.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.383+0000 7fdd101b9700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdcf40a1230 0x7fdcf40a3740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdd08199090 tx=0x7fdd0400b410 comp rx=0 tx=0).stop 2024-05-10T04:04:09.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.383+0000 7fdd101b9700 1 -- 172.21.15.150:0/2979378796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd081026b0 msgr2=0x7fdd08197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:09.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.383+0000 7fdd101b9700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd081026b0 0x7fdd08197ff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8006010 tx=0x7fdcf800f690 comp rx=0 tx=0).stop 2024-05-10T04:04:09.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.383+0000 7fdd101b9700 1 -- 172.21.15.150:0/2979378796 shutdown_connections 2024-05-10T04:04:09.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.383+0000 7fdd101b9700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdcf40a1230 0x7fdcf40a3740 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.383+0000 7fdd101b9700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd08107970 0x7fdd0819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.383+0000 7fdd101b9700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd081026b0 0x7fdd08197ff0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.383+0000 7fdd101b9700 1 --2- 172.21.15.150:0/2979378796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08105010 0x7fdd08198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.383+0000 7fdd101b9700 1 -- 172.21.15.150:0/2979378796 >> 172.21.15.150:0/2979378796 conn(0x7fdd080fc0d0 msgr2=0x7fdd08104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:09.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.383+0000 7fdd101b9700 1 -- 172.21.15.150:0/2979378796 shutdown_connections 2024-05-10T04:04:09.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.383+0000 7fdd101b9700 1 -- 172.21.15.150:0/2979378796 wait complete. 2024-05-10T04:04:09.393 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:04:09.397 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:09 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4198426637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:04:09.397 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:09 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1207869510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:04:09.398 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-05-10T04:04:09.106963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657261,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":567,"num_read_kb":398,"num_write":341,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180965,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372720,"kb_used_data":11620,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569104,"statfs":{"total":1151940427776,"available":1151558762496,"internally_reserved":0,"allocated":11898880,"data_stored":7725941,"data_compressed":322563,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":628,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":32,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003946"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195663+0000","last_change":"2024-05-10T04:02:31.642089+0000","last_active":"2024-05-10T04:03:29.195663+0000","last_peered":"2024-05-10T04:03:29.195663+0000","last_clean":"2024-05-10T04:03:29.195663+0000","last_became_active":"2024-05-10T04:02:31.641080+0000","last_became_peered":"2024-05-10T04:02:31.641080+0000","last_unstale":"2024-05-10T04:03:29.195663+0000","last_undegraded":"2024-05-10T04:03:29.195663+0000","last_fullsized":"2024-05-10T04:03:29.195663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:29.693139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197452+0000","last_change":"2024-05-10T04:02:31.633056+0000","last_active":"2024-05-10T04:03:29.197452+0000","last_peered":"2024-05-10T04:03:29.197452+0000","last_clean":"2024-05-10T04:03:29.197452+0000","last_became_active":"2024-05-10T04:02:31.632725+0000","last_became_peered":"2024-05-10T04:02:31.632725+0000","last_unstale":"2024-05-10T04:03:29.197452+0000","last_undegraded":"2024-05-10T04:03:29.197452+0000","last_fullsized":"2024-05-10T04:03:29.197452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:08:40.365433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197250+0000","last_change":"2024-05-10T04:02:31.625219+0000","last_active":"2024-05-10T04:03:29.197250+0000","last_peered":"2024-05-10T04:03:29.197250+0000","last_clean":"2024-05-10T04:03:29.197250+0000","last_became_active":"2024-05-10T04:02:31.624889+0000","last_became_peered":"2024-05-10T04:02:31.624889+0000","last_unstale":"2024-05-10T04:03:29.197250+0000","last_undegraded":"2024-05-10T04:03:29.197250+0000","last_fullsized":"2024-05-10T04:03:29.197250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:10:59.081031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.199344+0000","last_change":"2024-05-10T04:03:30.198971+0000","last_active":"2024-05-10T04:03:30.199344+0000","last_peered":"2024-05-10T04:03:30.199344+0000","last_clean":"2024-05-10T04:03:30.199344+0000","last_became_active":"2024-05-10T04:03:30.198664+0000","last_became_peered":"2024-05-10T04:03:30.198664+0000","last_unstale":"2024-05-10T04:03:30.199344+0000","last_undegraded":"2024-05-10T04:03:30.199344+0000","last_fullsized":"2024-05-10T04:03:30.199344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:22.655875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195511+0000","last_change":"2024-05-10T04:02:31.630688+0000","last_active":"2024-05-10T04:03:29.195511+0000","last_peered":"2024-05-10T04:03:29.195511+0000","last_clean":"2024-05-10T04:03:29.195511+0000","last_became_active":"2024-05-10T04:02:31.629252+0000","last_became_peered":"2024-05-10T04:02:31.629252+0000","last_unstale":"2024-05-10T04:03:29.195511+0000","last_undegraded":"2024-05-10T04:03:29.195511+0000","last_fullsized":"2024-05-10T04:03:29.195511+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:37.647516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198848+0000","last_change":"2024-05-10T04:02:31.642586+0000","last_active":"2024-05-10T04:03:29.198848+0000","last_peered":"2024-05-10T04:03:29.198848+0000","last_clean":"2024-05-10T04:03:29.198848+0000","last_became_active":"2024-05-10T04:02:31.642244+0000","last_became_peered":"2024-05-10T04:02:31.642244+0000","last_unstale":"2024-05-10T04:03:29.198848+0000","last_undegraded":"2024-05-10T04:03:29.198848+0000","last_fullsized":"2024-05-10T04:03:29.198848+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:28.020151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191697+0000","last_change":"2024-05-10T04:02:31.636706+0000","last_active":"2024-05-10T04:03:29.191697+0000","last_peered":"2024-05-10T04:03:29.191697+0000","last_clean":"2024-05-10T04:03:29.191697+0000","last_became_active":"2024-05-10T04:02:31.636303+0000","last_became_peered":"2024-05-10T04:02:31.636303+0000","last_unstale":"2024-05-10T04:03:29.191697+0000","last_undegraded":"2024-05-10T04:03:29.191697+0000","last_fullsized":"2024-05-10T04:03:29.191697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:16.941400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198297+0000","last_change":"2024-05-10T04:02:31.640032+0000","last_active":"2024-05-10T04:03:29.198297+0000","last_peered":"2024-05-10T04:03:29.198297+0000","last_clean":"2024-05-10T04:03:29.198297+0000","last_became_active":"2024-05-10T04:02:31.639733+0000","last_became_peered":"2024-05-10T04:02:31.639733+0000","last_unstale":"2024-05-10T04:03:29.198297+0000","last_undegraded":"2024-05-10T04:03:29.198297+0000","last_fullsized":"2024-05-10T04:03:29.198297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:02:36.963414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200412+0000","last_change":"2024-05-10T04:02:31.638940+0000","last_active":"2024-05-10T04:03:29.200412+0000","last_peered":"2024-05-10T04:03:29.200412+0000","last_clean":"2024-05-10T04:03:29.200412+0000","last_became_active":"2024-05-10T04:02:31.638617+0000","last_became_peered":"2024-05-10T04:02:31.638617+0000","last_unstale":"2024-05-10T04:03:29.200412+0000","last_undegraded":"2024-05-10T04:03:29.200412+0000","last_fullsized":"2024-05-10T04:03:29.200412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:10:36.348117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199245+0000","last_change":"2024-05-10T04:02:31.635702+0000","last_active":"2024-05-10T04:03:29.199245+0000","last_peered":"2024-05-10T04:03:29.199245+0000","last_clean":"2024-05-10T04:03:29.199245+0000","last_became_active":"2024-05-10T04:02:31.635330+0000","last_became_peered":"2024-05-10T04:02:31.635330+0000","last_unstale":"2024-05-10T04:03:29.199245+0000","last_undegraded":"2024-05-10T04:03:29.199245+0000","last_fullsized":"2024-05-10T04:03:29.199245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:21:15.160340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192388+0000","last_change":"2024-05-10T04:02:31.628252+0000","last_active":"2024-05-10T04:03:29.192388+0000","last_peered":"2024-05-10T04:03:29.192388+0000","last_clean":"2024-05-10T04:03:29.192388+0000","last_became_active":"2024-05-10T04:02:31.627394+0000","last_became_peered":"2024-05-10T04:02:31.627394+0000","last_unstale":"2024-05-10T04:03:29.192388+0000","last_undegraded":"2024-05-10T04:03:29.192388+0000","last_fullsized":"2024-05-10T04:03:29.192388+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:54.569786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196477+0000","last_change":"2024-05-10T04:02:31.641788+0000","last_active":"2024-05-10T04:03:29.196477+0000","last_peered":"2024-05-10T04:03:29.196477+0000","last_clean":"2024-05-10T04:03:29.196477+0000","last_became_active":"2024-05-10T04:02:31.641328+0000","last_became_peered":"2024-05-10T04:02:31.641328+0000","last_unstale":"2024-05-10T04:03:29.196477+0000","last_undegraded":"2024-05-10T04:03:29.196477+0000","last_fullsized":"2024-05-10T04:03:29.196477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:10:55.920245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191930+0000","last_change":"2024-05-10T04:02:31.631919+0000","last_active":"2024-05-10T04:03:29.191930+0000","last_peered":"2024-05-10T04:03:29.191930+0000","last_clean":"2024-05-10T04:03:29.191930+0000","last_became_active":"2024-05-10T04:02:31.631592+0000","last_became_peered":"2024-05-10T04:02:31.631592+0000","last_unstale":"2024-05-10T04:03:29.191930+0000","last_undegraded":"2024-05-10T04:03:29.191930+0000","last_fullsized":"2024-05-10T04:03:29.191930+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:59:03.792090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196091+0000","last_change":"2024-05-10T04:02:31.633288+0000","last_active":"2024-05-10T04:03:29.196091+0000","last_peered":"2024-05-10T04:03:29.196091+0000","last_clean":"2024-05-10T04:03:29.196091+0000","last_became_active":"2024-05-10T04:02:31.632582+0000","last_became_peered":"2024-05-10T04:02:31.632582+0000","last_unstale":"2024-05-10T04:03:29.196091+0000","last_undegraded":"2024-05-10T04:03:29.196091+0000","last_fullsized":"2024-05-10T04:03:29.196091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:27.991439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196261+0000","last_change":"2024-05-10T04:02:31.635833+0000","last_active":"2024-05-10T04:03:29.196261+0000","last_peered":"2024-05-10T04:03:29.196261+0000","last_clean":"2024-05-10T04:03:29.196261+0000","last_became_active":"2024-05-10T04:02:31.635520+0000","last_became_peered":"2024-05-10T04:02:31.635520+0000","last_unstale":"2024-05-10T04:03:29.196261+0000","last_undegraded":"2024-05-10T04:03:29.196261+0000","last_fullsized":"2024-05-10T04:03:29.196261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:40:06.312819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199745+0000","last_change":"2024-05-10T04:02:31.631734+0000","last_active":"2024-05-10T04:03:29.199745+0000","last_peered":"2024-05-10T04:03:29.199745+0000","last_clean":"2024-05-10T04:03:29.199745+0000","last_became_active":"2024-05-10T04:02:31.631375+0000","last_became_peered":"2024-05-10T04:02:31.631375+0000","last_unstale":"2024-05-10T04:03:29.199745+0000","last_undegraded":"2024-05-10T04:03:29.199745+0000","last_fullsized":"2024-05-10T04:03:29.199745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:25:13.815406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195179+0000","last_change":"2024-05-10T04:02:31.642292+0000","last_active":"2024-05-10T04:03:29.195179+0000","last_peered":"2024-05-10T04:03:29.195179+0000","last_clean":"2024-05-10T04:03:29.195179+0000","last_became_active":"2024-05-10T04:02:31.641570+0000","last_became_peered":"2024-05-10T04:02:31.641570+0000","last_unstale":"2024-05-10T04:03:29.195179+0000","last_undegraded":"2024-05-10T04:03:29.195179+0000","last_fullsized":"2024-05-10T04:03:29.195179+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:16:33.704809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195044+0000","last_change":"2024-05-10T04:02:31.625695+0000","last_active":"2024-05-10T04:03:29.195044+0000","last_peered":"2024-05-10T04:03:29.195044+0000","last_clean":"2024-05-10T04:03:29.195044+0000","last_became_active":"2024-05-10T04:02:31.625393+0000","last_became_peered":"2024-05-10T04:02:31.625393+0000","last_unstale":"2024-05-10T04:03:29.195044+0000","last_undegraded":"2024-05-10T04:03:29.195044+0000","last_fullsized":"2024-05-10T04:03:29.195044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:53.955921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188569+0000","last_change":"2024-05-10T04:02:31.629152+0000","last_active":"2024-05-10T04:03:29.188569+0000","last_peered":"2024-05-10T04:03:29.188569+0000","last_clean":"2024-05-10T04:03:29.188569+0000","last_became_active":"2024-05-10T04:02:31.628778+0000","last_became_peered":"2024-05-10T04:02:31.628778+0000","last_unstale":"2024-05-10T04:03:29.188569+0000","last_undegraded":"2024-05-10T04:03:29.188569+0000","last_fullsized":"2024-05-10T04:03:29.188569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:39.772500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188510+0000","last_change":"2024-05-10T04:02:35.650536+0000","last_active":"2024-05-10T04:03:29.188510+0000","last_peered":"2024-05-10T04:03:29.188510+0000","last_clean":"2024-05-10T04:03:29.188510+0000","last_became_active":"2024-05-10T04:02:35.650102+0000","last_became_peered":"2024-05-10T04:02:35.650102+0000","last_unstale":"2024-05-10T04:03:29.188510+0000","last_undegraded":"2024-05-10T04:03:29.188510+0000","last_fullsized":"2024-05-10T04:03:29.188510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:19:35.605844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192581+0000","last_change":"2024-05-10T04:02:31.186846+0000","last_active":"2024-05-10T04:03:29.192581+0000","last_peered":"2024-05-10T04:03:29.192581+0000","last_clean":"2024-05-10T04:03:29.192581+0000","last_became_active":"2024-05-10T04:02:31.186497+0000","last_became_peered":"2024-05-10T04:02:31.186497+0000","last_unstale":"2024-05-10T04:03:29.192581+0000","last_undegraded":"2024-05-10T04:03:29.192581+0000","last_fullsized":"2024-05-10T04:03:29.192581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:15:40.832455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.252299+0000","last_change":"2024-05-10T04:03:30.251863+0000","last_active":"2024-05-10T04:03:30.252299+0000","last_peered":"2024-05-10T04:03:30.252299+0000","last_clean":"2024-05-10T04:03:30.252299+0000","last_became_active":"2024-05-10T04:03:30.251466+0000","last_became_peered":"2024-05-10T04:03:30.251466+0000","last_unstale":"2024-05-10T04:03:30.252299+0000","last_undegraded":"2024-05-10T04:03:30.252299+0000","last_fullsized":"2024-05-10T04:03:30.252299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:56:12.810512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194320+0000","last_change":"2024-05-10T04:02:35.651622+0000","last_active":"2024-05-10T04:03:29.194320+0000","last_peered":"2024-05-10T04:03:29.194320+0000","last_clean":"2024-05-10T04:03:29.194320+0000","last_became_active":"2024-05-10T04:02:35.651148+0000","last_became_peered":"2024-05-10T04:02:35.651148+0000","last_unstale":"2024-05-10T04:03:29.194320+0000","last_undegraded":"2024-05-10T04:03:29.194320+0000","last_fullsized":"2024-05-10T04:03:29.194320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:40:09.019286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199659+0000","last_change":"2024-05-10T04:02:31.187785+0000","last_active":"2024-05-10T04:03:29.199659+0000","last_peered":"2024-05-10T04:03:29.199659+0000","last_clean":"2024-05-10T04:03:29.199659+0000","last_became_active":"2024-05-10T04:02:31.187499+0000","last_became_peered":"2024-05-10T04:02:31.187499+0000","last_unstale":"2024-05-10T04:03:29.199659+0000","last_undegraded":"2024-05-10T04:03:29.199659+0000","last_fullsized":"2024-05-10T04:03:29.199659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:05:29.015767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197457+0000","last_change":"2024-05-10T04:02:31.624433+0000","last_active":"2024-05-10T04:03:29.197457+0000","last_peered":"2024-05-10T04:03:29.197457+0000","last_clean":"2024-05-10T04:03:29.197457+0000","last_became_active":"2024-05-10T04:02:31.624115+0000","last_became_peered":"2024-05-10T04:02:31.624115+0000","last_unstale":"2024-05-10T04:03:29.197457+0000","last_undegraded":"2024-05-10T04:03:29.197457+0000","last_fullsized":"2024-05-10T04:03:29.197457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:54.584681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.252518+0000","last_change":"2024-05-10T04:03:30.252518+0000","last_active":"2024-05-10T04:03:30.252518+0000","last_peered":"2024-05-10T04:03:30.252518+0000","last_clean":"2024-05-10T04:03:30.252518+0000","last_became_active":"2024-05-10T04:03:30.251919+0000","last_became_peered":"2024-05-10T04:03:30.251919+0000","last_unstale":"2024-05-10T04:03:30.252518+0000","last_undegraded":"2024-05-10T04:03:30.252518+0000","last_fullsized":"2024-05-10T04:03:30.252518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:05:29.247977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.206678+0000","last_change":"2024-05-10T04:03:30.206145+0000","last_active":"2024-05-10T04:03:30.206678+0000","last_peered":"2024-05-10T04:03:30.206678+0000","last_clean":"2024-05-10T04:03:30.206678+0000","last_became_active":"2024-05-10T04:03:30.200138+0000","last_became_peered":"2024-05-10T04:03:30.200138+0000","last_unstale":"2024-05-10T04:03:30.206678+0000","last_undegraded":"2024-05-10T04:03:30.206678+0000","last_fullsized":"2024-05-10T04:03:30.206678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:49.885012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200208+0000","last_change":"2024-05-10T04:02:31.628124+0000","last_active":"2024-05-10T04:03:29.200208+0000","last_peered":"2024-05-10T04:03:29.200208+0000","last_clean":"2024-05-10T04:03:29.200208+0000","last_became_active":"2024-05-10T04:02:31.627789+0000","last_became_peered":"2024-05-10T04:02:31.627789+0000","last_unstale":"2024-05-10T04:03:29.200208+0000","last_undegraded":"2024-05-10T04:03:29.200208+0000","last_fullsized":"2024-05-10T04:03:29.200208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:40:11.240726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194237+0000","last_change":"2024-05-10T04:02:35.649536+0000","last_active":"2024-05-10T04:03:29.194237+0000","last_peered":"2024-05-10T04:03:29.194237+0000","last_clean":"2024-05-10T04:03:29.194237+0000","last_became_active":"2024-05-10T04:02:35.649130+0000","last_became_peered":"2024-05-10T04:02:35.649130+0000","last_unstale":"2024-05-10T04:03:29.194237+0000","last_undegraded":"2024-05-10T04:03:29.194237+0000","last_fullsized":"2024-05-10T04:03:29.194237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:03.870071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196512+0000","last_change":"2024-05-10T04:02:31.192627+0000","last_active":"2024-05-10T04:03:29.196512+0000","last_peered":"2024-05-10T04:03:29.196512+0000","last_clean":"2024-05-10T04:03:29.196512+0000","last_became_active":"2024-05-10T04:02:31.191433+0000","last_became_peered":"2024-05-10T04:02:31.191433+0000","last_unstale":"2024-05-10T04:03:29.196512+0000","last_undegraded":"2024-05-10T04:03:29.196512+0000","last_fullsized":"2024-05-10T04:03:29.196512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:29.235483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193100+0000","last_change":"2024-05-10T04:02:31.641192+0000","last_active":"2024-05-10T04:03:29.193100+0000","last_peered":"2024-05-10T04:03:29.193100+0000","last_clean":"2024-05-10T04:03:29.193100+0000","last_became_active":"2024-05-10T04:02:31.640540+0000","last_became_peered":"2024-05-10T04:02:31.640540+0000","last_unstale":"2024-05-10T04:03:29.193100+0000","last_undegraded":"2024-05-10T04:03:29.193100+0000","last_fullsized":"2024-05-10T04:03:29.193100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:36.734231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.200392+0000","last_change":"2024-05-10T04:03:30.200392+0000","last_active":"2024-05-10T04:03:30.200392+0000","last_peered":"2024-05-10T04:03:30.200392+0000","last_clean":"2024-05-10T04:03:30.200392+0000","last_became_active":"2024-05-10T04:03:30.199878+0000","last_became_peered":"2024-05-10T04:03:30.199878+0000","last_unstale":"2024-05-10T04:03:30.200392+0000","last_undegraded":"2024-05-10T04:03:30.200392+0000","last_fullsized":"2024-05-10T04:03:30.200392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:33:33.143992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:04.298481+0000","last_change":"2024-05-10T04:02:31.198037+0000","last_active":"2024-05-10T04:04:04.298481+0000","last_peered":"2024-05-10T04:04:04.298481+0000","last_clean":"2024-05-10T04:04:04.298481+0000","last_became_active":"2024-05-10T04:02:31.197679+0000","last_became_peered":"2024-05-10T04:02:31.197679+0000","last_unstale":"2024-05-10T04:04:04.298481+0000","last_undegraded":"2024-05-10T04:04:04.298481+0000","last_fullsized":"2024-05-10T04:04:04.298481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:19.059645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196289+0000","last_change":"2024-05-10T04:02:31.630020+0000","last_active":"2024-05-10T04:03:29.196289+0000","last_peered":"2024-05-10T04:03:29.196289+0000","last_clean":"2024-05-10T04:03:29.196289+0000","last_became_active":"2024-05-10T04:02:31.629738+0000","last_became_peered":"2024-05-10T04:02:31.629738+0000","last_unstale":"2024-05-10T04:03:29.196289+0000","last_undegraded":"2024-05-10T04:03:29.196289+0000","last_fullsized":"2024-05-10T04:03:29.196289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:58:42.317143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188383+0000","last_change":"2024-05-10T04:02:35.647242+0000","last_active":"2024-05-10T04:03:29.188383+0000","last_peered":"2024-05-10T04:03:29.188383+0000","last_clean":"2024-05-10T04:03:29.188383+0000","last_became_active":"2024-05-10T04:02:35.646774+0000","last_became_peered":"2024-05-10T04:02:35.646774+0000","last_unstale":"2024-05-10T04:03:29.188383+0000","last_undegraded":"2024-05-10T04:03:29.188383+0000","last_fullsized":"2024-05-10T04:03:29.188383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:20:05.602820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198723+0000","last_change":"2024-05-10T04:02:31.191981+0000","last_active":"2024-05-10T04:03:29.198723+0000","last_peered":"2024-05-10T04:03:29.198723+0000","last_clean":"2024-05-10T04:03:29.198723+0000","last_became_active":"2024-05-10T04:02:31.191632+0000","last_became_peered":"2024-05-10T04:02:31.191632+0000","last_unstale":"2024-05-10T04:03:29.198723+0000","last_undegraded":"2024-05-10T04:03:29.198723+0000","last_fullsized":"2024-05-10T04:03:29.198723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:44:52.744661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194359+0000","last_change":"2024-05-10T04:02:31.643240+0000","last_active":"2024-05-10T04:03:29.194359+0000","last_peered":"2024-05-10T04:03:29.194359+0000","last_clean":"2024-05-10T04:03:29.194359+0000","last_became_active":"2024-05-10T04:02:31.642891+0000","last_became_peered":"2024-05-10T04:02:31.642891+0000","last_unstale":"2024-05-10T04:03:29.194359+0000","last_undegraded":"2024-05-10T04:03:29.194359+0000","last_fullsized":"2024-05-10T04:03:29.194359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:41:42.901192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195070+0000","last_change":"2024-05-10T04:02:35.652370+0000","last_active":"2024-05-10T04:03:29.195070+0000","last_peered":"2024-05-10T04:03:29.195070+0000","last_clean":"2024-05-10T04:03:29.195070+0000","last_became_active":"2024-05-10T04:02:35.651803+0000","last_became_peered":"2024-05-10T04:02:35.651803+0000","last_unstale":"2024-05-10T04:03:29.195070+0000","last_undegraded":"2024-05-10T04:03:29.195070+0000","last_fullsized":"2024-05-10T04:03:29.195070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:33:36.049311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192316+0000","last_change":"2024-05-10T04:02:31.189053+0000","last_active":"2024-05-10T04:03:29.192316+0000","last_peered":"2024-05-10T04:03:29.192316+0000","last_clean":"2024-05-10T04:03:29.192316+0000","last_became_active":"2024-05-10T04:02:31.188614+0000","last_became_peered":"2024-05-10T04:02:31.188614+0000","last_unstale":"2024-05-10T04:03:29.192316+0000","last_undegraded":"2024-05-10T04:03:29.192316+0000","last_fullsized":"2024-05-10T04:03:29.192316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:59:37.750507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198409+0000","last_change":"2024-05-10T04:02:31.630379+0000","last_active":"2024-05-10T04:03:29.198409+0000","last_peered":"2024-05-10T04:03:29.198409+0000","last_clean":"2024-05-10T04:03:29.198409+0000","last_became_active":"2024-05-10T04:02:31.630094+0000","last_became_peered":"2024-05-10T04:02:31.630094+0000","last_unstale":"2024-05-10T04:03:29.198409+0000","last_undegraded":"2024-05-10T04:03:29.198409+0000","last_fullsized":"2024-05-10T04:03:29.198409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:05.699923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197360+0000","last_change":"2024-05-10T04:02:35.650770+0000","last_active":"2024-05-10T04:03:29.197360+0000","last_peered":"2024-05-10T04:03:29.197360+0000","last_clean":"2024-05-10T04:03:29.197360+0000","last_became_active":"2024-05-10T04:02:35.650351+0000","last_became_peered":"2024-05-10T04:02:35.650351+0000","last_unstale":"2024-05-10T04:03:29.197360+0000","last_undegraded":"2024-05-10T04:03:29.197360+0000","last_fullsized":"2024-05-10T04:03:29.197360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:41:06.728126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192986+0000","last_change":"2024-05-10T04:02:31.189557+0000","last_active":"2024-05-10T04:03:29.192986+0000","last_peered":"2024-05-10T04:03:29.192986+0000","last_clean":"2024-05-10T04:03:29.192986+0000","last_became_active":"2024-05-10T04:02:31.189206+0000","last_became_peered":"2024-05-10T04:02:31.189206+0000","last_unstale":"2024-05-10T04:03:29.192986+0000","last_undegraded":"2024-05-10T04:03:29.192986+0000","last_fullsized":"2024-05-10T04:03:29.192986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:11:08.225965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192249+0000","last_change":"2024-05-10T04:02:31.633561+0000","last_active":"2024-05-10T04:03:29.192249+0000","last_peered":"2024-05-10T04:03:29.192249+0000","last_clean":"2024-05-10T04:03:29.192249+0000","last_became_active":"2024-05-10T04:02:31.633224+0000","last_became_peered":"2024-05-10T04:02:31.633224+0000","last_unstale":"2024-05-10T04:03:29.192249+0000","last_undegraded":"2024-05-10T04:03:29.192249+0000","last_fullsized":"2024-05-10T04:03:29.192249+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:32.901146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198675+0000","last_change":"2024-05-10T04:02:35.649327+0000","last_active":"2024-05-10T04:03:29.198675+0000","last_peered":"2024-05-10T04:03:29.198675+0000","last_clean":"2024-05-10T04:03:29.198675+0000","last_became_active":"2024-05-10T04:02:35.648982+0000","last_became_peered":"2024-05-10T04:02:35.648982+0000","last_unstale":"2024-05-10T04:03:29.198675+0000","last_undegraded":"2024-05-10T04:03:29.198675+0000","last_fullsized":"2024-05-10T04:03:29.198675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:25.405748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":87,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:03.296425+0000","last_change":"2024-05-10T04:02:31.186447+0000","last_active":"2024-05-10T04:04:03.296425+0000","last_peered":"2024-05-10T04:04:03.296425+0000","last_clean":"2024-05-10T04:04:03.296425+0000","last_became_active":"2024-05-10T04:02:31.186073+0000","last_became_peered":"2024-05-10T04:02:31.186073+0000","last_unstale":"2024-05-10T04:04:03.296425+0000","last_undegraded":"2024-05-10T04:04:03.296425+0000","last_fullsized":"2024-05-10T04:04:03.296425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:53.381147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200415+0000","last_change":"2024-05-10T04:02:31.633772+0000","last_active":"2024-05-10T04:03:29.200415+0000","last_peered":"2024-05-10T04:03:29.200415+0000","last_clean":"2024-05-10T04:03:29.200415+0000","last_became_active":"2024-05-10T04:02:31.629592+0000","last_became_peered":"2024-05-10T04:02:31.629592+0000","last_unstale":"2024-05-10T04:03:29.200415+0000","last_undegraded":"2024-05-10T04:03:29.200415+0000","last_fullsized":"2024-05-10T04:03:29.200415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:38:37.342351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191885+0000","last_change":"2024-05-10T04:02:35.646960+0000","last_active":"2024-05-10T04:03:29.191885+0000","last_peered":"2024-05-10T04:03:29.191885+0000","last_clean":"2024-05-10T04:03:29.191885+0000","last_became_active":"2024-05-10T04:02:35.646598+0000","last_became_peered":"2024-05-10T04:02:35.646598+0000","last_unstale":"2024-05-10T04:03:29.191885+0000","last_undegraded":"2024-05-10T04:03:29.191885+0000","last_fullsized":"2024-05-10T04:03:29.191885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:40:10.478745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192245+0000","last_change":"2024-05-10T04:02:31.192846+0000","last_active":"2024-05-10T04:03:29.192245+0000","last_peered":"2024-05-10T04:03:29.192245+0000","last_clean":"2024-05-10T04:03:29.192245+0000","last_became_active":"2024-05-10T04:02:31.192514+0000","last_became_peered":"2024-05-10T04:02:31.192514+0000","last_unstale":"2024-05-10T04:03:29.192245+0000","last_undegraded":"2024-05-10T04:03:29.192245+0000","last_fullsized":"2024-05-10T04:03:29.192245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:50:32.649440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197690+0000","last_change":"2024-05-10T04:02:31.641578+0000","last_active":"2024-05-10T04:03:29.197690+0000","last_peered":"2024-05-10T04:03:29.197690+0000","last_clean":"2024-05-10T04:03:29.197690+0000","last_became_active":"2024-05-10T04:02:31.641268+0000","last_became_peered":"2024-05-10T04:02:31.641268+0000","last_unstale":"2024-05-10T04:03:29.197690+0000","last_undegraded":"2024-05-10T04:03:29.197690+0000","last_fullsized":"2024-05-10T04:03:29.197690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:31:00.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195504+0000","last_change":"2024-05-10T04:02:35.646754+0000","last_active":"2024-05-10T04:03:29.195504+0000","last_peered":"2024-05-10T04:03:29.195504+0000","last_clean":"2024-05-10T04:03:29.195504+0000","last_became_active":"2024-05-10T04:02:35.646342+0000","last_became_peered":"2024-05-10T04:02:35.646342+0000","last_unstale":"2024-05-10T04:03:29.195504+0000","last_undegraded":"2024-05-10T04:03:29.195504+0000","last_fullsized":"2024-05-10T04:03:29.195504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:28:33.373312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192990+0000","last_change":"2024-05-10T04:02:31.186427+0000","last_active":"2024-05-10T04:03:29.192990+0000","last_peered":"2024-05-10T04:03:29.192990+0000","last_clean":"2024-05-10T04:03:29.192990+0000","last_became_active":"2024-05-10T04:02:31.185937+0000","last_became_peered":"2024-05-10T04:02:31.185937+0000","last_unstale":"2024-05-10T04:03:29.192990+0000","last_undegraded":"2024-05-10T04:03:29.192990+0000","last_fullsized":"2024-05-10T04:03:29.192990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.818126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200293+0000","last_change":"2024-05-10T04:02:31.645582+0000","last_active":"2024-05-10T04:03:29.200293+0000","last_peered":"2024-05-10T04:03:29.200293+0000","last_clean":"2024-05-10T04:03:29.200293+0000","last_became_active":"2024-05-10T04:02:31.645259+0000","last_became_peered":"2024-05-10T04:02:31.645259+0000","last_unstale":"2024-05-10T04:03:29.200293+0000","last_undegraded":"2024-05-10T04:03:29.200293+0000","last_fullsized":"2024-05-10T04:03:29.200293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:17:31.818116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192983+0000","last_change":"2024-05-10T04:02:35.647341+0000","last_active":"2024-05-10T04:03:29.192983+0000","last_peered":"2024-05-10T04:03:29.192983+0000","last_clean":"2024-05-10T04:03:29.192983+0000","last_became_active":"2024-05-10T04:02:35.646806+0000","last_became_peered":"2024-05-10T04:02:35.646806+0000","last_unstale":"2024-05-10T04:03:29.192983+0000","last_undegraded":"2024-05-10T04:03:29.192983+0000","last_fullsized":"2024-05-10T04:03:29.192983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:01:51.252683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":193,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:04.298928+0000","last_change":"2024-05-10T04:02:31.195377+0000","last_active":"2024-05-10T04:04:04.298928+0000","last_peered":"2024-05-10T04:04:04.298928+0000","last_clean":"2024-05-10T04:04:04.298928+0000","last_became_active":"2024-05-10T04:02:31.194965+0000","last_became_peered":"2024-05-10T04:02:31.194965+0000","last_unstale":"2024-05-10T04:04:04.298928+0000","last_undegraded":"2024-05-10T04:04:04.298928+0000","last_fullsized":"2024-05-10T04:04:04.298928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:11:09.237525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62045,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":144,"num_read_kb":133,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198701+0000","last_change":"2024-05-10T04:02:31.629600+0000","last_active":"2024-05-10T04:03:29.198701+0000","last_peered":"2024-05-10T04:03:29.198701+0000","last_clean":"2024-05-10T04:03:29.198701+0000","last_became_active":"2024-05-10T04:02:31.629264+0000","last_became_peered":"2024-05-10T04:02:31.629264+0000","last_unstale":"2024-05-10T04:03:29.198701+0000","last_undegraded":"2024-05-10T04:03:29.198701+0000","last_fullsized":"2024-05-10T04:03:29.198701+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:45:08.867783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193519+0000","last_change":"2024-05-10T04:02:35.649468+0000","last_active":"2024-05-10T04:03:29.193519+0000","last_peered":"2024-05-10T04:03:29.193519+0000","last_clean":"2024-05-10T04:03:29.193519+0000","last_became_active":"2024-05-10T04:02:35.649053+0000","last_became_peered":"2024-05-10T04:02:35.649053+0000","last_unstale":"2024-05-10T04:03:29.193519+0000","last_undegraded":"2024-05-10T04:03:29.193519+0000","last_fullsized":"2024-05-10T04:03:29.193519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:01:05.761287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188394+0000","last_change":"2024-05-10T04:02:31.195169+0000","last_active":"2024-05-10T04:03:29.188394+0000","last_peered":"2024-05-10T04:03:29.188394+0000","last_clean":"2024-05-10T04:03:29.188394+0000","last_became_active":"2024-05-10T04:02:31.194720+0000","last_became_peered":"2024-05-10T04:02:31.194720+0000","last_unstale":"2024-05-10T04:03:29.188394+0000","last_undegraded":"2024-05-10T04:03:29.188394+0000","last_fullsized":"2024-05-10T04:03:29.188394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:42.727841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191480+0000","last_change":"2024-05-10T04:02:31.631739+0000","last_active":"2024-05-10T04:03:29.191480+0000","last_peered":"2024-05-10T04:03:29.191480+0000","last_clean":"2024-05-10T04:03:29.191480+0000","last_became_active":"2024-05-10T04:02:31.631401+0000","last_became_peered":"2024-05-10T04:02:31.631401+0000","last_unstale":"2024-05-10T04:03:29.191480+0000","last_undegraded":"2024-05-10T04:03:29.191480+0000","last_fullsized":"2024-05-10T04:03:29.191480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:06.958328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196041+0000","last_change":"2024-05-10T04:02:35.651484+0000","last_active":"2024-05-10T04:03:29.196041+0000","last_peered":"2024-05-10T04:03:29.196041+0000","last_clean":"2024-05-10T04:03:29.196041+0000","last_became_active":"2024-05-10T04:02:35.651069+0000","last_became_peered":"2024-05-10T04:02:35.651069+0000","last_unstale":"2024-05-10T04:03:29.196041+0000","last_undegraded":"2024-05-10T04:03:29.196041+0000","last_fullsized":"2024-05-10T04:03:29.196041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:26:45.229101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192929+0000","last_change":"2024-05-10T04:02:31.186314+0000","last_active":"2024-05-10T04:03:29.192929+0000","last_peered":"2024-05-10T04:03:29.192929+0000","last_clean":"2024-05-10T04:03:29.192929+0000","last_became_active":"2024-05-10T04:02:31.185861+0000","last_became_peered":"2024-05-10T04:02:31.185861+0000","last_unstale":"2024-05-10T04:03:29.192929+0000","last_undegraded":"2024-05-10T04:03:29.192929+0000","last_fullsized":"2024-05-10T04:03:29.192929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:07.386978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191954+0000","last_change":"2024-05-10T04:02:31.637151+0000","last_active":"2024-05-10T04:03:29.191954+0000","last_peered":"2024-05-10T04:03:29.191954+0000","last_clean":"2024-05-10T04:03:29.191954+0000","last_became_active":"2024-05-10T04:02:31.636745+0000","last_became_peered":"2024-05-10T04:02:31.636745+0000","last_unstale":"2024-05-10T04:03:29.191954+0000","last_undegraded":"2024-05-10T04:03:29.191954+0000","last_fullsized":"2024-05-10T04:03:29.191954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:59.804391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192369+0000","last_change":"2024-05-10T04:02:35.656108+0000","last_active":"2024-05-10T04:03:29.192369+0000","last_peered":"2024-05-10T04:03:29.192369+0000","last_clean":"2024-05-10T04:03:29.192369+0000","last_became_active":"2024-05-10T04:02:35.655664+0000","last_became_peered":"2024-05-10T04:02:35.655664+0000","last_unstale":"2024-05-10T04:03:29.192369+0000","last_undegraded":"2024-05-10T04:03:29.192369+0000","last_fullsized":"2024-05-10T04:03:29.192369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:10.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:07.303591+0000","last_change":"2024-05-10T04:02:31.189432+0000","last_active":"2024-05-10T04:04:07.303591+0000","last_peered":"2024-05-10T04:04:07.303591+0000","last_clean":"2024-05-10T04:04:07.303591+0000","last_became_active":"2024-05-10T04:02:31.189062+0000","last_became_peered":"2024-05-10T04:02:31.189062+0000","last_unstale":"2024-05-10T04:04:07.303591+0000","last_undegraded":"2024-05-10T04:04:07.303591+0000","last_fullsized":"2024-05-10T04:04:07.303591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:28.452100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191766+0000","last_change":"2024-05-10T04:02:31.637517+0000","last_active":"2024-05-10T04:03:29.191766+0000","last_peered":"2024-05-10T04:03:29.191766+0000","last_clean":"2024-05-10T04:03:29.191766+0000","last_became_active":"2024-05-10T04:02:31.636518+0000","last_became_peered":"2024-05-10T04:02:31.636518+0000","last_unstale":"2024-05-10T04:03:29.191766+0000","last_undegraded":"2024-05-10T04:03:29.191766+0000","last_fullsized":"2024-05-10T04:03:29.191766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:07:09.732117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192266+0000","last_change":"2024-05-10T04:02:35.649693+0000","last_active":"2024-05-10T04:03:29.192266+0000","last_peered":"2024-05-10T04:03:29.192266+0000","last_clean":"2024-05-10T04:03:29.192266+0000","last_became_active":"2024-05-10T04:02:35.649175+0000","last_became_peered":"2024-05-10T04:02:35.649175+0000","last_unstale":"2024-05-10T04:03:29.192266+0000","last_undegraded":"2024-05-10T04:03:29.192266+0000","last_fullsized":"2024-05-10T04:03:29.192266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:03:52.340444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.211403+0000","last_change":"2024-05-10T04:03:30.211403+0000","last_active":"2024-05-10T04:03:30.211403+0000","last_peered":"2024-05-10T04:03:30.211403+0000","last_clean":"2024-05-10T04:03:30.211403+0000","last_became_active":"2024-05-10T04:03:30.198739+0000","last_became_peered":"2024-05-10T04:03:30.198739+0000","last_unstale":"2024-05-10T04:03:30.211403+0000","last_undegraded":"2024-05-10T04:03:30.211403+0000","last_fullsized":"2024-05-10T04:03:30.211403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:52.430884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194588+0000","last_change":"2024-05-10T04:02:31.641426+0000","last_active":"2024-05-10T04:03:29.194588+0000","last_peered":"2024-05-10T04:03:29.194588+0000","last_clean":"2024-05-10T04:03:29.194588+0000","last_became_active":"2024-05-10T04:02:31.641054+0000","last_became_peered":"2024-05-10T04:02:31.641054+0000","last_unstale":"2024-05-10T04:03:29.194588+0000","last_undegraded":"2024-05-10T04:03:29.194588+0000","last_fullsized":"2024-05-10T04:03:29.194588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:12:48.472674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194503+0000","last_change":"2024-05-10T04:02:35.653258+0000","last_active":"2024-05-10T04:03:29.194503+0000","last_peered":"2024-05-10T04:03:29.194503+0000","last_clean":"2024-05-10T04:03:29.194503+0000","last_became_active":"2024-05-10T04:02:35.652859+0000","last_became_peered":"2024-05-10T04:02:35.652859+0000","last_unstale":"2024-05-10T04:03:29.194503+0000","last_undegraded":"2024-05-10T04:03:29.194503+0000","last_fullsized":"2024-05-10T04:03:29.194503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:15:57.814963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198019+0000","last_change":"2024-05-10T04:02:31.192231+0000","last_active":"2024-05-10T04:03:29.198019+0000","last_peered":"2024-05-10T04:03:29.198019+0000","last_clean":"2024-05-10T04:03:29.198019+0000","last_became_active":"2024-05-10T04:02:31.191902+0000","last_became_peered":"2024-05-10T04:02:31.191902+0000","last_unstale":"2024-05-10T04:03:29.198019+0000","last_undegraded":"2024-05-10T04:03:29.198019+0000","last_fullsized":"2024-05-10T04:03:29.198019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:18.637112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195378+0000","last_change":"2024-05-10T04:02:31.635514+0000","last_active":"2024-05-10T04:03:29.195378+0000","last_peered":"2024-05-10T04:03:29.195378+0000","last_clean":"2024-05-10T04:03:29.195378+0000","last_became_active":"2024-05-10T04:02:31.635171+0000","last_became_peered":"2024-05-10T04:02:31.635171+0000","last_unstale":"2024-05-10T04:03:29.195378+0000","last_undegraded":"2024-05-10T04:03:29.195378+0000","last_fullsized":"2024-05-10T04:03:29.195378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:48.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.475636+0000","last_change":"2024-05-10T04:03:30.475636+0000","last_active":"2024-05-10T04:03:30.475636+0000","last_peered":"2024-05-10T04:03:30.475636+0000","last_clean":"2024-05-10T04:03:30.475636+0000","last_became_active":"2024-05-10T04:03:30.475203+0000","last_became_peered":"2024-05-10T04:03:30.475203+0000","last_unstale":"2024-05-10T04:03:30.475636+0000","last_undegraded":"2024-05-10T04:03:30.475636+0000","last_fullsized":"2024-05-10T04:03:30.475636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:56:07.461905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194614+0000","last_change":"2024-05-10T04:02:31.190173+0000","last_active":"2024-05-10T04:03:29.194614+0000","last_peered":"2024-05-10T04:03:29.194614+0000","last_clean":"2024-05-10T04:03:29.194614+0000","last_became_active":"2024-05-10T04:02:31.189840+0000","last_became_peered":"2024-05-10T04:02:31.189840+0000","last_unstale":"2024-05-10T04:03:29.194614+0000","last_undegraded":"2024-05-10T04:03:29.194614+0000","last_fullsized":"2024-05-10T04:03:29.194614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:31:49.382482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199097+0000","last_change":"2024-05-10T04:02:31.629497+0000","last_active":"2024-05-10T04:03:29.199097+0000","last_peered":"2024-05-10T04:03:29.199097+0000","last_clean":"2024-05-10T04:03:29.199097+0000","last_became_active":"2024-05-10T04:02:31.629182+0000","last_became_peered":"2024-05-10T04:02:31.629182+0000","last_unstale":"2024-05-10T04:03:29.199097+0000","last_undegraded":"2024-05-10T04:03:29.199097+0000","last_fullsized":"2024-05-10T04:03:29.199097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:11:33.478737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192656+0000","last_change":"2024-05-10T04:02:35.647249+0000","last_active":"2024-05-10T04:03:29.192656+0000","last_peered":"2024-05-10T04:03:29.192656+0000","last_clean":"2024-05-10T04:03:29.192656+0000","last_became_active":"2024-05-10T04:02:35.646716+0000","last_became_peered":"2024-05-10T04:02:35.646716+0000","last_unstale":"2024-05-10T04:03:29.192656+0000","last_undegraded":"2024-05-10T04:03:29.192656+0000","last_fullsized":"2024-05-10T04:03:29.192656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:44.651836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191715+0000","last_change":"2024-05-10T04:02:31.191241+0000","last_active":"2024-05-10T04:03:29.191715+0000","last_peered":"2024-05-10T04:03:29.191715+0000","last_clean":"2024-05-10T04:03:29.191715+0000","last_became_active":"2024-05-10T04:02:31.190888+0000","last_became_peered":"2024-05-10T04:02:31.190888+0000","last_unstale":"2024-05-10T04:03:29.191715+0000","last_undegraded":"2024-05-10T04:03:29.191715+0000","last_fullsized":"2024-05-10T04:03:29.191715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:55.256119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198888+0000","last_change":"2024-05-10T04:02:31.638394+0000","last_active":"2024-05-10T04:03:29.198888+0000","last_peered":"2024-05-10T04:03:29.198888+0000","last_clean":"2024-05-10T04:03:29.198888+0000","last_became_active":"2024-05-10T04:02:31.638025+0000","last_became_peered":"2024-05-10T04:02:31.638025+0000","last_unstale":"2024-05-10T04:03:29.198888+0000","last_undegraded":"2024-05-10T04:03:29.198888+0000","last_fullsized":"2024-05-10T04:03:29.198888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:13.088393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192346+0000","last_change":"2024-05-10T04:02:35.899898+0000","last_active":"2024-05-10T04:03:29.192346+0000","last_peered":"2024-05-10T04:03:29.192346+0000","last_clean":"2024-05-10T04:03:29.192346+0000","last_became_active":"2024-05-10T04:02:35.899485+0000","last_became_peered":"2024-05-10T04:02:35.899485+0000","last_unstale":"2024-05-10T04:03:29.192346+0000","last_undegraded":"2024-05-10T04:03:29.192346+0000","last_fullsized":"2024-05-10T04:03:29.192346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:26:50.909122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193219+0000","last_change":"2024-05-10T04:02:31.640923+0000","last_active":"2024-05-10T04:03:29.193219+0000","last_peered":"2024-05-10T04:03:29.193219+0000","last_clean":"2024-05-10T04:03:29.193219+0000","last_became_active":"2024-05-10T04:02:31.640490+0000","last_became_peered":"2024-05-10T04:02:31.640490+0000","last_unstale":"2024-05-10T04:03:29.193219+0000","last_undegraded":"2024-05-10T04:03:29.193219+0000","last_fullsized":"2024-05-10T04:03:29.193219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:31:58.585789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:02.499077+0000","last_change":"2024-05-10T04:02:31.191326+0000","last_active":"2024-05-10T04:04:02.499077+0000","last_peered":"2024-05-10T04:04:02.499077+0000","last_clean":"2024-05-10T04:04:02.499077+0000","last_became_active":"2024-05-10T04:02:31.190974+0000","last_became_peered":"2024-05-10T04:02:31.190974+0000","last_unstale":"2024-05-10T04:04:02.499077+0000","last_undegraded":"2024-05-10T04:04:02.499077+0000","last_fullsized":"2024-05-10T04:04:02.499077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:43.670619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192264+0000","last_change":"2024-05-10T04:02:35.647526+0000","last_active":"2024-05-10T04:03:29.192264+0000","last_peered":"2024-05-10T04:03:29.192264+0000","last_clean":"2024-05-10T04:03:29.192264+0000","last_became_active":"2024-05-10T04:02:35.647018+0000","last_became_peered":"2024-05-10T04:02:35.647018+0000","last_unstale":"2024-05-10T04:03:29.192264+0000","last_undegraded":"2024-05-10T04:03:29.192264+0000","last_fullsized":"2024-05-10T04:03:29.192264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:17.173497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199043+0000","last_change":"2024-05-10T04:02:31.632263+0000","last_active":"2024-05-10T04:03:29.199043+0000","last_peered":"2024-05-10T04:03:29.199043+0000","last_clean":"2024-05-10T04:03:29.199043+0000","last_became_active":"2024-05-10T04:02:31.631785+0000","last_became_peered":"2024-05-10T04:02:31.631785+0000","last_unstale":"2024-05-10T04:03:29.199043+0000","last_undegraded":"2024-05-10T04:03:29.199043+0000","last_fullsized":"2024-05-10T04:03:29.199043+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:30.585862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195400+0000","last_change":"2024-05-10T04:02:31.182262+0000","last_active":"2024-05-10T04:03:29.195400+0000","last_peered":"2024-05-10T04:03:29.195400+0000","last_clean":"2024-05-10T04:03:29.195400+0000","last_became_active":"2024-05-10T04:02:31.181802+0000","last_became_peered":"2024-05-10T04:02:31.181802+0000","last_unstale":"2024-05-10T04:03:29.195400+0000","last_undegraded":"2024-05-10T04:03:29.195400+0000","last_fullsized":"2024-05-10T04:03:29.195400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:45:40.696571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":327,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191496+0000","last_change":"2024-05-10T04:01:41.866365+0000","last_active":"2024-05-10T04:03:29.191496+0000","last_peered":"2024-05-10T04:03:29.191496+0000","last_clean":"2024-05-10T04:03:29.191496+0000","last_became_active":"2024-05-10T04:01:41.865825+0000","last_became_peered":"2024-05-10T04:01:41.865825+0000","last_unstale":"2024-05-10T04:03:29.191496+0000","last_undegraded":"2024-05-10T04:03:29.191496+0000","last_fullsized":"2024-05-10T04:03:29.191496+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_clean_scrub_stamp":"2024-05-10T03:59:19.853473+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:28:29.598339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193669+0000","last_change":"2024-05-10T04:02:35.900841+0000","last_active":"2024-05-10T04:03:29.193669+0000","last_peered":"2024-05-10T04:03:29.193669+0000","last_clean":"2024-05-10T04:03:29.193669+0000","last_became_active":"2024-05-10T04:02:35.900323+0000","last_became_peered":"2024-05-10T04:02:35.900323+0000","last_unstale":"2024-05-10T04:03:29.193669+0000","last_undegraded":"2024-05-10T04:03:29.193669+0000","last_fullsized":"2024-05-10T04:03:29.193669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:00.554804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.196577+0000","last_change":"2024-05-10T04:03:30.196577+0000","last_active":"2024-05-10T04:03:30.196577+0000","last_peered":"2024-05-10T04:03:30.196577+0000","last_clean":"2024-05-10T04:03:30.196577+0000","last_became_active":"2024-05-10T04:03:30.196249+0000","last_became_peered":"2024-05-10T04:03:30.196249+0000","last_unstale":"2024-05-10T04:03:30.196577+0000","last_undegraded":"2024-05-10T04:03:30.196577+0000","last_fullsized":"2024-05-10T04:03:30.196577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:48:38.347871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191634+0000","last_change":"2024-05-10T04:02:31.187834+0000","last_active":"2024-05-10T04:03:29.191634+0000","last_peered":"2024-05-10T04:03:29.191634+0000","last_clean":"2024-05-10T04:03:29.191634+0000","last_became_active":"2024-05-10T04:02:31.187440+0000","last_became_peered":"2024-05-10T04:02:31.187440+0000","last_unstale":"2024-05-10T04:03:29.191634+0000","last_undegraded":"2024-05-10T04:03:29.191634+0000","last_fullsized":"2024-05-10T04:03:29.191634+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:50:37.678245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194942+0000","last_change":"2024-05-10T04:02:35.649664+0000","last_active":"2024-05-10T04:03:29.194942+0000","last_peered":"2024-05-10T04:03:29.194942+0000","last_clean":"2024-05-10T04:03:29.194942+0000","last_became_active":"2024-05-10T04:02:35.649076+0000","last_became_peered":"2024-05-10T04:02:35.649076+0000","last_unstale":"2024-05-10T04:03:29.194942+0000","last_undegraded":"2024-05-10T04:03:29.194942+0000","last_fullsized":"2024-05-10T04:03:29.194942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:14:42.864845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198666+0000","last_change":"2024-05-10T04:02:31.192126+0000","last_active":"2024-05-10T04:03:29.198666+0000","last_peered":"2024-05-10T04:03:29.198666+0000","last_clean":"2024-05-10T04:03:29.198666+0000","last_became_active":"2024-05-10T04:02:31.191771+0000","last_became_peered":"2024-05-10T04:02:31.191771+0000","last_unstale":"2024-05-10T04:03:29.198666+0000","last_undegraded":"2024-05-10T04:03:29.198666+0000","last_fullsized":"2024-05-10T04:03:29.198666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:43.550535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192799+0000","last_change":"2024-05-10T04:02:31.620788+0000","last_active":"2024-05-10T04:03:29.192799+0000","last_peered":"2024-05-10T04:03:29.192799+0000","last_clean":"2024-05-10T04:03:29.192799+0000","last_became_active":"2024-05-10T04:02:31.620456+0000","last_became_peered":"2024-05-10T04:02:31.620456+0000","last_unstale":"2024-05-10T04:03:29.192799+0000","last_undegraded":"2024-05-10T04:03:29.192799+0000","last_fullsized":"2024-05-10T04:03:29.192799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:02.956030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198321+0000","last_change":"2024-05-10T04:02:35.899935+0000","last_active":"2024-05-10T04:03:29.198321+0000","last_peered":"2024-05-10T04:03:29.198321+0000","last_clean":"2024-05-10T04:03:29.198321+0000","last_became_active":"2024-05-10T04:02:35.899488+0000","last_became_peered":"2024-05-10T04:02:35.899488+0000","last_unstale":"2024-05-10T04:03:29.198321+0000","last_undegraded":"2024-05-10T04:03:29.198321+0000","last_fullsized":"2024-05-10T04:03:29.198321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:28.600463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195179+0000","last_change":"2024-05-10T04:02:31.188605+0000","last_active":"2024-05-10T04:03:29.195179+0000","last_peered":"2024-05-10T04:03:29.195179+0000","last_clean":"2024-05-10T04:03:29.195179+0000","last_became_active":"2024-05-10T04:02:31.188283+0000","last_became_peered":"2024-05-10T04:02:31.188283+0000","last_unstale":"2024-05-10T04:03:29.195179+0000","last_undegraded":"2024-05-10T04:03:29.195179+0000","last_fullsized":"2024-05-10T04:03:29.195179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:40:03.940269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199712+0000","last_change":"2024-05-10T04:02:31.629476+0000","last_active":"2024-05-10T04:03:29.199712+0000","last_peered":"2024-05-10T04:03:29.199712+0000","last_clean":"2024-05-10T04:03:29.199712+0000","last_became_active":"2024-05-10T04:02:31.629104+0000","last_became_peered":"2024-05-10T04:02:31.629104+0000","last_unstale":"2024-05-10T04:03:29.199712+0000","last_undegraded":"2024-05-10T04:03:29.199712+0000","last_fullsized":"2024-05-10T04:03:29.199712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:57.873755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188055+0000","last_change":"2024-05-10T04:02:35.644094+0000","last_active":"2024-05-10T04:03:29.188055+0000","last_peered":"2024-05-10T04:03:29.188055+0000","last_clean":"2024-05-10T04:03:29.188055+0000","last_became_active":"2024-05-10T04:02:35.643382+0000","last_became_peered":"2024-05-10T04:02:35.643382+0000","last_unstale":"2024-05-10T04:03:29.188055+0000","last_undegraded":"2024-05-10T04:03:29.188055+0000","last_fullsized":"2024-05-10T04:03:29.188055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:33.250052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195430+0000","last_change":"2024-05-10T04:02:31.189302+0000","last_active":"2024-05-10T04:03:29.195430+0000","last_peered":"2024-05-10T04:03:29.195430+0000","last_clean":"2024-05-10T04:03:29.195430+0000","last_became_active":"2024-05-10T04:02:31.188323+0000","last_became_peered":"2024-05-10T04:02:31.188323+0000","last_unstale":"2024-05-10T04:03:29.195430+0000","last_undegraded":"2024-05-10T04:03:29.195430+0000","last_fullsized":"2024-05-10T04:03:29.195430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:11:02.922884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197540+0000","last_change":"2024-05-10T04:02:31.635864+0000","last_active":"2024-05-10T04:03:29.197540+0000","last_peered":"2024-05-10T04:03:29.197540+0000","last_clean":"2024-05-10T04:03:29.197540+0000","last_became_active":"2024-05-10T04:02:31.635512+0000","last_became_peered":"2024-05-10T04:02:31.635512+0000","last_unstale":"2024-05-10T04:03:29.197540+0000","last_undegraded":"2024-05-10T04:03:29.197540+0000","last_fullsized":"2024-05-10T04:03:29.197540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:26:10.957848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194647+0000","last_change":"2024-05-10T04:02:35.900489+0000","last_active":"2024-05-10T04:03:29.194647+0000","last_peered":"2024-05-10T04:03:29.194647+0000","last_clean":"2024-05-10T04:03:29.194647+0000","last_became_active":"2024-05-10T04:02:35.899908+0000","last_became_peered":"2024-05-10T04:02:35.899908+0000","last_unstale":"2024-05-10T04:03:29.194647+0000","last_undegraded":"2024-05-10T04:03:29.194647+0000","last_fullsized":"2024-05-10T04:03:29.194647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:08.386128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199847+0000","last_change":"2024-05-10T04:02:31.193956+0000","last_active":"2024-05-10T04:03:29.199847+0000","last_peered":"2024-05-10T04:03:29.199847+0000","last_clean":"2024-05-10T04:03:29.199847+0000","last_became_active":"2024-05-10T04:02:31.193574+0000","last_became_peered":"2024-05-10T04:02:31.193574+0000","last_unstale":"2024-05-10T04:03:29.199847+0000","last_undegraded":"2024-05-10T04:03:29.199847+0000","last_fullsized":"2024-05-10T04:03:29.199847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:46.369152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195487+0000","last_change":"2024-05-10T04:02:31.633263+0000","last_active":"2024-05-10T04:03:29.195487+0000","last_peered":"2024-05-10T04:03:29.195487+0000","last_clean":"2024-05-10T04:03:29.195487+0000","last_became_active":"2024-05-10T04:02:31.632903+0000","last_became_peered":"2024-05-10T04:02:31.632903+0000","last_unstale":"2024-05-10T04:03:29.195487+0000","last_undegraded":"2024-05-10T04:03:29.195487+0000","last_fullsized":"2024-05-10T04:03:29.195487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:28.047473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192521+0000","last_change":"2024-05-10T04:02:35.656003+0000","last_active":"2024-05-10T04:03:29.192521+0000","last_peered":"2024-05-10T04:03:29.192521+0000","last_clean":"2024-05-10T04:03:29.192521+0000","last_became_active":"2024-05-10T04:02:35.655518+0000","last_became_peered":"2024-05-10T04:02:35.655518+0000","last_unstale":"2024-05-10T04:03:29.192521+0000","last_undegraded":"2024-05-10T04:03:29.192521+0000","last_fullsized":"2024-05-10T04:03:29.192521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:49.696699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198559+0000","last_change":"2024-05-10T04:02:31.195297+0000","last_active":"2024-05-10T04:03:29.198559+0000","last_peered":"2024-05-10T04:03:29.198559+0000","last_clean":"2024-05-10T04:03:29.198559+0000","last_became_active":"2024-05-10T04:02:31.193960+0000","last_became_peered":"2024-05-10T04:02:31.193960+0000","last_unstale":"2024-05-10T04:03:29.198559+0000","last_undegraded":"2024-05-10T04:03:29.198559+0000","last_fullsized":"2024-05-10T04:03:29.198559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:04:10.314700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193216+0000","last_change":"2024-05-10T04:02:31.629247+0000","last_active":"2024-05-10T04:03:29.193216+0000","last_peered":"2024-05-10T04:03:29.193216+0000","last_clean":"2024-05-10T04:03:29.193216+0000","last_became_active":"2024-05-10T04:02:31.628838+0000","last_became_peered":"2024-05-10T04:02:31.628838+0000","last_unstale":"2024-05-10T04:03:29.193216+0000","last_undegraded":"2024-05-10T04:03:29.193216+0000","last_fullsized":"2024-05-10T04:03:29.193216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:46:39.483264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.252785+0000","last_change":"2024-05-10T04:03:30.252785+0000","last_active":"2024-05-10T04:03:30.252785+0000","last_peered":"2024-05-10T04:03:30.252785+0000","last_clean":"2024-05-10T04:03:30.252785+0000","last_became_active":"2024-05-10T04:03:30.248696+0000","last_became_peered":"2024-05-10T04:03:30.248696+0000","last_unstale":"2024-05-10T04:03:30.252785+0000","last_undegraded":"2024-05-10T04:03:30.252785+0000","last_fullsized":"2024-05-10T04:03:30.252785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:55:15.159906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.202901+0000","last_change":"2024-05-10T04:03:30.202456+0000","last_active":"2024-05-10T04:03:30.202901+0000","last_peered":"2024-05-10T04:03:30.202901+0000","last_clean":"2024-05-10T04:03:30.202901+0000","last_became_active":"2024-05-10T04:03:30.197484+0000","last_became_peered":"2024-05-10T04:03:30.197484+0000","last_unstale":"2024-05-10T04:03:30.202901+0000","last_undegraded":"2024-05-10T04:03:30.202901+0000","last_fullsized":"2024-05-10T04:03:30.202901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:15:07.899758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198950+0000","last_change":"2024-05-10T04:02:31.628029+0000","last_active":"2024-05-10T04:03:29.198950+0000","last_peered":"2024-05-10T04:03:29.198950+0000","last_clean":"2024-05-10T04:03:29.198950+0000","last_became_active":"2024-05-10T04:02:31.627667+0000","last_became_peered":"2024-05-10T04:02:31.627667+0000","last_unstale":"2024-05-10T04:03:29.198950+0000","last_undegraded":"2024-05-10T04:03:29.198950+0000","last_fullsized":"2024-05-10T04:03:29.198950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:39:10.878894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194397+0000","last_change":"2024-05-10T04:02:35.900237+0000","last_active":"2024-05-10T04:03:29.194397+0000","last_peered":"2024-05-10T04:03:29.194397+0000","last_clean":"2024-05-10T04:03:29.194397+0000","last_became_active":"2024-05-10T04:02:35.899781+0000","last_became_peered":"2024-05-10T04:02:35.899781+0000","last_unstale":"2024-05-10T04:03:29.194397+0000","last_undegraded":"2024-05-10T04:03:29.194397+0000","last_fullsized":"2024-05-10T04:03:29.194397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:52:46.132557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196543+0000","last_change":"2024-05-10T04:02:31.196920+0000","last_active":"2024-05-10T04:03:29.196543+0000","last_peered":"2024-05-10T04:03:29.196543+0000","last_clean":"2024-05-10T04:03:29.196543+0000","last_became_active":"2024-05-10T04:02:31.196191+0000","last_became_peered":"2024-05-10T04:02:31.196191+0000","last_unstale":"2024-05-10T04:03:29.196543+0000","last_undegraded":"2024-05-10T04:03:29.196543+0000","last_fullsized":"2024-05-10T04:03:29.196543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:45:17.980149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191841+0000","last_change":"2024-05-10T04:02:31.623678+0000","last_active":"2024-05-10T04:03:29.191841+0000","last_peered":"2024-05-10T04:03:29.191841+0000","last_clean":"2024-05-10T04:03:29.191841+0000","last_became_active":"2024-05-10T04:02:31.623090+0000","last_became_peered":"2024-05-10T04:02:31.623090+0000","last_unstale":"2024-05-10T04:03:29.191841+0000","last_undegraded":"2024-05-10T04:03:29.191841+0000","last_fullsized":"2024-05-10T04:03:29.191841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:34.321959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199997+0000","last_change":"2024-05-10T04:02:35.648833+0000","last_active":"2024-05-10T04:03:29.199997+0000","last_peered":"2024-05-10T04:03:29.199997+0000","last_clean":"2024-05-10T04:03:29.199997+0000","last_became_active":"2024-05-10T04:02:35.648391+0000","last_became_peered":"2024-05-10T04:02:35.648391+0000","last_unstale":"2024-05-10T04:03:29.199997+0000","last_undegraded":"2024-05-10T04:03:29.199997+0000","last_fullsized":"2024-05-10T04:03:29.199997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:12.773284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.189688+0000","last_change":"2024-05-10T04:02:31.192900+0000","last_active":"2024-05-10T04:03:29.189688+0000","last_peered":"2024-05-10T04:03:29.189688+0000","last_clean":"2024-05-10T04:03:29.189688+0000","last_became_active":"2024-05-10T04:02:31.192481+0000","last_became_peered":"2024-05-10T04:02:31.192481+0000","last_unstale":"2024-05-10T04:03:29.189688+0000","last_undegraded":"2024-05-10T04:03:29.189688+0000","last_fullsized":"2024-05-10T04:03:29.189688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:17.202383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192511+0000","last_change":"2024-05-10T04:02:31.628490+0000","last_active":"2024-05-10T04:03:29.192511+0000","last_peered":"2024-05-10T04:03:29.192511+0000","last_clean":"2024-05-10T04:03:29.192511+0000","last_became_active":"2024-05-10T04:02:31.627889+0000","last_became_peered":"2024-05-10T04:02:31.627889+0000","last_unstale":"2024-05-10T04:03:29.192511+0000","last_undegraded":"2024-05-10T04:03:29.192511+0000","last_fullsized":"2024-05-10T04:03:29.192511+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:32:55.366759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196442+0000","last_change":"2024-05-10T04:02:35.653530+0000","last_active":"2024-05-10T04:03:29.196442+0000","last_peered":"2024-05-10T04:03:29.196442+0000","last_clean":"2024-05-10T04:03:29.196442+0000","last_became_active":"2024-05-10T04:02:35.653095+0000","last_became_peered":"2024-05-10T04:02:35.653095+0000","last_unstale":"2024-05-10T04:03:29.196442+0000","last_undegraded":"2024-05-10T04:03:29.196442+0000","last_fullsized":"2024-05-10T04:03:29.196442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:09.921713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193200+0000","last_change":"2024-05-10T04:02:31.189434+0000","last_active":"2024-05-10T04:03:29.193200+0000","last_peered":"2024-05-10T04:03:29.193200+0000","last_clean":"2024-05-10T04:03:29.193200+0000","last_became_active":"2024-05-10T04:02:31.188843+0000","last_became_peered":"2024-05-10T04:02:31.188843+0000","last_unstale":"2024-05-10T04:03:29.193200+0000","last_undegraded":"2024-05-10T04:03:29.193200+0000","last_fullsized":"2024-05-10T04:03:29.193200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:31.508794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191797+0000","last_change":"2024-05-10T04:02:31.636775+0000","last_active":"2024-05-10T04:03:29.191797+0000","last_peered":"2024-05-10T04:03:29.191797+0000","last_clean":"2024-05-10T04:03:29.191797+0000","last_became_active":"2024-05-10T04:02:31.635946+0000","last_became_peered":"2024-05-10T04:02:31.635946+0000","last_unstale":"2024-05-10T04:03:29.191797+0000","last_undegraded":"2024-05-10T04:03:29.191797+0000","last_fullsized":"2024-05-10T04:03:29.191797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:48.830097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192629+0000","last_change":"2024-05-10T04:02:35.646991+0000","last_active":"2024-05-10T04:03:29.192629+0000","last_peered":"2024-05-10T04:03:29.192629+0000","last_clean":"2024-05-10T04:03:29.192629+0000","last_became_active":"2024-05-10T04:02:35.646585+0000","last_became_peered":"2024-05-10T04:02:35.646585+0000","last_unstale":"2024-05-10T04:03:29.192629+0000","last_undegraded":"2024-05-10T04:03:29.192629+0000","last_fullsized":"2024-05-10T04:03:29.192629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:32:50.329746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192244+0000","last_change":"2024-05-10T04:02:31.192103+0000","last_active":"2024-05-10T04:03:29.192244+0000","last_peered":"2024-05-10T04:03:29.192244+0000","last_clean":"2024-05-10T04:03:29.192244+0000","last_became_active":"2024-05-10T04:02:31.191378+0000","last_became_peered":"2024-05-10T04:02:31.191378+0000","last_unstale":"2024-05-10T04:03:29.192244+0000","last_undegraded":"2024-05-10T04:03:29.192244+0000","last_fullsized":"2024-05-10T04:03:29.192244+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:55.298158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192829+0000","last_change":"2024-05-10T04:02:31.628587+0000","last_active":"2024-05-10T04:03:29.192829+0000","last_peered":"2024-05-10T04:03:29.192829+0000","last_clean":"2024-05-10T04:03:29.192829+0000","last_became_active":"2024-05-10T04:02:31.627296+0000","last_became_peered":"2024-05-10T04:02:31.627296+0000","last_unstale":"2024-05-10T04:03:29.192829+0000","last_undegraded":"2024-05-10T04:03:29.192829+0000","last_fullsized":"2024-05-10T04:03:29.192829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:04:09.783266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193327+0000","last_change":"2024-05-10T04:02:31.634668+0000","last_active":"2024-05-10T04:03:29.193327+0000","last_peered":"2024-05-10T04:03:29.193327+0000","last_clean":"2024-05-10T04:03:29.193327+0000","last_became_active":"2024-05-10T04:02:31.634321+0000","last_became_peered":"2024-05-10T04:02:31.634321+0000","last_unstale":"2024-05-10T04:03:29.193327+0000","last_undegraded":"2024-05-10T04:03:29.193327+0000","last_fullsized":"2024-05-10T04:03:29.193327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:16:44.833804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188283+0000","last_change":"2024-05-10T04:02:31.646002+0000","last_active":"2024-05-10T04:03:29.188283+0000","last_peered":"2024-05-10T04:03:29.188283+0000","last_clean":"2024-05-10T04:03:29.188283+0000","last_became_active":"2024-05-10T04:02:31.645673+0000","last_became_peered":"2024-05-10T04:02:31.645673+0000","last_unstale":"2024-05-10T04:03:29.188283+0000","last_undegraded":"2024-05-10T04:03:29.188283+0000","last_fullsized":"2024-05-10T04:03:29.188283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:26:26.814523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188124+0000","last_change":"2024-05-10T04:02:31.630606+0000","last_active":"2024-05-10T04:03:29.188124+0000","last_peered":"2024-05-10T04:03:29.188124+0000","last_clean":"2024-05-10T04:03:29.188124+0000","last_became_active":"2024-05-10T04:02:31.630313+0000","last_became_peered":"2024-05-10T04:02:31.630313+0000","last_unstale":"2024-05-10T04:03:29.188124+0000","last_undegraded":"2024-05-10T04:03:29.188124+0000","last_fullsized":"2024-05-10T04:03:29.188124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:39.876016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196756+0000","last_change":"2024-05-10T04:02:31.636239+0000","last_active":"2024-05-10T04:03:29.196756+0000","last_peered":"2024-05-10T04:03:29.196756+0000","last_clean":"2024-05-10T04:03:29.196756+0000","last_became_active":"2024-05-10T04:02:31.635377+0000","last_became_peered":"2024-05-10T04:02:31.635377+0000","last_unstale":"2024-05-10T04:03:29.196756+0000","last_undegraded":"2024-05-10T04:03:29.196756+0000","last_fullsized":"2024-05-10T04:03:29.196756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:02.612285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192883+0000","last_change":"2024-05-10T04:02:31.643231+0000","last_active":"2024-05-10T04:03:29.192883+0000","last_peered":"2024-05-10T04:03:29.192883+0000","last_clean":"2024-05-10T04:03:29.192883+0000","last_became_active":"2024-05-10T04:02:31.642797+0000","last_became_peered":"2024-05-10T04:02:31.642797+0000","last_unstale":"2024-05-10T04:03:29.192883+0000","last_undegraded":"2024-05-10T04:03:29.192883+0000","last_fullsized":"2024-05-10T04:03:29.192883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:01:57.365339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196100+0000","last_change":"2024-05-10T04:02:31.633928+0000","last_active":"2024-05-10T04:03:29.196100+0000","last_peered":"2024-05-10T04:03:29.196100+0000","last_clean":"2024-05-10T04:03:29.196100+0000","last_became_active":"2024-05-10T04:02:31.633423+0000","last_became_peered":"2024-05-10T04:02:31.633423+0000","last_unstale":"2024-05-10T04:03:29.196100+0000","last_undegraded":"2024-05-10T04:03:29.196100+0000","last_fullsized":"2024-05-10T04:03:29.196100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:07.011354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.189591+0000","last_change":"2024-05-10T04:02:31.627151+0000","last_active":"2024-05-10T04:03:29.189591+0000","last_peered":"2024-05-10T04:03:29.189591+0000","last_clean":"2024-05-10T04:03:29.189591+0000","last_became_active":"2024-05-10T04:02:31.626780+0000","last_became_peered":"2024-05-10T04:02:31.626780+0000","last_unstale":"2024-05-10T04:03:29.189591+0000","last_undegraded":"2024-05-10T04:03:29.189591+0000","last_fullsized":"2024-05-10T04:03:29.189591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:32.070530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199643+0000","last_change":"2024-05-10T04:02:31.640669+0000","last_active":"2024-05-10T04:03:29.199643+0000","last_peered":"2024-05-10T04:03:29.199643+0000","last_clean":"2024-05-10T04:03:29.199643+0000","last_became_active":"2024-05-10T04:02:31.640410+0000","last_became_peered":"2024-05-10T04:02:31.640410+0000","last_unstale":"2024-05-10T04:03:29.199643+0000","last_undegraded":"2024-05-10T04:03:29.199643+0000","last_fullsized":"2024-05-10T04:03:29.199643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:02.235391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193090+0000","last_change":"2024-05-10T04:02:31.633624+0000","last_active":"2024-05-10T04:03:29.193090+0000","last_peered":"2024-05-10T04:03:29.193090+0000","last_clean":"2024-05-10T04:03:29.193090+0000","last_became_active":"2024-05-10T04:02:31.633241+0000","last_became_peered":"2024-05-10T04:02:31.633241+0000","last_unstale":"2024-05-10T04:03:29.193090+0000","last_undegraded":"2024-05-10T04:03:29.193090+0000","last_fullsized":"2024-05-10T04:03:29.193090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:58.750912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198385+0000","last_change":"2024-05-10T04:02:31.628632+0000","last_active":"2024-05-10T04:03:29.198385+0000","last_peered":"2024-05-10T04:03:29.198385+0000","last_clean":"2024-05-10T04:03:29.198385+0000","last_became_active":"2024-05-10T04:02:31.628349+0000","last_became_peered":"2024-05-10T04:02:31.628349+0000","last_unstale":"2024-05-10T04:03:29.198385+0000","last_undegraded":"2024-05-10T04:03:29.198385+0000","last_fullsized":"2024-05-10T04:03:29.198385+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:32.575168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200128+0000","last_change":"2024-05-10T04:02:31.631328+0000","last_active":"2024-05-10T04:03:29.200128+0000","last_peered":"2024-05-10T04:03:29.200128+0000","last_clean":"2024-05-10T04:03:29.200128+0000","last_became_active":"2024-05-10T04:02:31.630957+0000","last_became_peered":"2024-05-10T04:02:31.630957+0000","last_unstale":"2024-05-10T04:03:29.200128+0000","last_undegraded":"2024-05-10T04:03:29.200128+0000","last_fullsized":"2024-05-10T04:03:29.200128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:40.400911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191789+0000","last_change":"2024-05-10T04:02:31.617972+0000","last_active":"2024-05-10T04:03:29.191789+0000","last_peered":"2024-05-10T04:03:29.191789+0000","last_clean":"2024-05-10T04:03:29.191789+0000","last_became_active":"2024-05-10T04:02:31.617663+0000","last_became_peered":"2024-05-10T04:02:31.617663+0000","last_unstale":"2024-05-10T04:03:29.191789+0000","last_undegraded":"2024-05-10T04:03:29.191789+0000","last_fullsized":"2024-05-10T04:03:29.191789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:50.121955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195144+0000","last_change":"2024-05-10T04:02:31.644256+0000","last_active":"2024-05-10T04:03:29.195144+0000","last_peered":"2024-05-10T04:03:29.195144+0000","last_clean":"2024-05-10T04:03:29.195144+0000","last_became_active":"2024-05-10T04:02:31.643819+0000","last_became_peered":"2024-05-10T04:02:31.643819+0000","last_unstale":"2024-05-10T04:03:29.195144+0000","last_undegraded":"2024-05-10T04:03:29.195144+0000","last_fullsized":"2024-05-10T04:03:29.195144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:54.932547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279970,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66659,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":441,"num_read_kb":289,"num_write":158,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":211125,"data_compressed":4764,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514526,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":950272,"data_stored":480641,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90900000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73299999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.622,"15min":0.626},"min":{"1min":0.48599999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.42399999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.625,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86199999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.76300000000000001}]},{"osd":6,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63,"15min":0.623},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.5},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72099999999999997,"15min":0.71499999999999997},"min":{"1min":0.61199999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.73899999999999999}]},{"osd":7,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.83499999999999996,"15min":0.82599999999999996},"min":{"1min":0.68300000000000005,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1379999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.879},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77000000000000002,"15min":0.76800000000000002},"min":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.69199999999999995}]},{"osd":8,"last update":"Fri May 10 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.753}]},{"osd":9,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83899999999999997,"15min":0.84099999999999997},"min":{"1min":0.69099999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0469999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.90400000000000003,"15min":0.89900000000000002},"min":{"1min":0.76900000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1599999999999999,"5min":1.282,"15min":1.282},"last":0.91600000000000004}]},{"osd":10,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83099999999999996,"15min":0.82899999999999996},"min":{"1min":0.67600000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0309999999999999,"5min":1.319,"15min":1.319},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.84499999999999997,"15min":0.84599999999999997},"min":{"1min":0.68999999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0880000000000001,"5min":1.304,"15min":1.304},"last":0.77800000000000002}]}]},{"osd":10,"up_from":68,"seq":292057776161,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1089536,"data_stored":1134428,"data_compressed":54778,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.63600000000000001}]},{"osd":1,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.32200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.47799999999999998,"15min":0.48099999999999998},"min":{"1min":0.29899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67300000000000004,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.69899999999999995}]},{"osd":3,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.74099999999999999,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.40200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68799999999999994,"5min":0.872,"15min":0.872},"last":0.80000000000000004}]},{"osd":4,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.75700000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.42799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88800000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95599999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91400000000000003,"5min":1.175,"15min":1.175},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70099999999999996,"15min":0.70799999999999996},"min":{"1min":0.57699999999999996,"5min":0.504,"15min":0.504},"max":{"1min":0.81499999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.86899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":8,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90000000000000002,"5min":1.131,"15min":1.131},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.92900000000000005,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.85399999999999998}]},{"osd":9,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.76100000000000001},"min":{"1min":0.58399999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.88100000000000001,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.53100000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.79900000000000004,"5min":1.151,"15min":1.151},"last":0.81899999999999995}]},{"osd":11,"last update":"Fri May 10 04:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.871,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.78700000000000003,"15min":0.79400000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90900000000000003,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.96899999999999997}]}]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":966656,"data_stored":480877,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.435,"15min":0.443},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.46500000000000002,"15min":0.47599999999999998},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.89100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.38800000000000001}]},{"osd":1,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80700000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.61499999999999999,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71399999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.755},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.63300000000000001}]},{"osd":3,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.34799999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.52100000000000002,"15min":0.54300000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.58799999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48799999999999999}]},{"osd":4,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.67100000000000004,"15min":0.68400000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.751,"5min":1.149,"15min":1.149},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.1699999999999999}]},{"osd":7,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.74299999999999999,"15min":0.76300000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2470000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.752,"15min":0.78000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88100000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.65900000000000003}]},{"osd":8,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.53000000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.81100000000000005}]},{"osd":10,"last update":"Fri May 10 04:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93999999999999995,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.753,"15min":0.76800000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97299999999999998,"5min":1.306,"15min":1.306},"last":0.69499999999999995}]},{"osd":11,"last update":"Fri May 10 04:03:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79600000000000004}]}]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":966656,"data_stored":480797,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.378,"5min":0.376,"15min":0.376},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.54700000000000004},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.74199999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.65800000000000003,"15min":0.627},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95499999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.58499999999999996},"min":{"1min":0.46400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.5,"15min":0.47799999999999998},"min":{"1min":0.312,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81599999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48899999999999999,"15min":0.47899999999999998},"min":{"1min":0.39200000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53500000000000003}]},{"osd":3,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.35399999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80300000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.46200000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90300000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65600000000000003}]},{"osd":4,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63500000000000001,"15min":0.61399999999999999},"min":{"1min":0.47699999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88300000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.69799999999999995,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56299999999999994,"15min":0.54400000000000004},"min":{"1min":0.41899999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.74299999999999999,"15min":0.72999999999999998},"min":{"1min":0.56000000000000005,"5min":0.499,"15min":0.499},"max":{"1min":1.0489999999999999,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.55100000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.68999999999999995,"15min":0.66500000000000004},"min":{"1min":0.54300000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.71399999999999997,"15min":0.68799999999999994},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri May 10 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.69899999999999995,"15min":0.68700000000000006},"min":{"1min":0.58199999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.746},{"interface":"front","average":{"1min":0.874,"5min":0.76500000000000001,"15min":0.747},"min":{"1min":0.63600000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.81200000000000006}]},{"osd":10,"last update":"Fri May 10 04:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.71999999999999997,"15min":0.69499999999999995},"min":{"1min":0.69999999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.77400000000000002,"15min":0.74399999999999999},"min":{"1min":0.67400000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.84799999999999998}]},{"osd":11,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.873},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.89500000000000002}]}]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1015808,"data_stored":1077319,"data_compressed":53190,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.623,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.45700000000000002},"min":{"1min":0.36599999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.626,"5min":0.752,"15min":0.752},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85499999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.56399999999999995,"5min":0.376,"15min":0.376},"max":{"1min":0.83499999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66900000000000004}]},{"osd":3,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78900000000000003,"5min":1.143,"15min":1.143},"last":0.628},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76000000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.74299999999999999}]},{"osd":4,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.42599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77300000000000002,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.36699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73899999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.54300000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88600000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.67900000000000005},"min":{"1min":0.61399999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97299999999999998,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69499999999999995,"15min":0.69399999999999995},"min":{"1min":0.51100000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.90600000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.69199999999999995,"15min":0.70299999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85099999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.68500000000000005}]},{"osd":8,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.204,"15min":1.286},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88900000000000001,"5min":15.521000000000001,"15min":15.521000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":1.2829999999999999,"15min":1.3660000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0049999999999999,"5min":16.306999999999999,"15min":16.306999999999999},"last":0.89300000000000002}]},{"osd":9,"last update":"Fri May 10 04:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0509999999999999,"5min":1.167,"15min":1.167},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.753,"15min":0.751},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.83099999999999996}]},{"osd":10,"last update":"Fri May 10 04:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.622,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97099999999999997,"5min":1.232,"15min":1.232},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.66300000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93899999999999995,"5min":1.111,"15min":1.111},"last":0.874}]},{"osd":11,"last update":"Fri May 10 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.94299999999999995}]}]},{"osd":6,"up_from":44,"seq":188978561070,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":946176,"data_stored":476053,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.40200000000000002,"15min":0.39100000000000001},"min":{"1min":0.311,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.61599999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.371},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.438,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.5739999999999998,"5min":3.5739999999999998,"15min":3.5739999999999998},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.57299999999999995},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.84499999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.48299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.49299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.002,"5min":1.087,"15min":1.087},"last":0.443}]},{"osd":3,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48499999999999999,"15min":0.45600000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.50700000000000001,"15min":0.47699999999999998},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.1819999999999999,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":0.53300000000000003}]},{"osd":4,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89400000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94399999999999995,"5min":1.018,"15min":1.018},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.52200000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98199999999999998,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri May 10 04:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63500000000000001,"15min":0.63200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86899999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.629}]},{"osd":8,"last update":"Fri May 10 04:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.55000000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.503},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.57899999999999996}]},{"osd":10,"last update":"Fri May 10 04:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.75,"15min":0.76300000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.70699999999999996}]},{"osd":11,"last update":"Fri May 10 04:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1,"5min":1,"15min":1},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.73299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1015808,"data_stored":542322,"data_compressed":19169,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80700000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56100000000000005,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72699999999999998}]},{"osd":2,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.58199999999999996,"15min":0.53700000000000003},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.56299999999999994,"15min":0.52300000000000002},"min":{"1min":0.41599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri May 10 04:03:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56999999999999995,"15min":0.52000000000000002},"min":{"1min":0.46800000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81100000000000005,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57699999999999996,"15min":0.54100000000000004},"min":{"1min":0.442,"5min":0.373,"15min":0.373},"max":{"1min":0.88400000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri May 10 04:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.496,"15min":0.46200000000000002},"min":{"1min":0.36699999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.371},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.59099999999999997,"15min":0.53100000000000003},"min":{"1min":0.47099999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri May 10 04:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.56499999999999995,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri May 10 04:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.47199999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94599999999999995,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.621},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.66900000000000004,"15min":0.627},"min":{"1min":0.45500000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.069,"5min":1.119,"15min":1.119},"last":0.64300000000000002}]},{"osd":7,"last update":"Fri May 10 04:03:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.61499999999999999,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92900000000000005,"5min":1.075,"15min":1.075},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.68100000000000005,"15min":0.63500000000000001},"min":{"1min":0.47399999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.115,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.44400000000000001}]},{"osd":8,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69999999999999996,"15min":0.68799999999999994},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.55100000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83199999999999996}]},{"osd":9,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.81999999999999995}]},{"osd":10,"last update":"Fri May 10 04:03:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.80500000000000005,"15min":0.80100000000000005},"min":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0329999999999999,"5min":1.046,"15min":1.046},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.83899999999999997,"15min":0.82899999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0940000000000001,"5min":1.143,"15min":1.143},"last":0.755}]},{"osd":11,"last update":"Fri May 10 04:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.80000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":477441,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.52200000000000002},"min":{"1min":0.39900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.58999999999999997,"15min":0.53900000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.8689999999999998,"5min":4.8689999999999998,"15min":4.8689999999999998},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.62,"15min":0.55000000000000004},"min":{"1min":0.443,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.9009999999999998,"5min":4.9009999999999998,"15min":4.9009999999999998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82499999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61499999999999999}]},{"osd":3,"last update":"Fri May 10 04:03:33 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.61599999999999999,"15min":0.56599999999999995},"min":{"1min":0.42499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.8369999999999997,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.47299999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79700000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.64800000000000002,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":0.624}]},{"osd":6,"last update":"Fri May 10 04:03:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.59899999999999998,"15min":0.57999999999999996},"min":{"1min":0.32400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":4.7439999999999998,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.375},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.43099999999999999}]},{"osd":7,"last update":"Fri May 10 04:03:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.52900000000000003}]},{"osd":8,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.47799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.90000000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72999999999999998}]},{"osd":9,"last update":"Fri May 10 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.72099999999999997,"15min":0.69299999999999995},"min":{"1min":0.38500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.7999999999999998,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.47499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.54300000000000004}]},{"osd":10,"last update":"Fri May 10 04:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68300000000000005,"15min":0.67500000000000004},"min":{"1min":0.53500000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.77800000000000002,"15min":0.746},"min":{"1min":0.57999999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":4.96,"5min":4.96,"15min":4.96},"last":0.65700000000000003}]},{"osd":11,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.499,"5min":0.495,"15min":0.495},"max":{"1min":0.876,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.69699999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":1032192,"data_stored":546425,"data_compressed":19169,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.54200000000000004,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.496}]},{"osd":1,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56799999999999995,"15min":0.53500000000000003},"min":{"1min":0.41799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.59799999999999998,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.57899999999999996},"min":{"1min":0.48299999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.58999999999999997}]},{"osd":4,"last update":"Fri May 10 04:03:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.76500000000000001}]},{"osd":5,"last update":"Fri May 10 04:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.51700000000000002,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79000000000000004,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.40799999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.65500000000000003}]},{"osd":6,"last update":"Fri May 10 04:03:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.48899999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.66800000000000004,"15min":0.63900000000000001},"min":{"1min":0.52600000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri May 10 04:03:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.83899999999999997}]},{"osd":8,"last update":"Fri May 10 04:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.66300000000000003,"15min":0.621},"min":{"1min":0.61899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.753},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.66500000000000004,"15min":0.63500000000000001},"min":{"1min":0.41799999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.81799999999999995}]},{"osd":9,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.74299999999999999,"15min":0.72299999999999998},"min":{"1min":0.66800000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0189999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.874},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.59199999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.91900000000000004,"5min":1.016,"15min":1.016},"last":0.73499999999999999}]},{"osd":10,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.753,"15min":0.73699999999999999},"min":{"1min":0.66000000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.875,"5min":0.79200000000000004,"15min":0.77800000000000002},"min":{"1min":0.73399999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.88600000000000001}]},{"osd":11,"last update":"Fri May 10 04:03:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.92800000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":476247,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36199999999999999,"15min":0.35599999999999998},"min":{"1min":0.29199999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.503,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.377,"15min":0.36399999999999999},"min":{"1min":0.32600000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.41899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.747,"15min":0.747},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.53600000000000003,"15min":0.50800000000000001},"min":{"1min":0.47599999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77300000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80300000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri May 10 04:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.48899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.69899999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.52500000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78700000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri May 10 04:03:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80100000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.59199999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78800000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.502}]},{"osd":6,"last update":"Fri May 10 04:03:33 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.68000000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.53900000000000003,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.41299999999999998}]},{"osd":7,"last update":"Fri May 10 04:03:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.435,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73299999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.52700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.71999999999999997}]},{"osd":8,"last update":"Fri May 10 04:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.439}]},{"osd":9,"last update":"Fri May 10 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69599999999999995,"15min":0.71199999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89900000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83999999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.443}]},{"osd":10,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70299999999999996,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83799999999999997,"5min":1.228,"15min":1.228},"last":0.625},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.499,"5min":0.433,"15min":0.433},"max":{"1min":0.82099999999999995,"5min":1.25,"15min":1.25},"last":0.66100000000000003}]},{"osd":11,"last update":"Fri May 10 04:03:33 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.69999999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":481474,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.33900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50900000000000001,"15min":0.502},"min":{"1min":0.371,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.79000000000000004}]},{"osd":1,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41699999999999998,"15min":0.39300000000000002},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53900000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.39100000000000001},"min":{"1min":0.35099999999999998,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.53500000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.64400000000000002,"15min":0.57699999999999996},"min":{"1min":0.59399999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.91000000000000003,"5min":1.046,"15min":1.046},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79800000000000004,"5min":0.873,"15min":0.873},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.59299999999999997,"15min":0.54700000000000004},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.6050000000000004,"5min":4.6050000000000004,"15min":4.6050000000000004},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri May 10 04:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.46400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.92300000000000004,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81699999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.874}]},{"osd":6,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64000000000000001,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.89700000000000002,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.77500000000000002}]},{"osd":7,"last update":"Fri May 10 04:03:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72999999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.44900000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":1.01,"15min":1.01},"last":0.95299999999999996}]},{"osd":8,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68100000000000005,"15min":0.68500000000000005},"min":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.79900000000000004,"5min":0.879,"15min":0.879},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.56899999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.90300000000000002}]},{"osd":9,"last update":"Fri May 10 04:03:26 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.80000000000000004,"15min":0.76700000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.6449999999999996,"5min":5.6449999999999996,"15min":5.6449999999999996},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.82899999999999996},"min":{"1min":0.54500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":5.6100000000000003,"5min":5.6100000000000003,"15min":5.6100000000000003},"last":0.92600000000000005}]},{"osd":10,"last update":"Fri May 10 04:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.52300000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.93300000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83899999999999997,"15min":0.84499999999999997},"min":{"1min":0.56799999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.997,"5min":1.161,"15min":1.161},"last":0.97799999999999998}]},{"osd":11,"last update":"Fri May 10 04:03:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":1.002}]}]},{"osd":5,"up_from":37,"seq":158913790002,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":1019904,"data_stored":1071917,"data_compressed":53190,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52400000000000002,"15min":0.498},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77800000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.433,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74299999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57499999999999996,"15min":0.55100000000000005},"min":{"1min":0.45400000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.505},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.47999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.40400000000000003},"min":{"1min":0.35799999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.44,"15min":0.42199999999999999},"min":{"1min":0.37,"5min":0.249,"15min":0.249},"max":{"1min":0.66100000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80500000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.74299999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61099999999999999,"15min":0.59199999999999997},"min":{"1min":0.48199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.874,"15min":0.874},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66300000000000003,"15min":0.65500000000000003},"min":{"1min":0.51700000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85999999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78000000000000003}]},{"osd":7,"last update":"Fri May 10 04:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.70799999999999996,"15min":0.68500000000000005},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.68899999999999995,"15min":0.65600000000000003},"min":{"1min":0.56100000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.81000000000000005}]},{"osd":8,"last update":"Fri May 10 04:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91900000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.46000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.74199999999999999}]},{"osd":9,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.55000000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.88400000000000001}]},{"osd":10,"last update":"Fri May 10 04:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77100000000000002,"15min":0.76900000000000002},"min":{"1min":0.56299999999999994,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80300000000000005,"15min":0.80200000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.85099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":66359,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70462,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64282,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:04:09.440 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-10T04:04:09.440 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T04:04:09.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:09 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4198426637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:04:09.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:09 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1207869510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:04:09.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:09 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4198426637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:04:09.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:09 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1207869510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:04:09.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.572+0000 7f97add6a700 1 -- 172.21.15.150:0/2561727281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a81031a0 msgr2=0x7f97a81035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:09.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.572+0000 7f97add6a700 1 --2- 172.21.15.150:0/2561727281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a81031a0 0x7f97a81035b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f979c009990 tx=0x7f979c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.572+0000 7f97add6a700 1 -- 172.21.15.150:0/2561727281 shutdown_connections 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.572+0000 7f97add6a700 1 --2- 172.21.15.150:0/2561727281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a8100530 0x7f97a810a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.572+0000 7f97add6a700 1 --2- 172.21.15.150:0/2561727281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a81031a0 0x7f97a81035b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.572+0000 7f97add6a700 1 --2- 172.21.15.150:0/2561727281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a80ffbe0 0x7f97a80ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.572+0000 7f97add6a700 1 -- 172.21.15.150:0/2561727281 >> 172.21.15.150:0/2561727281 conn(0x7f97a80fb8b0 msgr2=0x7f97a80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.572+0000 7f97add6a700 1 -- 172.21.15.150:0/2561727281 shutdown_connections 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.572+0000 7f97add6a700 1 -- 172.21.15.150:0/2561727281 wait complete. 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.572+0000 7f97add6a700 1 Processor -- start 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97add6a700 1 -- start start 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97add6a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a80ffbe0 0x7f97a8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97add6a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a8100530 0x7f97a81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97add6a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a81031a0 0x7f97a819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97add6a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a819fa90 con 0x7f97a81031a0 2024-05-10T04:04:09.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97add6a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a819fc00 con 0x7f97a8100530 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97add6a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a819fdc0 con 0x7f97a80ffbe0 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97a7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a81031a0 0x7f97a819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97a7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a81031a0 0x7f97a819f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50088/0 (socket says 172.21.15.150:50088) 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97a7fff700 1 -- 172.21.15.150:0/2152164072 learned_addr learned my addr 172.21.15.150:0/2152164072 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97a6ffd700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a8100530 0x7f97a81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97a77fe700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a80ffbe0 0x7f97a8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97a7fff700 1 -- 172.21.15.150:0/2152164072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a80ffbe0 msgr2=0x7f97a8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97a7fff700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a80ffbe0 0x7f97a8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97a7fff700 1 -- 172.21.15.150:0/2152164072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a8100530 msgr2=0x7f97a81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97a7fff700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a8100530 0x7f97a81984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97a7fff700 1 -- 172.21.15.150:0/2152164072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0095f0 con 0x7f97a81031a0 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.573+0000 7f97a77fe700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a80ffbe0 0x7f97a8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.574+0000 7f97a7fff700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a81031a0 0x7f97a819f480 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f97a0007ee0 tx=0x7f97a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:09.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.574+0000 7f97a4ff9700 1 -- 172.21.15.150:0/2152164072 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000ddd0 con 0x7f97a81031a0 2024-05-10T04:04:09.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.574+0000 7f97a4ff9700 1 -- 172.21.15.150:0/2152164072 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97a000f040 con 0x7f97a81031a0 2024-05-10T04:04:09.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.574+0000 7f97add6a700 1 -- 172.21.15.150:0/2152164072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a81a00a0 con 0x7f97a81031a0 2024-05-10T04:04:09.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.574+0000 7f97a4ff9700 1 -- 172.21.15.150:0/2152164072 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a00048e0 con 0x7f97a81031a0 2024-05-10T04:04:09.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.574+0000 7f97add6a700 1 -- 172.21.15.150:0/2152164072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a81a0560 con 0x7f97a81031a0 2024-05-10T04:04:09.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.575+0000 7f97add6a700 1 -- 172.21.15.150:0/2152164072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a8108ff0 con 0x7f97a81031a0 2024-05-10T04:04:09.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.576+0000 7f97a4ff9700 1 -- 172.21.15.150:0/2152164072 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f97a0003a00 con 0x7f97a81031a0 2024-05-10T04:04:09.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.576+0000 7f97a4ff9700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f97900a1230 0x7f97900a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:09.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.577+0000 7f97a77fe700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f97900a1230 0x7f97900a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:09.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.577+0000 7f97a4ff9700 1 -- 172.21.15.150:0/2152164072 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f97a00ccf40 con 0x7f97a81031a0 2024-05-10T04:04:09.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.577+0000 7f97a77fe700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f97900a1230 0x7f97900a3740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f979c003730 tx=0x7f979c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:09.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.580+0000 7f97a4ff9700 1 -- 172.21.15.150:0/2152164072 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f97a0096200 con 0x7f97a81031a0 2024-05-10T04:04:09.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.775+0000 7f97add6a700 1 -- 172.21.15.150:0/2152164072 --> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97a8061650 con 0x7f97900a1230 2024-05-10T04:04:09.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.788+0000 7f97a4ff9700 1 -- 172.21.15.150:0/2152164072 <== mgr.14152 v2:172.21.15.150:6800/1954703650 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427782 (secure 0 0 0) 0x7f97a8061650 con 0x7f97900a1230 2024-05-10T04:04:09.796 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:04:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.791+0000 7f97add6a700 1 -- 172.21.15.150:0/2152164072 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f97900a1230 msgr2=0x7f97900a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.791+0000 7f97add6a700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f97900a1230 0x7f97900a3740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f979c003730 tx=0x7f979c01a040 comp rx=0 tx=0).stop 2024-05-10T04:04:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.791+0000 7f97add6a700 1 -- 172.21.15.150:0/2152164072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a81031a0 msgr2=0x7f97a819f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.792+0000 7f97add6a700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a81031a0 0x7f97a819f480 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f97a0007ee0 tx=0x7f97a000d3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.792+0000 7f97add6a700 1 -- 172.21.15.150:0/2152164072 shutdown_connections 2024-05-10T04:04:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.792+0000 7f97add6a700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f97900a1230 0x7f97900a3740 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.792+0000 7f97add6a700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a81031a0 0x7f97a819f480 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.792+0000 7f97add6a700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a80ffbe0 0x7f97a8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.792+0000 7f97add6a700 1 --2- 172.21.15.150:0/2152164072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a8100530 0x7f97a81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.792+0000 7f97add6a700 1 -- 172.21.15.150:0/2152164072 >> 172.21.15.150:0/2152164072 conn(0x7f97a80fb8b0 msgr2=0x7f97a8106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.792+0000 7f97add6a700 1 -- 172.21.15.150:0/2152164072 shutdown_connections 2024-05-10T04:04:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.792+0000 7f97add6a700 1 -- 172.21.15.150:0/2152164072 wait complete. 2024-05-10T04:04:09.801 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:04:09.807 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-05-10T04:04:09.106963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657261,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":567,"num_read_kb":398,"num_write":341,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180965,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372720,"kb_used_data":11620,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569104,"statfs":{"total":1151940427776,"available":1151558762496,"internally_reserved":0,"allocated":11898880,"data_stored":7725941,"data_compressed":322563,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":628,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":32,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003946"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195663+0000","last_change":"2024-05-10T04:02:31.642089+0000","last_active":"2024-05-10T04:03:29.195663+0000","last_peered":"2024-05-10T04:03:29.195663+0000","last_clean":"2024-05-10T04:03:29.195663+0000","last_became_active":"2024-05-10T04:02:31.641080+0000","last_became_peered":"2024-05-10T04:02:31.641080+0000","last_unstale":"2024-05-10T04:03:29.195663+0000","last_undegraded":"2024-05-10T04:03:29.195663+0000","last_fullsized":"2024-05-10T04:03:29.195663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:29.693139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197452+0000","last_change":"2024-05-10T04:02:31.633056+0000","last_active":"2024-05-10T04:03:29.197452+0000","last_peered":"2024-05-10T04:03:29.197452+0000","last_clean":"2024-05-10T04:03:29.197452+0000","last_became_active":"2024-05-10T04:02:31.632725+0000","last_became_peered":"2024-05-10T04:02:31.632725+0000","last_unstale":"2024-05-10T04:03:29.197452+0000","last_undegraded":"2024-05-10T04:03:29.197452+0000","last_fullsized":"2024-05-10T04:03:29.197452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:08:40.365433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197250+0000","last_change":"2024-05-10T04:02:31.625219+0000","last_active":"2024-05-10T04:03:29.197250+0000","last_peered":"2024-05-10T04:03:29.197250+0000","last_clean":"2024-05-10T04:03:29.197250+0000","last_became_active":"2024-05-10T04:02:31.624889+0000","last_became_peered":"2024-05-10T04:02:31.624889+0000","last_unstale":"2024-05-10T04:03:29.197250+0000","last_undegraded":"2024-05-10T04:03:29.197250+0000","last_fullsized":"2024-05-10T04:03:29.197250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:10:59.081031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.199344+0000","last_change":"2024-05-10T04:03:30.198971+0000","last_active":"2024-05-10T04:03:30.199344+0000","last_peered":"2024-05-10T04:03:30.199344+0000","last_clean":"2024-05-10T04:03:30.199344+0000","last_became_active":"2024-05-10T04:03:30.198664+0000","last_became_peered":"2024-05-10T04:03:30.198664+0000","last_unstale":"2024-05-10T04:03:30.199344+0000","last_undegraded":"2024-05-10T04:03:30.199344+0000","last_fullsized":"2024-05-10T04:03:30.199344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:22.655875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195511+0000","last_change":"2024-05-10T04:02:31.630688+0000","last_active":"2024-05-10T04:03:29.195511+0000","last_peered":"2024-05-10T04:03:29.195511+0000","last_clean":"2024-05-10T04:03:29.195511+0000","last_became_active":"2024-05-10T04:02:31.629252+0000","last_became_peered":"2024-05-10T04:02:31.629252+0000","last_unstale":"2024-05-10T04:03:29.195511+0000","last_undegraded":"2024-05-10T04:03:29.195511+0000","last_fullsized":"2024-05-10T04:03:29.195511+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:37.647516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198848+0000","last_change":"2024-05-10T04:02:31.642586+0000","last_active":"2024-05-10T04:03:29.198848+0000","last_peered":"2024-05-10T04:03:29.198848+0000","last_clean":"2024-05-10T04:03:29.198848+0000","last_became_active":"2024-05-10T04:02:31.642244+0000","last_became_peered":"2024-05-10T04:02:31.642244+0000","last_unstale":"2024-05-10T04:03:29.198848+0000","last_undegraded":"2024-05-10T04:03:29.198848+0000","last_fullsized":"2024-05-10T04:03:29.198848+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:28.020151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191697+0000","last_change":"2024-05-10T04:02:31.636706+0000","last_active":"2024-05-10T04:03:29.191697+0000","last_peered":"2024-05-10T04:03:29.191697+0000","last_clean":"2024-05-10T04:03:29.191697+0000","last_became_active":"2024-05-10T04:02:31.636303+0000","last_became_peered":"2024-05-10T04:02:31.636303+0000","last_unstale":"2024-05-10T04:03:29.191697+0000","last_undegraded":"2024-05-10T04:03:29.191697+0000","last_fullsized":"2024-05-10T04:03:29.191697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:16.941400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198297+0000","last_change":"2024-05-10T04:02:31.640032+0000","last_active":"2024-05-10T04:03:29.198297+0000","last_peered":"2024-05-10T04:03:29.198297+0000","last_clean":"2024-05-10T04:03:29.198297+0000","last_became_active":"2024-05-10T04:02:31.639733+0000","last_became_peered":"2024-05-10T04:02:31.639733+0000","last_unstale":"2024-05-10T04:03:29.198297+0000","last_undegraded":"2024-05-10T04:03:29.198297+0000","last_fullsized":"2024-05-10T04:03:29.198297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:02:36.963414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200412+0000","last_change":"2024-05-10T04:02:31.638940+0000","last_active":"2024-05-10T04:03:29.200412+0000","last_peered":"2024-05-10T04:03:29.200412+0000","last_clean":"2024-05-10T04:03:29.200412+0000","last_became_active":"2024-05-10T04:02:31.638617+0000","last_became_peered":"2024-05-10T04:02:31.638617+0000","last_unstale":"2024-05-10T04:03:29.200412+0000","last_undegraded":"2024-05-10T04:03:29.200412+0000","last_fullsized":"2024-05-10T04:03:29.200412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:10:36.348117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199245+0000","last_change":"2024-05-10T04:02:31.635702+0000","last_active":"2024-05-10T04:03:29.199245+0000","last_peered":"2024-05-10T04:03:29.199245+0000","last_clean":"2024-05-10T04:03:29.199245+0000","last_became_active":"2024-05-10T04:02:31.635330+0000","last_became_peered":"2024-05-10T04:02:31.635330+0000","last_unstale":"2024-05-10T04:03:29.199245+0000","last_undegraded":"2024-05-10T04:03:29.199245+0000","last_fullsized":"2024-05-10T04:03:29.199245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:21:15.160340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192388+0000","last_change":"2024-05-10T04:02:31.628252+0000","last_active":"2024-05-10T04:03:29.192388+0000","last_peered":"2024-05-10T04:03:29.192388+0000","last_clean":"2024-05-10T04:03:29.192388+0000","last_became_active":"2024-05-10T04:02:31.627394+0000","last_became_peered":"2024-05-10T04:02:31.627394+0000","last_unstale":"2024-05-10T04:03:29.192388+0000","last_undegraded":"2024-05-10T04:03:29.192388+0000","last_fullsized":"2024-05-10T04:03:29.192388+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:54.569786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196477+0000","last_change":"2024-05-10T04:02:31.641788+0000","last_active":"2024-05-10T04:03:29.196477+0000","last_peered":"2024-05-10T04:03:29.196477+0000","last_clean":"2024-05-10T04:03:29.196477+0000","last_became_active":"2024-05-10T04:02:31.641328+0000","last_became_peered":"2024-05-10T04:02:31.641328+0000","last_unstale":"2024-05-10T04:03:29.196477+0000","last_undegraded":"2024-05-10T04:03:29.196477+0000","last_fullsized":"2024-05-10T04:03:29.196477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:10:55.920245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191930+0000","last_change":"2024-05-10T04:02:31.631919+0000","last_active":"2024-05-10T04:03:29.191930+0000","last_peered":"2024-05-10T04:03:29.191930+0000","last_clean":"2024-05-10T04:03:29.191930+0000","last_became_active":"2024-05-10T04:02:31.631592+0000","last_became_peered":"2024-05-10T04:02:31.631592+0000","last_unstale":"2024-05-10T04:03:29.191930+0000","last_undegraded":"2024-05-10T04:03:29.191930+0000","last_fullsized":"2024-05-10T04:03:29.191930+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:59:03.792090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196091+0000","last_change":"2024-05-10T04:02:31.633288+0000","last_active":"2024-05-10T04:03:29.196091+0000","last_peered":"2024-05-10T04:03:29.196091+0000","last_clean":"2024-05-10T04:03:29.196091+0000","last_became_active":"2024-05-10T04:02:31.632582+0000","last_became_peered":"2024-05-10T04:02:31.632582+0000","last_unstale":"2024-05-10T04:03:29.196091+0000","last_undegraded":"2024-05-10T04:03:29.196091+0000","last_fullsized":"2024-05-10T04:03:29.196091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:27.991439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196261+0000","last_change":"2024-05-10T04:02:31.635833+0000","last_active":"2024-05-10T04:03:29.196261+0000","last_peered":"2024-05-10T04:03:29.196261+0000","last_clean":"2024-05-10T04:03:29.196261+0000","last_became_active":"2024-05-10T04:02:31.635520+0000","last_became_peered":"2024-05-10T04:02:31.635520+0000","last_unstale":"2024-05-10T04:03:29.196261+0000","last_undegraded":"2024-05-10T04:03:29.196261+0000","last_fullsized":"2024-05-10T04:03:29.196261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:40:06.312819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199745+0000","last_change":"2024-05-10T04:02:31.631734+0000","last_active":"2024-05-10T04:03:29.199745+0000","last_peered":"2024-05-10T04:03:29.199745+0000","last_clean":"2024-05-10T04:03:29.199745+0000","last_became_active":"2024-05-10T04:02:31.631375+0000","last_became_peered":"2024-05-10T04:02:31.631375+0000","last_unstale":"2024-05-10T04:03:29.199745+0000","last_undegraded":"2024-05-10T04:03:29.199745+0000","last_fullsized":"2024-05-10T04:03:29.199745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:25:13.815406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195179+0000","last_change":"2024-05-10T04:02:31.642292+0000","last_active":"2024-05-10T04:03:29.195179+0000","last_peered":"2024-05-10T04:03:29.195179+0000","last_clean":"2024-05-10T04:03:29.195179+0000","last_became_active":"2024-05-10T04:02:31.641570+0000","last_became_peered":"2024-05-10T04:02:31.641570+0000","last_unstale":"2024-05-10T04:03:29.195179+0000","last_undegraded":"2024-05-10T04:03:29.195179+0000","last_fullsized":"2024-05-10T04:03:29.195179+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:16:33.704809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195044+0000","last_change":"2024-05-10T04:02:31.625695+0000","last_active":"2024-05-10T04:03:29.195044+0000","last_peered":"2024-05-10T04:03:29.195044+0000","last_clean":"2024-05-10T04:03:29.195044+0000","last_became_active":"2024-05-10T04:02:31.625393+0000","last_became_peered":"2024-05-10T04:02:31.625393+0000","last_unstale":"2024-05-10T04:03:29.195044+0000","last_undegraded":"2024-05-10T04:03:29.195044+0000","last_fullsized":"2024-05-10T04:03:29.195044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:53.955921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188569+0000","last_change":"2024-05-10T04:02:31.629152+0000","last_active":"2024-05-10T04:03:29.188569+0000","last_peered":"2024-05-10T04:03:29.188569+0000","last_clean":"2024-05-10T04:03:29.188569+0000","last_became_active":"2024-05-10T04:02:31.628778+0000","last_became_peered":"2024-05-10T04:02:31.628778+0000","last_unstale":"2024-05-10T04:03:29.188569+0000","last_undegraded":"2024-05-10T04:03:29.188569+0000","last_fullsized":"2024-05-10T04:03:29.188569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:39.772500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188510+0000","last_change":"2024-05-10T04:02:35.650536+0000","last_active":"2024-05-10T04:03:29.188510+0000","last_peered":"2024-05-10T04:03:29.188510+0000","last_clean":"2024-05-10T04:03:29.188510+0000","last_became_active":"2024-05-10T04:02:35.650102+0000","last_became_peered":"2024-05-10T04:02:35.650102+0000","last_unstale":"2024-05-10T04:03:29.188510+0000","last_undegraded":"2024-05-10T04:03:29.188510+0000","last_fullsized":"2024-05-10T04:03:29.188510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:19:35.605844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192581+0000","last_change":"2024-05-10T04:02:31.186846+0000","last_active":"2024-05-10T04:03:29.192581+0000","last_peered":"2024-05-10T04:03:29.192581+0000","last_clean":"2024-05-10T04:03:29.192581+0000","last_became_active":"2024-05-10T04:02:31.186497+0000","last_became_peered":"2024-05-10T04:02:31.186497+0000","last_unstale":"2024-05-10T04:03:29.192581+0000","last_undegraded":"2024-05-10T04:03:29.192581+0000","last_fullsized":"2024-05-10T04:03:29.192581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:15:40.832455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.252299+0000","last_change":"2024-05-10T04:03:30.251863+0000","last_active":"2024-05-10T04:03:30.252299+0000","last_peered":"2024-05-10T04:03:30.252299+0000","last_clean":"2024-05-10T04:03:30.252299+0000","last_became_active":"2024-05-10T04:03:30.251466+0000","last_became_peered":"2024-05-10T04:03:30.251466+0000","last_unstale":"2024-05-10T04:03:30.252299+0000","last_undegraded":"2024-05-10T04:03:30.252299+0000","last_fullsized":"2024-05-10T04:03:30.252299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:56:12.810512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194320+0000","last_change":"2024-05-10T04:02:35.651622+0000","last_active":"2024-05-10T04:03:29.194320+0000","last_peered":"2024-05-10T04:03:29.194320+0000","last_clean":"2024-05-10T04:03:29.194320+0000","last_became_active":"2024-05-10T04:02:35.651148+0000","last_became_peered":"2024-05-10T04:02:35.651148+0000","last_unstale":"2024-05-10T04:03:29.194320+0000","last_undegraded":"2024-05-10T04:03:29.194320+0000","last_fullsized":"2024-05-10T04:03:29.194320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:40:09.019286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199659+0000","last_change":"2024-05-10T04:02:31.187785+0000","last_active":"2024-05-10T04:03:29.199659+0000","last_peered":"2024-05-10T04:03:29.199659+0000","last_clean":"2024-05-10T04:03:29.199659+0000","last_became_active":"2024-05-10T04:02:31.187499+0000","last_became_peered":"2024-05-10T04:02:31.187499+0000","last_unstale":"2024-05-10T04:03:29.199659+0000","last_undegraded":"2024-05-10T04:03:29.199659+0000","last_fullsized":"2024-05-10T04:03:29.199659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:05:29.015767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197457+0000","last_change":"2024-05-10T04:02:31.624433+0000","last_active":"2024-05-10T04:03:29.197457+0000","last_peered":"2024-05-10T04:03:29.197457+0000","last_clean":"2024-05-10T04:03:29.197457+0000","last_became_active":"2024-05-10T04:02:31.624115+0000","last_became_peered":"2024-05-10T04:02:31.624115+0000","last_unstale":"2024-05-10T04:03:29.197457+0000","last_undegraded":"2024-05-10T04:03:29.197457+0000","last_fullsized":"2024-05-10T04:03:29.197457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:54.584681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.252518+0000","last_change":"2024-05-10T04:03:30.252518+0000","last_active":"2024-05-10T04:03:30.252518+0000","last_peered":"2024-05-10T04:03:30.252518+0000","last_clean":"2024-05-10T04:03:30.252518+0000","last_became_active":"2024-05-10T04:03:30.251919+0000","last_became_peered":"2024-05-10T04:03:30.251919+0000","last_unstale":"2024-05-10T04:03:30.252518+0000","last_undegraded":"2024-05-10T04:03:30.252518+0000","last_fullsized":"2024-05-10T04:03:30.252518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:05:29.247977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.206678+0000","last_change":"2024-05-10T04:03:30.206145+0000","last_active":"2024-05-10T04:03:30.206678+0000","last_peered":"2024-05-10T04:03:30.206678+0000","last_clean":"2024-05-10T04:03:30.206678+0000","last_became_active":"2024-05-10T04:03:30.200138+0000","last_became_peered":"2024-05-10T04:03:30.200138+0000","last_unstale":"2024-05-10T04:03:30.206678+0000","last_undegraded":"2024-05-10T04:03:30.206678+0000","last_fullsized":"2024-05-10T04:03:30.206678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:49.885012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200208+0000","last_change":"2024-05-10T04:02:31.628124+0000","last_active":"2024-05-10T04:03:29.200208+0000","last_peered":"2024-05-10T04:03:29.200208+0000","last_clean":"2024-05-10T04:03:29.200208+0000","last_became_active":"2024-05-10T04:02:31.627789+0000","last_became_peered":"2024-05-10T04:02:31.627789+0000","last_unstale":"2024-05-10T04:03:29.200208+0000","last_undegraded":"2024-05-10T04:03:29.200208+0000","last_fullsized":"2024-05-10T04:03:29.200208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:40:11.240726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194237+0000","last_change":"2024-05-10T04:02:35.649536+0000","last_active":"2024-05-10T04:03:29.194237+0000","last_peered":"2024-05-10T04:03:29.194237+0000","last_clean":"2024-05-10T04:03:29.194237+0000","last_became_active":"2024-05-10T04:02:35.649130+0000","last_became_peered":"2024-05-10T04:02:35.649130+0000","last_unstale":"2024-05-10T04:03:29.194237+0000","last_undegraded":"2024-05-10T04:03:29.194237+0000","last_fullsized":"2024-05-10T04:03:29.194237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:03.870071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196512+0000","last_change":"2024-05-10T04:02:31.192627+0000","last_active":"2024-05-10T04:03:29.196512+0000","last_peered":"2024-05-10T04:03:29.196512+0000","last_clean":"2024-05-10T04:03:29.196512+0000","last_became_active":"2024-05-10T04:02:31.191433+0000","last_became_peered":"2024-05-10T04:02:31.191433+0000","last_unstale":"2024-05-10T04:03:29.196512+0000","last_undegraded":"2024-05-10T04:03:29.196512+0000","last_fullsized":"2024-05-10T04:03:29.196512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:29.235483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193100+0000","last_change":"2024-05-10T04:02:31.641192+0000","last_active":"2024-05-10T04:03:29.193100+0000","last_peered":"2024-05-10T04:03:29.193100+0000","last_clean":"2024-05-10T04:03:29.193100+0000","last_became_active":"2024-05-10T04:02:31.640540+0000","last_became_peered":"2024-05-10T04:02:31.640540+0000","last_unstale":"2024-05-10T04:03:29.193100+0000","last_undegraded":"2024-05-10T04:03:29.193100+0000","last_fullsized":"2024-05-10T04:03:29.193100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:36.734231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.200392+0000","last_change":"2024-05-10T04:03:30.200392+0000","last_active":"2024-05-10T04:03:30.200392+0000","last_peered":"2024-05-10T04:03:30.200392+0000","last_clean":"2024-05-10T04:03:30.200392+0000","last_became_active":"2024-05-10T04:03:30.199878+0000","last_became_peered":"2024-05-10T04:03:30.199878+0000","last_unstale":"2024-05-10T04:03:30.200392+0000","last_undegraded":"2024-05-10T04:03:30.200392+0000","last_fullsized":"2024-05-10T04:03:30.200392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:33:33.143992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:04.298481+0000","last_change":"2024-05-10T04:02:31.198037+0000","last_active":"2024-05-10T04:04:04.298481+0000","last_peered":"2024-05-10T04:04:04.298481+0000","last_clean":"2024-05-10T04:04:04.298481+0000","last_became_active":"2024-05-10T04:02:31.197679+0000","last_became_peered":"2024-05-10T04:02:31.197679+0000","last_unstale":"2024-05-10T04:04:04.298481+0000","last_undegraded":"2024-05-10T04:04:04.298481+0000","last_fullsized":"2024-05-10T04:04:04.298481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:19.059645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196289+0000","last_change":"2024-05-10T04:02:31.630020+0000","last_active":"2024-05-10T04:03:29.196289+0000","last_peered":"2024-05-10T04:03:29.196289+0000","last_clean":"2024-05-10T04:03:29.196289+0000","last_became_active":"2024-05-10T04:02:31.629738+0000","last_became_peered":"2024-05-10T04:02:31.629738+0000","last_unstale":"2024-05-10T04:03:29.196289+0000","last_undegraded":"2024-05-10T04:03:29.196289+0000","last_fullsized":"2024-05-10T04:03:29.196289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:58:42.317143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188383+0000","last_change":"2024-05-10T04:02:35.647242+0000","last_active":"2024-05-10T04:03:29.188383+0000","last_peered":"2024-05-10T04:03:29.188383+0000","last_clean":"2024-05-10T04:03:29.188383+0000","last_became_active":"2024-05-10T04:02:35.646774+0000","last_became_peered":"2024-05-10T04:02:35.646774+0000","last_unstale":"2024-05-10T04:03:29.188383+0000","last_undegraded":"2024-05-10T04:03:29.188383+0000","last_fullsized":"2024-05-10T04:03:29.188383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:20:05.602820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198723+0000","last_change":"2024-05-10T04:02:31.191981+0000","last_active":"2024-05-10T04:03:29.198723+0000","last_peered":"2024-05-10T04:03:29.198723+0000","last_clean":"2024-05-10T04:03:29.198723+0000","last_became_active":"2024-05-10T04:02:31.191632+0000","last_became_peered":"2024-05-10T04:02:31.191632+0000","last_unstale":"2024-05-10T04:03:29.198723+0000","last_undegraded":"2024-05-10T04:03:29.198723+0000","last_fullsized":"2024-05-10T04:03:29.198723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:44:52.744661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194359+0000","last_change":"2024-05-10T04:02:31.643240+0000","last_active":"2024-05-10T04:03:29.194359+0000","last_peered":"2024-05-10T04:03:29.194359+0000","last_clean":"2024-05-10T04:03:29.194359+0000","last_became_active":"2024-05-10T04:02:31.642891+0000","last_became_peered":"2024-05-10T04:02:31.642891+0000","last_unstale":"2024-05-10T04:03:29.194359+0000","last_undegraded":"2024-05-10T04:03:29.194359+0000","last_fullsized":"2024-05-10T04:03:29.194359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:41:42.901192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195070+0000","last_change":"2024-05-10T04:02:35.652370+0000","last_active":"2024-05-10T04:03:29.195070+0000","last_peered":"2024-05-10T04:03:29.195070+0000","last_clean":"2024-05-10T04:03:29.195070+0000","last_became_active":"2024-05-10T04:02:35.651803+0000","last_became_peered":"2024-05-10T04:02:35.651803+0000","last_unstale":"2024-05-10T04:03:29.195070+0000","last_undegraded":"2024-05-10T04:03:29.195070+0000","last_fullsized":"2024-05-10T04:03:29.195070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:33:36.049311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192316+0000","last_change":"2024-05-10T04:02:31.189053+0000","last_active":"2024-05-10T04:03:29.192316+0000","last_peered":"2024-05-10T04:03:29.192316+0000","last_clean":"2024-05-10T04:03:29.192316+0000","last_became_active":"2024-05-10T04:02:31.188614+0000","last_became_peered":"2024-05-10T04:02:31.188614+0000","last_unstale":"2024-05-10T04:03:29.192316+0000","last_undegraded":"2024-05-10T04:03:29.192316+0000","last_fullsized":"2024-05-10T04:03:29.192316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:59:37.750507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198409+0000","last_change":"2024-05-10T04:02:31.630379+0000","last_active":"2024-05-10T04:03:29.198409+0000","last_peered":"2024-05-10T04:03:29.198409+0000","last_clean":"2024-05-10T04:03:29.198409+0000","last_became_active":"2024-05-10T04:02:31.630094+0000","last_became_peered":"2024-05-10T04:02:31.630094+0000","last_unstale":"2024-05-10T04:03:29.198409+0000","last_undegraded":"2024-05-10T04:03:29.198409+0000","last_fullsized":"2024-05-10T04:03:29.198409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:05.699923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197360+0000","last_change":"2024-05-10T04:02:35.650770+0000","last_active":"2024-05-10T04:03:29.197360+0000","last_peered":"2024-05-10T04:03:29.197360+0000","last_clean":"2024-05-10T04:03:29.197360+0000","last_became_active":"2024-05-10T04:02:35.650351+0000","last_became_peered":"2024-05-10T04:02:35.650351+0000","last_unstale":"2024-05-10T04:03:29.197360+0000","last_undegraded":"2024-05-10T04:03:29.197360+0000","last_fullsized":"2024-05-10T04:03:29.197360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:41:06.728126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192986+0000","last_change":"2024-05-10T04:02:31.189557+0000","last_active":"2024-05-10T04:03:29.192986+0000","last_peered":"2024-05-10T04:03:29.192986+0000","last_clean":"2024-05-10T04:03:29.192986+0000","last_became_active":"2024-05-10T04:02:31.189206+0000","last_became_peered":"2024-05-10T04:02:31.189206+0000","last_unstale":"2024-05-10T04:03:29.192986+0000","last_undegraded":"2024-05-10T04:03:29.192986+0000","last_fullsized":"2024-05-10T04:03:29.192986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:11:08.225965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192249+0000","last_change":"2024-05-10T04:02:31.633561+0000","last_active":"2024-05-10T04:03:29.192249+0000","last_peered":"2024-05-10T04:03:29.192249+0000","last_clean":"2024-05-10T04:03:29.192249+0000","last_became_active":"2024-05-10T04:02:31.633224+0000","last_became_peered":"2024-05-10T04:02:31.633224+0000","last_unstale":"2024-05-10T04:03:29.192249+0000","last_undegraded":"2024-05-10T04:03:29.192249+0000","last_fullsized":"2024-05-10T04:03:29.192249+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:32.901146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198675+0000","last_change":"2024-05-10T04:02:35.649327+0000","last_active":"2024-05-10T04:03:29.198675+0000","last_peered":"2024-05-10T04:03:29.198675+0000","last_clean":"2024-05-10T04:03:29.198675+0000","last_became_active":"2024-05-10T04:02:35.648982+0000","last_became_peered":"2024-05-10T04:02:35.648982+0000","last_unstale":"2024-05-10T04:03:29.198675+0000","last_undegraded":"2024-05-10T04:03:29.198675+0000","last_fullsized":"2024-05-10T04:03:29.198675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:25.405748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":87,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:03.296425+0000","last_change":"2024-05-10T04:02:31.186447+0000","last_active":"2024-05-10T04:04:03.296425+0000","last_peered":"2024-05-10T04:04:03.296425+0000","last_clean":"2024-05-10T04:04:03.296425+0000","last_became_active":"2024-05-10T04:02:31.186073+0000","last_became_peered":"2024-05-10T04:02:31.186073+0000","last_unstale":"2024-05-10T04:04:03.296425+0000","last_undegraded":"2024-05-10T04:04:03.296425+0000","last_fullsized":"2024-05-10T04:04:03.296425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:53.381147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200415+0000","last_change":"2024-05-10T04:02:31.633772+0000","last_active":"2024-05-10T04:03:29.200415+0000","last_peered":"2024-05-10T04:03:29.200415+0000","last_clean":"2024-05-10T04:03:29.200415+0000","last_became_active":"2024-05-10T04:02:31.629592+0000","last_became_peered":"2024-05-10T04:02:31.629592+0000","last_unstale":"2024-05-10T04:03:29.200415+0000","last_undegraded":"2024-05-10T04:03:29.200415+0000","last_fullsized":"2024-05-10T04:03:29.200415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:38:37.342351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191885+0000","last_change":"2024-05-10T04:02:35.646960+0000","last_active":"2024-05-10T04:03:29.191885+0000","last_peered":"2024-05-10T04:03:29.191885+0000","last_clean":"2024-05-10T04:03:29.191885+0000","last_became_active":"2024-05-10T04:02:35.646598+0000","last_became_peered":"2024-05-10T04:02:35.646598+0000","last_unstale":"2024-05-10T04:03:29.191885+0000","last_undegraded":"2024-05-10T04:03:29.191885+0000","last_fullsized":"2024-05-10T04:03:29.191885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:40:10.478745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192245+0000","last_change":"2024-05-10T04:02:31.192846+0000","last_active":"2024-05-10T04:03:29.192245+0000","last_peered":"2024-05-10T04:03:29.192245+0000","last_clean":"2024-05-10T04:03:29.192245+0000","last_became_active":"2024-05-10T04:02:31.192514+0000","last_became_peered":"2024-05-10T04:02:31.192514+0000","last_unstale":"2024-05-10T04:03:29.192245+0000","last_undegraded":"2024-05-10T04:03:29.192245+0000","last_fullsized":"2024-05-10T04:03:29.192245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:50:32.649440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197690+0000","last_change":"2024-05-10T04:02:31.641578+0000","last_active":"2024-05-10T04:03:29.197690+0000","last_peered":"2024-05-10T04:03:29.197690+0000","last_clean":"2024-05-10T04:03:29.197690+0000","last_became_active":"2024-05-10T04:02:31.641268+0000","last_became_peered":"2024-05-10T04:02:31.641268+0000","last_unstale":"2024-05-10T04:03:29.197690+0000","last_undegraded":"2024-05-10T04:03:29.197690+0000","last_fullsized":"2024-05-10T04:03:29.197690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:31:00.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195504+0000","last_change":"2024-05-10T04:02:35.646754+0000","last_active":"2024-05-10T04:03:29.195504+0000","last_peered":"2024-05-10T04:03:29.195504+0000","last_clean":"2024-05-10T04:03:29.195504+0000","last_became_active":"2024-05-10T04:02:35.646342+0000","last_became_peered":"2024-05-10T04:02:35.646342+0000","last_unstale":"2024-05-10T04:03:29.195504+0000","last_undegraded":"2024-05-10T04:03:29.195504+0000","last_fullsized":"2024-05-10T04:03:29.195504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:28:33.373312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192990+0000","last_change":"2024-05-10T04:02:31.186427+0000","last_active":"2024-05-10T04:03:29.192990+0000","last_peered":"2024-05-10T04:03:29.192990+0000","last_clean":"2024-05-10T04:03:29.192990+0000","last_became_active":"2024-05-10T04:02:31.185937+0000","last_became_peered":"2024-05-10T04:02:31.185937+0000","last_unstale":"2024-05-10T04:03:29.192990+0000","last_undegraded":"2024-05-10T04:03:29.192990+0000","last_fullsized":"2024-05-10T04:03:29.192990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.818126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200293+0000","last_change":"2024-05-10T04:02:31.645582+0000","last_active":"2024-05-10T04:03:29.200293+0000","last_peered":"2024-05-10T04:03:29.200293+0000","last_clean":"2024-05-10T04:03:29.200293+0000","last_became_active":"2024-05-10T04:02:31.645259+0000","last_became_peered":"2024-05-10T04:02:31.645259+0000","last_unstale":"2024-05-10T04:03:29.200293+0000","last_undegraded":"2024-05-10T04:03:29.200293+0000","last_fullsized":"2024-05-10T04:03:29.200293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:17:31.818116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192983+0000","last_change":"2024-05-10T04:02:35.647341+0000","last_active":"2024-05-10T04:03:29.192983+0000","last_peered":"2024-05-10T04:03:29.192983+0000","last_clean":"2024-05-10T04:03:29.192983+0000","last_became_active":"2024-05-10T04:02:35.646806+0000","last_became_peered":"2024-05-10T04:02:35.646806+0000","last_unstale":"2024-05-10T04:03:29.192983+0000","last_undegraded":"2024-05-10T04:03:29.192983+0000","last_fullsized":"2024-05-10T04:03:29.192983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:01:51.252683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":193,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:04.298928+0000","last_change":"2024-05-10T04:02:31.195377+0000","last_active":"2024-05-10T04:04:04.298928+0000","last_peered":"2024-05-10T04:04:04.298928+0000","last_clean":"2024-05-10T04:04:04.298928+0000","last_became_active":"2024-05-10T04:02:31.194965+0000","last_became_peered":"2024-05-10T04:02:31.194965+0000","last_unstale":"2024-05-10T04:04:04.298928+0000","last_undegraded":"2024-05-10T04:04:04.298928+0000","last_fullsized":"2024-05-10T04:04:04.298928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:11:09.237525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62045,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":144,"num_read_kb":133,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198701+0000","last_change":"2024-05-10T04:02:31.629600+0000","last_active":"2024-05-10T04:03:29.198701+0000","last_peered":"2024-05-10T04:03:29.198701+0000","last_clean":"2024-05-10T04:03:29.198701+0000","last_became_active":"2024-05-10T04:02:31.629264+0000","last_became_peered":"2024-05-10T04:02:31.629264+0000","last_unstale":"2024-05-10T04:03:29.198701+0000","last_undegraded":"2024-05-10T04:03:29.198701+0000","last_fullsized":"2024-05-10T04:03:29.198701+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:45:08.867783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193519+0000","last_change":"2024-05-10T04:02:35.649468+0000","last_active":"2024-05-10T04:03:29.193519+0000","last_peered":"2024-05-10T04:03:29.193519+0000","last_clean":"2024-05-10T04:03:29.193519+0000","last_became_active":"2024-05-10T04:02:35.649053+0000","last_became_peered":"2024-05-10T04:02:35.649053+0000","last_unstale":"2024-05-10T04:03:29.193519+0000","last_undegraded":"2024-05-10T04:03:29.193519+0000","last_fullsized":"2024-05-10T04:03:29.193519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:01:05.761287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188394+0000","last_change":"2024-05-10T04:02:31.195169+0000","last_active":"2024-05-10T04:03:29.188394+0000","last_peered":"2024-05-10T04:03:29.188394+0000","last_clean":"2024-05-10T04:03:29.188394+0000","last_became_active":"2024-05-10T04:02:31.194720+0000","last_became_peered":"2024-05-10T04:02:31.194720+0000","last_unstale":"2024-05-10T04:03:29.188394+0000","last_undegraded":"2024-05-10T04:03:29.188394+0000","last_fullsized":"2024-05-10T04:03:29.188394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:42.727841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191480+0000","last_change":"2024-05-10T04:02:31.631739+0000","last_active":"2024-05-10T04:03:29.191480+0000","last_peered":"2024-05-10T04:03:29.191480+0000","last_clean":"2024-05-10T04:03:29.191480+0000","last_became_active":"2024-05-10T04:02:31.631401+0000","last_became_peered":"2024-05-10T04:02:31.631401+0000","last_unstale":"2024-05-10T04:03:29.191480+0000","last_undegraded":"2024-05-10T04:03:29.191480+0000","last_fullsized":"2024-05-10T04:03:29.191480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:06.958328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196041+0000","last_change":"2024-05-10T04:02:35.651484+0000","last_active":"2024-05-10T04:03:29.196041+0000","last_peered":"2024-05-10T04:03:29.196041+0000","last_clean":"2024-05-10T04:03:29.196041+0000","last_became_active":"2024-05-10T04:02:35.651069+0000","last_became_peered":"2024-05-10T04:02:35.651069+0000","last_unstale":"2024-05-10T04:03:29.196041+0000","last_undegraded":"2024-05-10T04:03:29.196041+0000","last_fullsized":"2024-05-10T04:03:29.196041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:26:45.229101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192929+0000","last_change":"2024-05-10T04:02:31.186314+0000","last_active":"2024-05-10T04:03:29.192929+0000","last_peered":"2024-05-10T04:03:29.192929+0000","last_clean":"2024-05-10T04:03:29.192929+0000","last_became_active":"2024-05-10T04:02:31.185861+0000","last_became_peered":"2024-05-10T04:02:31.185861+0000","last_unstale":"2024-05-10T04:03:29.192929+0000","last_undegraded":"2024-05-10T04:03:29.192929+0000","last_fullsized":"2024-05-10T04:03:29.192929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:07.386978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191954+0000","last_change":"2024-05-10T04:02:31.637151+0000","last_active":"2024-05-10T04:03:29.191954+0000","last_peered":"2024-05-10T04:03:29.191954+0000","last_clean":"2024-05-10T04:03:29.191954+0000","last_became_active":"2024-05-10T04:02:31.636745+0000","last_became_peered":"2024-05-10T04:02:31.636745+0000","last_unstale":"2024-05-10T04:03:29.191954+0000","last_undegraded":"2024-05-10T04:03:29.191954+0000","last_fullsized":"2024-05-10T04:03:29.191954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:59.804391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192369+0000","last_change":"2024-05-10T04:02:35.656108+0000","last_active":"2024-05-10T04:03:29.192369+0000","last_peered":"2024-05-10T04:03:29.192369+0000","last_clean":"2024-05-10T04:03:29.192369+0000","last_became_active":"2024-05-10T04:02:35.655664+0000","last_became_peered":"2024-05-10T04:02:35.655664+0000","last_unstale":"2024-05-10T04:03:29.192369+0000","last_undegraded":"2024-05-10T04:03:29.192369+0000","last_fullsized":"2024-05-10T04:03:29.192369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:10.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:07.303591+0000","last_change":"2024-05-10T04:02:31.189432+0000","last_active":"2024-05-10T04:04:07.303591+0000","last_peered":"2024-05-10T04:04:07.303591+0000","last_clean":"2024-05-10T04:04:07.303591+0000","last_became_active":"2024-05-10T04:02:31.189062+0000","last_became_peered":"2024-05-10T04:02:31.189062+0000","last_unstale":"2024-05-10T04:04:07.303591+0000","last_undegraded":"2024-05-10T04:04:07.303591+0000","last_fullsized":"2024-05-10T04:04:07.303591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:28.452100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191766+0000","last_change":"2024-05-10T04:02:31.637517+0000","last_active":"2024-05-10T04:03:29.191766+0000","last_peered":"2024-05-10T04:03:29.191766+0000","last_clean":"2024-05-10T04:03:29.191766+0000","last_became_active":"2024-05-10T04:02:31.636518+0000","last_became_peered":"2024-05-10T04:02:31.636518+0000","last_unstale":"2024-05-10T04:03:29.191766+0000","last_undegraded":"2024-05-10T04:03:29.191766+0000","last_fullsized":"2024-05-10T04:03:29.191766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:07:09.732117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192266+0000","last_change":"2024-05-10T04:02:35.649693+0000","last_active":"2024-05-10T04:03:29.192266+0000","last_peered":"2024-05-10T04:03:29.192266+0000","last_clean":"2024-05-10T04:03:29.192266+0000","last_became_active":"2024-05-10T04:02:35.649175+0000","last_became_peered":"2024-05-10T04:02:35.649175+0000","last_unstale":"2024-05-10T04:03:29.192266+0000","last_undegraded":"2024-05-10T04:03:29.192266+0000","last_fullsized":"2024-05-10T04:03:29.192266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:03:52.340444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.211403+0000","last_change":"2024-05-10T04:03:30.211403+0000","last_active":"2024-05-10T04:03:30.211403+0000","last_peered":"2024-05-10T04:03:30.211403+0000","last_clean":"2024-05-10T04:03:30.211403+0000","last_became_active":"2024-05-10T04:03:30.198739+0000","last_became_peered":"2024-05-10T04:03:30.198739+0000","last_unstale":"2024-05-10T04:03:30.211403+0000","last_undegraded":"2024-05-10T04:03:30.211403+0000","last_fullsized":"2024-05-10T04:03:30.211403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:52.430884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194588+0000","last_change":"2024-05-10T04:02:31.641426+0000","last_active":"2024-05-10T04:03:29.194588+0000","last_peered":"2024-05-10T04:03:29.194588+0000","last_clean":"2024-05-10T04:03:29.194588+0000","last_became_active":"2024-05-10T04:02:31.641054+0000","last_became_peered":"2024-05-10T04:02:31.641054+0000","last_unstale":"2024-05-10T04:03:29.194588+0000","last_undegraded":"2024-05-10T04:03:29.194588+0000","last_fullsized":"2024-05-10T04:03:29.194588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:12:48.472674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194503+0000","last_change":"2024-05-10T04:02:35.653258+0000","last_active":"2024-05-10T04:03:29.194503+0000","last_peered":"2024-05-10T04:03:29.194503+0000","last_clean":"2024-05-10T04:03:29.194503+0000","last_became_active":"2024-05-10T04:02:35.652859+0000","last_became_peered":"2024-05-10T04:02:35.652859+0000","last_unstale":"2024-05-10T04:03:29.194503+0000","last_undegraded":"2024-05-10T04:03:29.194503+0000","last_fullsized":"2024-05-10T04:03:29.194503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:15:57.814963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198019+0000","last_change":"2024-05-10T04:02:31.192231+0000","last_active":"2024-05-10T04:03:29.198019+0000","last_peered":"2024-05-10T04:03:29.198019+0000","last_clean":"2024-05-10T04:03:29.198019+0000","last_became_active":"2024-05-10T04:02:31.191902+0000","last_became_peered":"2024-05-10T04:02:31.191902+0000","last_unstale":"2024-05-10T04:03:29.198019+0000","last_undegraded":"2024-05-10T04:03:29.198019+0000","last_fullsized":"2024-05-10T04:03:29.198019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:18.637112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195378+0000","last_change":"2024-05-10T04:02:31.635514+0000","last_active":"2024-05-10T04:03:29.195378+0000","last_peered":"2024-05-10T04:03:29.195378+0000","last_clean":"2024-05-10T04:03:29.195378+0000","last_became_active":"2024-05-10T04:02:31.635171+0000","last_became_peered":"2024-05-10T04:02:31.635171+0000","last_unstale":"2024-05-10T04:03:29.195378+0000","last_undegraded":"2024-05-10T04:03:29.195378+0000","last_fullsized":"2024-05-10T04:03:29.195378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:48.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.475636+0000","last_change":"2024-05-10T04:03:30.475636+0000","last_active":"2024-05-10T04:03:30.475636+0000","last_peered":"2024-05-10T04:03:30.475636+0000","last_clean":"2024-05-10T04:03:30.475636+0000","last_became_active":"2024-05-10T04:03:30.475203+0000","last_became_peered":"2024-05-10T04:03:30.475203+0000","last_unstale":"2024-05-10T04:03:30.475636+0000","last_undegraded":"2024-05-10T04:03:30.475636+0000","last_fullsized":"2024-05-10T04:03:30.475636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:56:07.461905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194614+0000","last_change":"2024-05-10T04:02:31.190173+0000","last_active":"2024-05-10T04:03:29.194614+0000","last_peered":"2024-05-10T04:03:29.194614+0000","last_clean":"2024-05-10T04:03:29.194614+0000","last_became_active":"2024-05-10T04:02:31.189840+0000","last_became_peered":"2024-05-10T04:02:31.189840+0000","last_unstale":"2024-05-10T04:03:29.194614+0000","last_undegraded":"2024-05-10T04:03:29.194614+0000","last_fullsized":"2024-05-10T04:03:29.194614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:31:49.382482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199097+0000","last_change":"2024-05-10T04:02:31.629497+0000","last_active":"2024-05-10T04:03:29.199097+0000","last_peered":"2024-05-10T04:03:29.199097+0000","last_clean":"2024-05-10T04:03:29.199097+0000","last_became_active":"2024-05-10T04:02:31.629182+0000","last_became_peered":"2024-05-10T04:02:31.629182+0000","last_unstale":"2024-05-10T04:03:29.199097+0000","last_undegraded":"2024-05-10T04:03:29.199097+0000","last_fullsized":"2024-05-10T04:03:29.199097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:11:33.478737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192656+0000","last_change":"2024-05-10T04:02:35.647249+0000","last_active":"2024-05-10T04:03:29.192656+0000","last_peered":"2024-05-10T04:03:29.192656+0000","last_clean":"2024-05-10T04:03:29.192656+0000","last_became_active":"2024-05-10T04:02:35.646716+0000","last_became_peered":"2024-05-10T04:02:35.646716+0000","last_unstale":"2024-05-10T04:03:29.192656+0000","last_undegraded":"2024-05-10T04:03:29.192656+0000","last_fullsized":"2024-05-10T04:03:29.192656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:44.651836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191715+0000","last_change":"2024-05-10T04:02:31.191241+0000","last_active":"2024-05-10T04:03:29.191715+0000","last_peered":"2024-05-10T04:03:29.191715+0000","last_clean":"2024-05-10T04:03:29.191715+0000","last_became_active":"2024-05-10T04:02:31.190888+0000","last_became_peered":"2024-05-10T04:02:31.190888+0000","last_unstale":"2024-05-10T04:03:29.191715+0000","last_undegraded":"2024-05-10T04:03:29.191715+0000","last_fullsized":"2024-05-10T04:03:29.191715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:55.256119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198888+0000","last_change":"2024-05-10T04:02:31.638394+0000","last_active":"2024-05-10T04:03:29.198888+0000","last_peered":"2024-05-10T04:03:29.198888+0000","last_clean":"2024-05-10T04:03:29.198888+0000","last_became_active":"2024-05-10T04:02:31.638025+0000","last_became_peered":"2024-05-10T04:02:31.638025+0000","last_unstale":"2024-05-10T04:03:29.198888+0000","last_undegraded":"2024-05-10T04:03:29.198888+0000","last_fullsized":"2024-05-10T04:03:29.198888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:13.088393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192346+0000","last_change":"2024-05-10T04:02:35.899898+0000","last_active":"2024-05-10T04:03:29.192346+0000","last_peered":"2024-05-10T04:03:29.192346+0000","last_clean":"2024-05-10T04:03:29.192346+0000","last_became_active":"2024-05-10T04:02:35.899485+0000","last_became_peered":"2024-05-10T04:02:35.899485+0000","last_unstale":"2024-05-10T04:03:29.192346+0000","last_undegraded":"2024-05-10T04:03:29.192346+0000","last_fullsized":"2024-05-10T04:03:29.192346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:26:50.909122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193219+0000","last_change":"2024-05-10T04:02:31.640923+0000","last_active":"2024-05-10T04:03:29.193219+0000","last_peered":"2024-05-10T04:03:29.193219+0000","last_clean":"2024-05-10T04:03:29.193219+0000","last_became_active":"2024-05-10T04:02:31.640490+0000","last_became_peered":"2024-05-10T04:02:31.640490+0000","last_unstale":"2024-05-10T04:03:29.193219+0000","last_undegraded":"2024-05-10T04:03:29.193219+0000","last_fullsized":"2024-05-10T04:03:29.193219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:31:58.585789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:04:02.499077+0000","last_change":"2024-05-10T04:02:31.191326+0000","last_active":"2024-05-10T04:04:02.499077+0000","last_peered":"2024-05-10T04:04:02.499077+0000","last_clean":"2024-05-10T04:04:02.499077+0000","last_became_active":"2024-05-10T04:02:31.190974+0000","last_became_peered":"2024-05-10T04:02:31.190974+0000","last_unstale":"2024-05-10T04:04:02.499077+0000","last_undegraded":"2024-05-10T04:04:02.499077+0000","last_fullsized":"2024-05-10T04:04:02.499077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:43.670619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192264+0000","last_change":"2024-05-10T04:02:35.647526+0000","last_active":"2024-05-10T04:03:29.192264+0000","last_peered":"2024-05-10T04:03:29.192264+0000","last_clean":"2024-05-10T04:03:29.192264+0000","last_became_active":"2024-05-10T04:02:35.647018+0000","last_became_peered":"2024-05-10T04:02:35.647018+0000","last_unstale":"2024-05-10T04:03:29.192264+0000","last_undegraded":"2024-05-10T04:03:29.192264+0000","last_fullsized":"2024-05-10T04:03:29.192264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:14:17.173497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199043+0000","last_change":"2024-05-10T04:02:31.632263+0000","last_active":"2024-05-10T04:03:29.199043+0000","last_peered":"2024-05-10T04:03:29.199043+0000","last_clean":"2024-05-10T04:03:29.199043+0000","last_became_active":"2024-05-10T04:02:31.631785+0000","last_became_peered":"2024-05-10T04:02:31.631785+0000","last_unstale":"2024-05-10T04:03:29.199043+0000","last_undegraded":"2024-05-10T04:03:29.199043+0000","last_fullsized":"2024-05-10T04:03:29.199043+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:30.585862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195400+0000","last_change":"2024-05-10T04:02:31.182262+0000","last_active":"2024-05-10T04:03:29.195400+0000","last_peered":"2024-05-10T04:03:29.195400+0000","last_clean":"2024-05-10T04:03:29.195400+0000","last_became_active":"2024-05-10T04:02:31.181802+0000","last_became_peered":"2024-05-10T04:02:31.181802+0000","last_unstale":"2024-05-10T04:03:29.195400+0000","last_undegraded":"2024-05-10T04:03:29.195400+0000","last_fullsized":"2024-05-10T04:03:29.195400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:45:40.696571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":327,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191496+0000","last_change":"2024-05-10T04:01:41.866365+0000","last_active":"2024-05-10T04:03:29.191496+0000","last_peered":"2024-05-10T04:03:29.191496+0000","last_clean":"2024-05-10T04:03:29.191496+0000","last_became_active":"2024-05-10T04:01:41.865825+0000","last_became_peered":"2024-05-10T04:01:41.865825+0000","last_unstale":"2024-05-10T04:03:29.191496+0000","last_undegraded":"2024-05-10T04:03:29.191496+0000","last_fullsized":"2024-05-10T04:03:29.191496+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:59:19.853473+0000","last_clean_scrub_stamp":"2024-05-10T03:59:19.853473+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:28:29.598339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193669+0000","last_change":"2024-05-10T04:02:35.900841+0000","last_active":"2024-05-10T04:03:29.193669+0000","last_peered":"2024-05-10T04:03:29.193669+0000","last_clean":"2024-05-10T04:03:29.193669+0000","last_became_active":"2024-05-10T04:02:35.900323+0000","last_became_peered":"2024-05-10T04:02:35.900323+0000","last_unstale":"2024-05-10T04:03:29.193669+0000","last_undegraded":"2024-05-10T04:03:29.193669+0000","last_fullsized":"2024-05-10T04:03:29.193669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:00.554804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.196577+0000","last_change":"2024-05-10T04:03:30.196577+0000","last_active":"2024-05-10T04:03:30.196577+0000","last_peered":"2024-05-10T04:03:30.196577+0000","last_clean":"2024-05-10T04:03:30.196577+0000","last_became_active":"2024-05-10T04:03:30.196249+0000","last_became_peered":"2024-05-10T04:03:30.196249+0000","last_unstale":"2024-05-10T04:03:30.196577+0000","last_undegraded":"2024-05-10T04:03:30.196577+0000","last_fullsized":"2024-05-10T04:03:30.196577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:48:38.347871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191634+0000","last_change":"2024-05-10T04:02:31.187834+0000","last_active":"2024-05-10T04:03:29.191634+0000","last_peered":"2024-05-10T04:03:29.191634+0000","last_clean":"2024-05-10T04:03:29.191634+0000","last_became_active":"2024-05-10T04:02:31.187440+0000","last_became_peered":"2024-05-10T04:02:31.187440+0000","last_unstale":"2024-05-10T04:03:29.191634+0000","last_undegraded":"2024-05-10T04:03:29.191634+0000","last_fullsized":"2024-05-10T04:03:29.191634+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:50:37.678245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194942+0000","last_change":"2024-05-10T04:02:35.649664+0000","last_active":"2024-05-10T04:03:29.194942+0000","last_peered":"2024-05-10T04:03:29.194942+0000","last_clean":"2024-05-10T04:03:29.194942+0000","last_became_active":"2024-05-10T04:02:35.649076+0000","last_became_peered":"2024-05-10T04:02:35.649076+0000","last_unstale":"2024-05-10T04:03:29.194942+0000","last_undegraded":"2024-05-10T04:03:29.194942+0000","last_fullsized":"2024-05-10T04:03:29.194942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:14:42.864845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198666+0000","last_change":"2024-05-10T04:02:31.192126+0000","last_active":"2024-05-10T04:03:29.198666+0000","last_peered":"2024-05-10T04:03:29.198666+0000","last_clean":"2024-05-10T04:03:29.198666+0000","last_became_active":"2024-05-10T04:02:31.191771+0000","last_became_peered":"2024-05-10T04:02:31.191771+0000","last_unstale":"2024-05-10T04:03:29.198666+0000","last_undegraded":"2024-05-10T04:03:29.198666+0000","last_fullsized":"2024-05-10T04:03:29.198666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:43.550535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192799+0000","last_change":"2024-05-10T04:02:31.620788+0000","last_active":"2024-05-10T04:03:29.192799+0000","last_peered":"2024-05-10T04:03:29.192799+0000","last_clean":"2024-05-10T04:03:29.192799+0000","last_became_active":"2024-05-10T04:02:31.620456+0000","last_became_peered":"2024-05-10T04:02:31.620456+0000","last_unstale":"2024-05-10T04:03:29.192799+0000","last_undegraded":"2024-05-10T04:03:29.192799+0000","last_fullsized":"2024-05-10T04:03:29.192799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:02.956030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198321+0000","last_change":"2024-05-10T04:02:35.899935+0000","last_active":"2024-05-10T04:03:29.198321+0000","last_peered":"2024-05-10T04:03:29.198321+0000","last_clean":"2024-05-10T04:03:29.198321+0000","last_became_active":"2024-05-10T04:02:35.899488+0000","last_became_peered":"2024-05-10T04:02:35.899488+0000","last_unstale":"2024-05-10T04:03:29.198321+0000","last_undegraded":"2024-05-10T04:03:29.198321+0000","last_fullsized":"2024-05-10T04:03:29.198321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:28.600463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195179+0000","last_change":"2024-05-10T04:02:31.188605+0000","last_active":"2024-05-10T04:03:29.195179+0000","last_peered":"2024-05-10T04:03:29.195179+0000","last_clean":"2024-05-10T04:03:29.195179+0000","last_became_active":"2024-05-10T04:02:31.188283+0000","last_became_peered":"2024-05-10T04:02:31.188283+0000","last_unstale":"2024-05-10T04:03:29.195179+0000","last_undegraded":"2024-05-10T04:03:29.195179+0000","last_fullsized":"2024-05-10T04:03:29.195179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:40:03.940269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199712+0000","last_change":"2024-05-10T04:02:31.629476+0000","last_active":"2024-05-10T04:03:29.199712+0000","last_peered":"2024-05-10T04:03:29.199712+0000","last_clean":"2024-05-10T04:03:29.199712+0000","last_became_active":"2024-05-10T04:02:31.629104+0000","last_became_peered":"2024-05-10T04:02:31.629104+0000","last_unstale":"2024-05-10T04:03:29.199712+0000","last_undegraded":"2024-05-10T04:03:29.199712+0000","last_fullsized":"2024-05-10T04:03:29.199712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:57.873755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188055+0000","last_change":"2024-05-10T04:02:35.644094+0000","last_active":"2024-05-10T04:03:29.188055+0000","last_peered":"2024-05-10T04:03:29.188055+0000","last_clean":"2024-05-10T04:03:29.188055+0000","last_became_active":"2024-05-10T04:02:35.643382+0000","last_became_peered":"2024-05-10T04:02:35.643382+0000","last_unstale":"2024-05-10T04:03:29.188055+0000","last_undegraded":"2024-05-10T04:03:29.188055+0000","last_fullsized":"2024-05-10T04:03:29.188055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:33.250052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195430+0000","last_change":"2024-05-10T04:02:31.189302+0000","last_active":"2024-05-10T04:03:29.195430+0000","last_peered":"2024-05-10T04:03:29.195430+0000","last_clean":"2024-05-10T04:03:29.195430+0000","last_became_active":"2024-05-10T04:02:31.188323+0000","last_became_peered":"2024-05-10T04:02:31.188323+0000","last_unstale":"2024-05-10T04:03:29.195430+0000","last_undegraded":"2024-05-10T04:03:29.195430+0000","last_fullsized":"2024-05-10T04:03:29.195430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:11:02.922884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.197540+0000","last_change":"2024-05-10T04:02:31.635864+0000","last_active":"2024-05-10T04:03:29.197540+0000","last_peered":"2024-05-10T04:03:29.197540+0000","last_clean":"2024-05-10T04:03:29.197540+0000","last_became_active":"2024-05-10T04:02:31.635512+0000","last_became_peered":"2024-05-10T04:02:31.635512+0000","last_unstale":"2024-05-10T04:03:29.197540+0000","last_undegraded":"2024-05-10T04:03:29.197540+0000","last_fullsized":"2024-05-10T04:03:29.197540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:26:10.957848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194647+0000","last_change":"2024-05-10T04:02:35.900489+0000","last_active":"2024-05-10T04:03:29.194647+0000","last_peered":"2024-05-10T04:03:29.194647+0000","last_clean":"2024-05-10T04:03:29.194647+0000","last_became_active":"2024-05-10T04:02:35.899908+0000","last_became_peered":"2024-05-10T04:02:35.899908+0000","last_unstale":"2024-05-10T04:03:29.194647+0000","last_undegraded":"2024-05-10T04:03:29.194647+0000","last_fullsized":"2024-05-10T04:03:29.194647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:08.386128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199847+0000","last_change":"2024-05-10T04:02:31.193956+0000","last_active":"2024-05-10T04:03:29.199847+0000","last_peered":"2024-05-10T04:03:29.199847+0000","last_clean":"2024-05-10T04:03:29.199847+0000","last_became_active":"2024-05-10T04:02:31.193574+0000","last_became_peered":"2024-05-10T04:02:31.193574+0000","last_unstale":"2024-05-10T04:03:29.199847+0000","last_undegraded":"2024-05-10T04:03:29.199847+0000","last_fullsized":"2024-05-10T04:03:29.199847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:46.369152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195487+0000","last_change":"2024-05-10T04:02:31.633263+0000","last_active":"2024-05-10T04:03:29.195487+0000","last_peered":"2024-05-10T04:03:29.195487+0000","last_clean":"2024-05-10T04:03:29.195487+0000","last_became_active":"2024-05-10T04:02:31.632903+0000","last_became_peered":"2024-05-10T04:02:31.632903+0000","last_unstale":"2024-05-10T04:03:29.195487+0000","last_undegraded":"2024-05-10T04:03:29.195487+0000","last_fullsized":"2024-05-10T04:03:29.195487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:28.047473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192521+0000","last_change":"2024-05-10T04:02:35.656003+0000","last_active":"2024-05-10T04:03:29.192521+0000","last_peered":"2024-05-10T04:03:29.192521+0000","last_clean":"2024-05-10T04:03:29.192521+0000","last_became_active":"2024-05-10T04:02:35.655518+0000","last_became_peered":"2024-05-10T04:02:35.655518+0000","last_unstale":"2024-05-10T04:03:29.192521+0000","last_undegraded":"2024-05-10T04:03:29.192521+0000","last_fullsized":"2024-05-10T04:03:29.192521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:49.696699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198559+0000","last_change":"2024-05-10T04:02:31.195297+0000","last_active":"2024-05-10T04:03:29.198559+0000","last_peered":"2024-05-10T04:03:29.198559+0000","last_clean":"2024-05-10T04:03:29.198559+0000","last_became_active":"2024-05-10T04:02:31.193960+0000","last_became_peered":"2024-05-10T04:02:31.193960+0000","last_unstale":"2024-05-10T04:03:29.198559+0000","last_undegraded":"2024-05-10T04:03:29.198559+0000","last_fullsized":"2024-05-10T04:03:29.198559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:04:10.314700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193216+0000","last_change":"2024-05-10T04:02:31.629247+0000","last_active":"2024-05-10T04:03:29.193216+0000","last_peered":"2024-05-10T04:03:29.193216+0000","last_clean":"2024-05-10T04:03:29.193216+0000","last_became_active":"2024-05-10T04:02:31.628838+0000","last_became_peered":"2024-05-10T04:02:31.628838+0000","last_unstale":"2024-05-10T04:03:29.193216+0000","last_undegraded":"2024-05-10T04:03:29.193216+0000","last_fullsized":"2024-05-10T04:03:29.193216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:46:39.483264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.252785+0000","last_change":"2024-05-10T04:03:30.252785+0000","last_active":"2024-05-10T04:03:30.252785+0000","last_peered":"2024-05-10T04:03:30.252785+0000","last_clean":"2024-05-10T04:03:30.252785+0000","last_became_active":"2024-05-10T04:03:30.248696+0000","last_became_peered":"2024-05-10T04:03:30.248696+0000","last_unstale":"2024-05-10T04:03:30.252785+0000","last_undegraded":"2024-05-10T04:03:30.252785+0000","last_fullsized":"2024-05-10T04:03:30.252785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:55:15.159906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:03:30.202901+0000","last_change":"2024-05-10T04:03:30.202456+0000","last_active":"2024-05-10T04:03:30.202901+0000","last_peered":"2024-05-10T04:03:30.202901+0000","last_clean":"2024-05-10T04:03:30.202901+0000","last_became_active":"2024-05-10T04:03:30.197484+0000","last_became_peered":"2024-05-10T04:03:30.197484+0000","last_unstale":"2024-05-10T04:03:30.202901+0000","last_undegraded":"2024-05-10T04:03:30.202901+0000","last_fullsized":"2024-05-10T04:03:30.202901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:15:07.899758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198950+0000","last_change":"2024-05-10T04:02:31.628029+0000","last_active":"2024-05-10T04:03:29.198950+0000","last_peered":"2024-05-10T04:03:29.198950+0000","last_clean":"2024-05-10T04:03:29.198950+0000","last_became_active":"2024-05-10T04:02:31.627667+0000","last_became_peered":"2024-05-10T04:02:31.627667+0000","last_unstale":"2024-05-10T04:03:29.198950+0000","last_undegraded":"2024-05-10T04:03:29.198950+0000","last_fullsized":"2024-05-10T04:03:29.198950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:39:10.878894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.194397+0000","last_change":"2024-05-10T04:02:35.900237+0000","last_active":"2024-05-10T04:03:29.194397+0000","last_peered":"2024-05-10T04:03:29.194397+0000","last_clean":"2024-05-10T04:03:29.194397+0000","last_became_active":"2024-05-10T04:02:35.899781+0000","last_became_peered":"2024-05-10T04:02:35.899781+0000","last_unstale":"2024-05-10T04:03:29.194397+0000","last_undegraded":"2024-05-10T04:03:29.194397+0000","last_fullsized":"2024-05-10T04:03:29.194397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:52:46.132557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196543+0000","last_change":"2024-05-10T04:02:31.196920+0000","last_active":"2024-05-10T04:03:29.196543+0000","last_peered":"2024-05-10T04:03:29.196543+0000","last_clean":"2024-05-10T04:03:29.196543+0000","last_became_active":"2024-05-10T04:02:31.196191+0000","last_became_peered":"2024-05-10T04:02:31.196191+0000","last_unstale":"2024-05-10T04:03:29.196543+0000","last_undegraded":"2024-05-10T04:03:29.196543+0000","last_fullsized":"2024-05-10T04:03:29.196543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:45:17.980149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191841+0000","last_change":"2024-05-10T04:02:31.623678+0000","last_active":"2024-05-10T04:03:29.191841+0000","last_peered":"2024-05-10T04:03:29.191841+0000","last_clean":"2024-05-10T04:03:29.191841+0000","last_became_active":"2024-05-10T04:02:31.623090+0000","last_became_peered":"2024-05-10T04:02:31.623090+0000","last_unstale":"2024-05-10T04:03:29.191841+0000","last_undegraded":"2024-05-10T04:03:29.191841+0000","last_fullsized":"2024-05-10T04:03:29.191841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:34.321959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199997+0000","last_change":"2024-05-10T04:02:35.648833+0000","last_active":"2024-05-10T04:03:29.199997+0000","last_peered":"2024-05-10T04:03:29.199997+0000","last_clean":"2024-05-10T04:03:29.199997+0000","last_became_active":"2024-05-10T04:02:35.648391+0000","last_became_peered":"2024-05-10T04:02:35.648391+0000","last_unstale":"2024-05-10T04:03:29.199997+0000","last_undegraded":"2024-05-10T04:03:29.199997+0000","last_fullsized":"2024-05-10T04:03:29.199997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:12.773284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.189688+0000","last_change":"2024-05-10T04:02:31.192900+0000","last_active":"2024-05-10T04:03:29.189688+0000","last_peered":"2024-05-10T04:03:29.189688+0000","last_clean":"2024-05-10T04:03:29.189688+0000","last_became_active":"2024-05-10T04:02:31.192481+0000","last_became_peered":"2024-05-10T04:02:31.192481+0000","last_unstale":"2024-05-10T04:03:29.189688+0000","last_undegraded":"2024-05-10T04:03:29.189688+0000","last_fullsized":"2024-05-10T04:03:29.189688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:17.202383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192511+0000","last_change":"2024-05-10T04:02:31.628490+0000","last_active":"2024-05-10T04:03:29.192511+0000","last_peered":"2024-05-10T04:03:29.192511+0000","last_clean":"2024-05-10T04:03:29.192511+0000","last_became_active":"2024-05-10T04:02:31.627889+0000","last_became_peered":"2024-05-10T04:02:31.627889+0000","last_unstale":"2024-05-10T04:03:29.192511+0000","last_undegraded":"2024-05-10T04:03:29.192511+0000","last_fullsized":"2024-05-10T04:03:29.192511+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:32:55.366759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196442+0000","last_change":"2024-05-10T04:02:35.653530+0000","last_active":"2024-05-10T04:03:29.196442+0000","last_peered":"2024-05-10T04:03:29.196442+0000","last_clean":"2024-05-10T04:03:29.196442+0000","last_became_active":"2024-05-10T04:02:35.653095+0000","last_became_peered":"2024-05-10T04:02:35.653095+0000","last_unstale":"2024-05-10T04:03:29.196442+0000","last_undegraded":"2024-05-10T04:03:29.196442+0000","last_fullsized":"2024-05-10T04:03:29.196442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:09.921713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193200+0000","last_change":"2024-05-10T04:02:31.189434+0000","last_active":"2024-05-10T04:03:29.193200+0000","last_peered":"2024-05-10T04:03:29.193200+0000","last_clean":"2024-05-10T04:03:29.193200+0000","last_became_active":"2024-05-10T04:02:31.188843+0000","last_became_peered":"2024-05-10T04:02:31.188843+0000","last_unstale":"2024-05-10T04:03:29.193200+0000","last_undegraded":"2024-05-10T04:03:29.193200+0000","last_fullsized":"2024-05-10T04:03:29.193200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:31.508794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191797+0000","last_change":"2024-05-10T04:02:31.636775+0000","last_active":"2024-05-10T04:03:29.191797+0000","last_peered":"2024-05-10T04:03:29.191797+0000","last_clean":"2024-05-10T04:03:29.191797+0000","last_became_active":"2024-05-10T04:02:31.635946+0000","last_became_peered":"2024-05-10T04:02:31.635946+0000","last_unstale":"2024-05-10T04:03:29.191797+0000","last_undegraded":"2024-05-10T04:03:29.191797+0000","last_fullsized":"2024-05-10T04:03:29.191797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:48.830097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192629+0000","last_change":"2024-05-10T04:02:35.646991+0000","last_active":"2024-05-10T04:03:29.192629+0000","last_peered":"2024-05-10T04:03:29.192629+0000","last_clean":"2024-05-10T04:03:29.192629+0000","last_became_active":"2024-05-10T04:02:35.646585+0000","last_became_peered":"2024-05-10T04:02:35.646585+0000","last_unstale":"2024-05-10T04:03:29.192629+0000","last_undegraded":"2024-05-10T04:03:29.192629+0000","last_fullsized":"2024-05-10T04:03:29.192629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:34.619113+0000","last_clean_scrub_stamp":"2024-05-10T04:02:34.619113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:32:50.329746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192244+0000","last_change":"2024-05-10T04:02:31.192103+0000","last_active":"2024-05-10T04:03:29.192244+0000","last_peered":"2024-05-10T04:03:29.192244+0000","last_clean":"2024-05-10T04:03:29.192244+0000","last_became_active":"2024-05-10T04:02:31.191378+0000","last_became_peered":"2024-05-10T04:02:31.191378+0000","last_unstale":"2024-05-10T04:03:29.192244+0000","last_undegraded":"2024-05-10T04:03:29.192244+0000","last_fullsized":"2024-05-10T04:03:29.192244+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:30.159958+0000","last_clean_scrub_stamp":"2024-05-10T04:02:30.159958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:55.298158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192829+0000","last_change":"2024-05-10T04:02:31.628587+0000","last_active":"2024-05-10T04:03:29.192829+0000","last_peered":"2024-05-10T04:03:29.192829+0000","last_clean":"2024-05-10T04:03:29.192829+0000","last_became_active":"2024-05-10T04:02:31.627296+0000","last_became_peered":"2024-05-10T04:02:31.627296+0000","last_unstale":"2024-05-10T04:03:29.192829+0000","last_undegraded":"2024-05-10T04:03:29.192829+0000","last_fullsized":"2024-05-10T04:03:29.192829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:04:09.783266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193327+0000","last_change":"2024-05-10T04:02:31.634668+0000","last_active":"2024-05-10T04:03:29.193327+0000","last_peered":"2024-05-10T04:03:29.193327+0000","last_clean":"2024-05-10T04:03:29.193327+0000","last_became_active":"2024-05-10T04:02:31.634321+0000","last_became_peered":"2024-05-10T04:02:31.634321+0000","last_unstale":"2024-05-10T04:03:29.193327+0000","last_undegraded":"2024-05-10T04:03:29.193327+0000","last_fullsized":"2024-05-10T04:03:29.193327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:16:44.833804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188283+0000","last_change":"2024-05-10T04:02:31.646002+0000","last_active":"2024-05-10T04:03:29.188283+0000","last_peered":"2024-05-10T04:03:29.188283+0000","last_clean":"2024-05-10T04:03:29.188283+0000","last_became_active":"2024-05-10T04:02:31.645673+0000","last_became_peered":"2024-05-10T04:02:31.645673+0000","last_unstale":"2024-05-10T04:03:29.188283+0000","last_undegraded":"2024-05-10T04:03:29.188283+0000","last_fullsized":"2024-05-10T04:03:29.188283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:26:26.814523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.188124+0000","last_change":"2024-05-10T04:02:31.630606+0000","last_active":"2024-05-10T04:03:29.188124+0000","last_peered":"2024-05-10T04:03:29.188124+0000","last_clean":"2024-05-10T04:03:29.188124+0000","last_became_active":"2024-05-10T04:02:31.630313+0000","last_became_peered":"2024-05-10T04:02:31.630313+0000","last_unstale":"2024-05-10T04:03:29.188124+0000","last_undegraded":"2024-05-10T04:03:29.188124+0000","last_fullsized":"2024-05-10T04:03:29.188124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:51:39.876016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196756+0000","last_change":"2024-05-10T04:02:31.636239+0000","last_active":"2024-05-10T04:03:29.196756+0000","last_peered":"2024-05-10T04:03:29.196756+0000","last_clean":"2024-05-10T04:03:29.196756+0000","last_became_active":"2024-05-10T04:02:31.635377+0000","last_became_peered":"2024-05-10T04:02:31.635377+0000","last_unstale":"2024-05-10T04:03:29.196756+0000","last_undegraded":"2024-05-10T04:03:29.196756+0000","last_fullsized":"2024-05-10T04:03:29.196756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:02.612285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.192883+0000","last_change":"2024-05-10T04:02:31.643231+0000","last_active":"2024-05-10T04:03:29.192883+0000","last_peered":"2024-05-10T04:03:29.192883+0000","last_clean":"2024-05-10T04:03:29.192883+0000","last_became_active":"2024-05-10T04:02:31.642797+0000","last_became_peered":"2024-05-10T04:02:31.642797+0000","last_unstale":"2024-05-10T04:03:29.192883+0000","last_undegraded":"2024-05-10T04:03:29.192883+0000","last_fullsized":"2024-05-10T04:03:29.192883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:01:57.365339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.196100+0000","last_change":"2024-05-10T04:02:31.633928+0000","last_active":"2024-05-10T04:03:29.196100+0000","last_peered":"2024-05-10T04:03:29.196100+0000","last_clean":"2024-05-10T04:03:29.196100+0000","last_became_active":"2024-05-10T04:02:31.633423+0000","last_became_peered":"2024-05-10T04:02:31.633423+0000","last_unstale":"2024-05-10T04:03:29.196100+0000","last_undegraded":"2024-05-10T04:03:29.196100+0000","last_fullsized":"2024-05-10T04:03:29.196100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:07.011354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.189591+0000","last_change":"2024-05-10T04:02:31.627151+0000","last_active":"2024-05-10T04:03:29.189591+0000","last_peered":"2024-05-10T04:03:29.189591+0000","last_clean":"2024-05-10T04:03:29.189591+0000","last_became_active":"2024-05-10T04:02:31.626780+0000","last_became_peered":"2024-05-10T04:02:31.626780+0000","last_unstale":"2024-05-10T04:03:29.189591+0000","last_undegraded":"2024-05-10T04:03:29.189591+0000","last_fullsized":"2024-05-10T04:03:29.189591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:32.070530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.199643+0000","last_change":"2024-05-10T04:02:31.640669+0000","last_active":"2024-05-10T04:03:29.199643+0000","last_peered":"2024-05-10T04:03:29.199643+0000","last_clean":"2024-05-10T04:03:29.199643+0000","last_became_active":"2024-05-10T04:02:31.640410+0000","last_became_peered":"2024-05-10T04:02:31.640410+0000","last_unstale":"2024-05-10T04:03:29.199643+0000","last_undegraded":"2024-05-10T04:03:29.199643+0000","last_fullsized":"2024-05-10T04:03:29.199643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:02.235391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.193090+0000","last_change":"2024-05-10T04:02:31.633624+0000","last_active":"2024-05-10T04:03:29.193090+0000","last_peered":"2024-05-10T04:03:29.193090+0000","last_clean":"2024-05-10T04:03:29.193090+0000","last_became_active":"2024-05-10T04:02:31.633241+0000","last_became_peered":"2024-05-10T04:02:31.633241+0000","last_unstale":"2024-05-10T04:03:29.193090+0000","last_undegraded":"2024-05-10T04:03:29.193090+0000","last_fullsized":"2024-05-10T04:03:29.193090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:58.750912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.198385+0000","last_change":"2024-05-10T04:02:31.628632+0000","last_active":"2024-05-10T04:03:29.198385+0000","last_peered":"2024-05-10T04:03:29.198385+0000","last_clean":"2024-05-10T04:03:29.198385+0000","last_became_active":"2024-05-10T04:02:31.628349+0000","last_became_peered":"2024-05-10T04:02:31.628349+0000","last_unstale":"2024-05-10T04:03:29.198385+0000","last_undegraded":"2024-05-10T04:03:29.198385+0000","last_fullsized":"2024-05-10T04:03:29.198385+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:32.575168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.200128+0000","last_change":"2024-05-10T04:02:31.631328+0000","last_active":"2024-05-10T04:03:29.200128+0000","last_peered":"2024-05-10T04:03:29.200128+0000","last_clean":"2024-05-10T04:03:29.200128+0000","last_became_active":"2024-05-10T04:02:31.630957+0000","last_became_peered":"2024-05-10T04:02:31.630957+0000","last_unstale":"2024-05-10T04:03:29.200128+0000","last_undegraded":"2024-05-10T04:03:29.200128+0000","last_fullsized":"2024-05-10T04:03:29.200128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:40.400911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.191789+0000","last_change":"2024-05-10T04:02:31.617972+0000","last_active":"2024-05-10T04:03:29.191789+0000","last_peered":"2024-05-10T04:03:29.191789+0000","last_clean":"2024-05-10T04:03:29.191789+0000","last_became_active":"2024-05-10T04:02:31.617663+0000","last_became_peered":"2024-05-10T04:02:31.617663+0000","last_unstale":"2024-05-10T04:03:29.191789+0000","last_undegraded":"2024-05-10T04:03:29.191789+0000","last_fullsized":"2024-05-10T04:03:29.191789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:50.121955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:03:29.195144+0000","last_change":"2024-05-10T04:02:31.644256+0000","last_active":"2024-05-10T04:03:29.195144+0000","last_peered":"2024-05-10T04:03:29.195144+0000","last_clean":"2024-05-10T04:03:29.195144+0000","last_became_active":"2024-05-10T04:02:31.643819+0000","last_became_peered":"2024-05-10T04:02:31.643819+0000","last_unstale":"2024-05-10T04:03:29.195144+0000","last_undegraded":"2024-05-10T04:03:29.195144+0000","last_fullsized":"2024-05-10T04:03:29.195144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:02:31.161883+0000","last_clean_scrub_stamp":"2024-05-10T04:02:31.161883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:54.932547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279970,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66659,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":441,"num_read_kb":289,"num_write":158,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":211125,"data_compressed":4764,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514526,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":950272,"data_stored":480641,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90900000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73299999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.622,"15min":0.626},"min":{"1min":0.48599999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.42399999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.625,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86199999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.76300000000000001}]},{"osd":6,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63,"15min":0.623},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.5},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72099999999999997,"15min":0.71499999999999997},"min":{"1min":0.61199999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.73899999999999999}]},{"osd":7,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.83499999999999996,"15min":0.82599999999999996},"min":{"1min":0.68300000000000005,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1379999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.879},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77000000000000002,"15min":0.76800000000000002},"min":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.69199999999999995}]},{"osd":8,"last update":"Fri May 10 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.753}]},{"osd":9,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83899999999999997,"15min":0.84099999999999997},"min":{"1min":0.69099999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0469999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.90400000000000003,"15min":0.89900000000000002},"min":{"1min":0.76900000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1599999999999999,"5min":1.282,"15min":1.282},"last":0.91600000000000004}]},{"osd":10,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83099999999999996,"15min":0.82899999999999996},"min":{"1min":0.67600000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0309999999999999,"5min":1.319,"15min":1.319},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.84499999999999997,"15min":0.84599999999999997},"min":{"1min":0.68999999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0880000000000001,"5min":1.304,"15min":1.304},"last":0.77800000000000002}]}]},{"osd":10,"up_from":68,"seq":292057776161,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1089536,"data_stored":1134428,"data_compressed":54778,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.63600000000000001}]},{"osd":1,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.32200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.47799999999999998,"15min":0.48099999999999998},"min":{"1min":0.29899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67300000000000004,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.69899999999999995}]},{"osd":3,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.74099999999999999,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.40200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68799999999999994,"5min":0.872,"15min":0.872},"last":0.80000000000000004}]},{"osd":4,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.75700000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.42799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88800000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95599999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91400000000000003,"5min":1.175,"15min":1.175},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70099999999999996,"15min":0.70799999999999996},"min":{"1min":0.57699999999999996,"5min":0.504,"15min":0.504},"max":{"1min":0.81499999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.86899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":8,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90000000000000002,"5min":1.131,"15min":1.131},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.92900000000000005,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.85399999999999998}]},{"osd":9,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.76100000000000001},"min":{"1min":0.58399999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.88100000000000001,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.53100000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.79900000000000004,"5min":1.151,"15min":1.151},"last":0.81899999999999995}]},{"osd":11,"last update":"Fri May 10 04:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.871,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.78700000000000003,"15min":0.79400000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90900000000000003,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.96899999999999997}]}]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":966656,"data_stored":480877,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.435,"15min":0.443},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.46500000000000002,"15min":0.47599999999999998},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.89100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.38800000000000001}]},{"osd":1,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80700000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.61499999999999999,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71399999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.755},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.63300000000000001}]},{"osd":3,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.34799999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.52100000000000002,"15min":0.54300000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.58799999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48799999999999999}]},{"osd":4,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.67100000000000004,"15min":0.68400000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.751,"5min":1.149,"15min":1.149},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri May 10 04:02:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.1699999999999999}]},{"osd":7,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.74299999999999999,"15min":0.76300000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2470000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.752,"15min":0.78000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88100000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.65900000000000003}]},{"osd":8,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.53000000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.81100000000000005}]},{"osd":10,"last update":"Fri May 10 04:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93999999999999995,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.753,"15min":0.76800000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97299999999999998,"5min":1.306,"15min":1.306},"last":0.69499999999999995}]},{"osd":11,"last update":"Fri May 10 04:03:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79600000000000004}]}]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":966656,"data_stored":480797,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.378,"5min":0.376,"15min":0.376},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.54700000000000004},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.74199999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.65800000000000003,"15min":0.627},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95499999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.58499999999999996},"min":{"1min":0.46400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.5,"15min":0.47799999999999998},"min":{"1min":0.312,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81599999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48899999999999999,"15min":0.47899999999999998},"min":{"1min":0.39200000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53500000000000003}]},{"osd":3,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.35399999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80300000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.46200000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90300000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65600000000000003}]},{"osd":4,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63500000000000001,"15min":0.61399999999999999},"min":{"1min":0.47699999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88300000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.69799999999999995,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56299999999999994,"15min":0.54400000000000004},"min":{"1min":0.41899999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.74299999999999999,"15min":0.72999999999999998},"min":{"1min":0.56000000000000005,"5min":0.499,"15min":0.499},"max":{"1min":1.0489999999999999,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.55100000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.68999999999999995,"15min":0.66500000000000004},"min":{"1min":0.54300000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.71399999999999997,"15min":0.68799999999999994},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri May 10 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.69899999999999995,"15min":0.68700000000000006},"min":{"1min":0.58199999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.746},{"interface":"front","average":{"1min":0.874,"5min":0.76500000000000001,"15min":0.747},"min":{"1min":0.63600000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.81200000000000006}]},{"osd":10,"last update":"Fri May 10 04:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.71999999999999997,"15min":0.69499999999999995},"min":{"1min":0.69999999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.77400000000000002,"15min":0.74399999999999999},"min":{"1min":0.67400000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.84799999999999998}]},{"osd":11,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.873},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.89500000000000002}]}]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1015808,"data_stored":1077319,"data_compressed":53190,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.623,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.45700000000000002},"min":{"1min":0.36599999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.626,"5min":0.752,"15min":0.752},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85499999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.56399999999999995,"5min":0.376,"15min":0.376},"max":{"1min":0.83499999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66900000000000004}]},{"osd":3,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78900000000000003,"5min":1.143,"15min":1.143},"last":0.628},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76000000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.74299999999999999}]},{"osd":4,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.42599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77300000000000002,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.36699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73899999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.54300000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88600000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.67900000000000005},"min":{"1min":0.61399999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97299999999999998,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri May 10 04:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69499999999999995,"15min":0.69399999999999995},"min":{"1min":0.51100000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.90600000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.69199999999999995,"15min":0.70299999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85099999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.68500000000000005}]},{"osd":8,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.204,"15min":1.286},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88900000000000001,"5min":15.521000000000001,"15min":15.521000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":1.2829999999999999,"15min":1.3660000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0049999999999999,"5min":16.306999999999999,"15min":16.306999999999999},"last":0.89300000000000002}]},{"osd":9,"last update":"Fri May 10 04:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0509999999999999,"5min":1.167,"15min":1.167},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.753,"15min":0.751},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.83099999999999996}]},{"osd":10,"last update":"Fri May 10 04:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.622,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97099999999999997,"5min":1.232,"15min":1.232},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.66300000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93899999999999995,"5min":1.111,"15min":1.111},"last":0.874}]},{"osd":11,"last update":"Fri May 10 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.94299999999999995}]}]},{"osd":6,"up_from":44,"seq":188978561070,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":946176,"data_stored":476053,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.40200000000000002,"15min":0.39100000000000001},"min":{"1min":0.311,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.61599999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.371},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.438,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.5739999999999998,"5min":3.5739999999999998,"15min":3.5739999999999998},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.57299999999999995},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.84499999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.48299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.49299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.002,"5min":1.087,"15min":1.087},"last":0.443}]},{"osd":3,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48499999999999999,"15min":0.45600000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.50700000000000001,"15min":0.47699999999999998},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.1819999999999999,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":0.53300000000000003}]},{"osd":4,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89400000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94399999999999995,"5min":1.018,"15min":1.018},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.52200000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98199999999999998,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri May 10 04:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63500000000000001,"15min":0.63200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86899999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.629}]},{"osd":8,"last update":"Fri May 10 04:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.55000000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.503},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.57899999999999996}]},{"osd":10,"last update":"Fri May 10 04:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.75,"15min":0.76300000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.70699999999999996}]},{"osd":11,"last update":"Fri May 10 04:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1,"5min":1,"15min":1},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.73299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1015808,"data_stored":542322,"data_compressed":19169,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80700000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56100000000000005,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72699999999999998}]},{"osd":2,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.58199999999999996,"15min":0.53700000000000003},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.56299999999999994,"15min":0.52300000000000002},"min":{"1min":0.41599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri May 10 04:03:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56999999999999995,"15min":0.52000000000000002},"min":{"1min":0.46800000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81100000000000005,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57699999999999996,"15min":0.54100000000000004},"min":{"1min":0.442,"5min":0.373,"15min":0.373},"max":{"1min":0.88400000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri May 10 04:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.496,"15min":0.46200000000000002},"min":{"1min":0.36699999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.371},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.59099999999999997,"15min":0.53100000000000003},"min":{"1min":0.47099999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri May 10 04:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.56499999999999995,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri May 10 04:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.47199999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94599999999999995,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.621},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.66900000000000004,"15min":0.627},"min":{"1min":0.45500000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.069,"5min":1.119,"15min":1.119},"last":0.64300000000000002}]},{"osd":7,"last update":"Fri May 10 04:03:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.61499999999999999,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92900000000000005,"5min":1.075,"15min":1.075},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.68100000000000005,"15min":0.63500000000000001},"min":{"1min":0.47399999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.115,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.44400000000000001}]},{"osd":8,"last update":"Fri May 10 04:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69999999999999996,"15min":0.68799999999999994},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.55100000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83199999999999996}]},{"osd":9,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.81999999999999995}]},{"osd":10,"last update":"Fri May 10 04:03:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.80500000000000005,"15min":0.80100000000000005},"min":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0329999999999999,"5min":1.046,"15min":1.046},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.83899999999999997,"15min":0.82899999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0940000000000001,"5min":1.143,"15min":1.143},"last":0.755}]},{"osd":11,"last update":"Fri May 10 04:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.80000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":477441,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.52200000000000002},"min":{"1min":0.39900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.58999999999999997,"15min":0.53900000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.8689999999999998,"5min":4.8689999999999998,"15min":4.8689999999999998},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.62,"15min":0.55000000000000004},"min":{"1min":0.443,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.9009999999999998,"5min":4.9009999999999998,"15min":4.9009999999999998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82499999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61499999999999999}]},{"osd":3,"last update":"Fri May 10 04:03:33 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.61599999999999999,"15min":0.56599999999999995},"min":{"1min":0.42499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.8369999999999997,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.47299999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79700000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.64800000000000002,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":0.624}]},{"osd":6,"last update":"Fri May 10 04:03:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.59899999999999998,"15min":0.57999999999999996},"min":{"1min":0.32400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":4.7439999999999998,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.375},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.43099999999999999}]},{"osd":7,"last update":"Fri May 10 04:03:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.52900000000000003}]},{"osd":8,"last update":"Fri May 10 04:04:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.47799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.90000000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72999999999999998}]},{"osd":9,"last update":"Fri May 10 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.72099999999999997,"15min":0.69299999999999995},"min":{"1min":0.38500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.7999999999999998,"5min":4.7999999999999998,"15min":4.7999999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.47499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.54300000000000004}]},{"osd":10,"last update":"Fri May 10 04:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68300000000000005,"15min":0.67500000000000004},"min":{"1min":0.53500000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.77800000000000002,"15min":0.746},"min":{"1min":0.57999999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":4.96,"5min":4.96,"15min":4.96},"last":0.65700000000000003}]},{"osd":11,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.499,"5min":0.495,"15min":0.495},"max":{"1min":0.876,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.69699999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":1032192,"data_stored":546425,"data_compressed":19169,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.54200000000000004,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.496}]},{"osd":1,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56799999999999995,"15min":0.53500000000000003},"min":{"1min":0.41799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.59799999999999998,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.57899999999999996},"min":{"1min":0.48299999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.58999999999999997}]},{"osd":4,"last update":"Fri May 10 04:03:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.76500000000000001}]},{"osd":5,"last update":"Fri May 10 04:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.51700000000000002,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79000000000000004,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.40799999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.65500000000000003}]},{"osd":6,"last update":"Fri May 10 04:03:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.48899999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.66800000000000004,"15min":0.63900000000000001},"min":{"1min":0.52600000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri May 10 04:03:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.83899999999999997}]},{"osd":8,"last update":"Fri May 10 04:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.66300000000000003,"15min":0.621},"min":{"1min":0.61899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.753},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.66500000000000004,"15min":0.63500000000000001},"min":{"1min":0.41799999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.81799999999999995}]},{"osd":9,"last update":"Fri May 10 04:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.74299999999999999,"15min":0.72299999999999998},"min":{"1min":0.66800000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0189999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.874},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.59199999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.91900000000000004,"5min":1.016,"15min":1.016},"last":0.73499999999999999}]},{"osd":10,"last update":"Fri May 10 04:03:45 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.753,"15min":0.73699999999999999},"min":{"1min":0.66000000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.875,"5min":0.79200000000000004,"15min":0.77800000000000002},"min":{"1min":0.73399999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.88600000000000001}]},{"osd":11,"last update":"Fri May 10 04:03:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.92800000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":476247,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36199999999999999,"15min":0.35599999999999998},"min":{"1min":0.29199999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.503,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.377,"15min":0.36399999999999999},"min":{"1min":0.32600000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.41899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.747,"15min":0.747},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.53600000000000003,"15min":0.50800000000000001},"min":{"1min":0.47599999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77300000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80300000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri May 10 04:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.48899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.69899999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.52500000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78700000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri May 10 04:03:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80100000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.59199999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78800000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.502}]},{"osd":6,"last update":"Fri May 10 04:03:33 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.68000000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.53900000000000003,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.41299999999999998}]},{"osd":7,"last update":"Fri May 10 04:03:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.435,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73299999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.52700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.71999999999999997}]},{"osd":8,"last update":"Fri May 10 04:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.439}]},{"osd":9,"last update":"Fri May 10 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69599999999999995,"15min":0.71199999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89900000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83999999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.443}]},{"osd":10,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70299999999999996,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83799999999999997,"5min":1.228,"15min":1.228},"last":0.625},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.499,"5min":0.433,"15min":0.433},"max":{"1min":0.82099999999999995,"5min":1.25,"15min":1.25},"last":0.66100000000000003}]},{"osd":11,"last update":"Fri May 10 04:03:33 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.69999999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":481474,"data_compressed":17581,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.33900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.50900000000000001,"15min":0.502},"min":{"1min":0.371,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.79000000000000004}]},{"osd":1,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41699999999999998,"15min":0.39300000000000002},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53900000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.39100000000000001},"min":{"1min":0.35099999999999998,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.53500000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.64400000000000002,"15min":0.57699999999999996},"min":{"1min":0.59399999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.91000000000000003,"5min":1.046,"15min":1.046},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri May 10 04:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79800000000000004,"5min":0.873,"15min":0.873},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.59299999999999997,"15min":0.54700000000000004},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.6050000000000004,"5min":4.6050000000000004,"15min":4.6050000000000004},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri May 10 04:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.46400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.92300000000000004,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81699999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.874}]},{"osd":6,"last update":"Fri May 10 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64000000000000001,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.89700000000000002,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.77500000000000002}]},{"osd":7,"last update":"Fri May 10 04:03:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72999999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.44900000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":1.01,"15min":1.01},"last":0.95299999999999996}]},{"osd":8,"last update":"Fri May 10 04:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68100000000000005,"15min":0.68500000000000005},"min":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.79900000000000004,"5min":0.879,"15min":0.879},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.56899999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.90300000000000002}]},{"osd":9,"last update":"Fri May 10 04:03:26 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.80000000000000004,"15min":0.76700000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.6449999999999996,"5min":5.6449999999999996,"15min":5.6449999999999996},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.82899999999999996},"min":{"1min":0.54500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":5.6100000000000003,"5min":5.6100000000000003,"15min":5.6100000000000003},"last":0.92600000000000005}]},{"osd":10,"last update":"Fri May 10 04:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.52300000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.93300000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83899999999999997,"15min":0.84499999999999997},"min":{"1min":0.56799999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.997,"5min":1.161,"15min":1.161},"last":0.97799999999999998}]},{"osd":11,"last update":"Fri May 10 04:03:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":1.002}]}]},{"osd":5,"up_from":37,"seq":158913790002,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":1019904,"data_stored":1071917,"data_compressed":53190,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52400000000000002,"15min":0.498},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77800000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.433,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74299999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57499999999999996,"15min":0.55100000000000005},"min":{"1min":0.45400000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.505},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.47999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.40400000000000003},"min":{"1min":0.35799999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.44,"15min":0.42199999999999999},"min":{"1min":0.37,"5min":0.249,"15min":0.249},"max":{"1min":0.66100000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80500000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.74299999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri May 10 04:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61099999999999999,"15min":0.59199999999999997},"min":{"1min":0.48199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri May 10 04:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.874,"15min":0.874},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66300000000000003,"15min":0.65500000000000003},"min":{"1min":0.51700000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85999999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78000000000000003}]},{"osd":7,"last update":"Fri May 10 04:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.70799999999999996,"15min":0.68500000000000005},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.68899999999999995,"15min":0.65600000000000003},"min":{"1min":0.56100000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.81000000000000005}]},{"osd":8,"last update":"Fri May 10 04:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91900000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.46000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.74199999999999999}]},{"osd":9,"last update":"Fri May 10 04:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.55000000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.88400000000000001}]},{"osd":10,"last update":"Fri May 10 04:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77100000000000002,"15min":0.76900000000000002},"min":{"1min":0.56299999999999994,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80300000000000005,"15min":0.80200000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.85099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33975,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35609,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":66359,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70462,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64282,"data_compressed":1588,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:04:09.814 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:04:09.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.978+0000 7f016e80d700 1 -- 172.21.15.150:0/2737728354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168101bd0 msgr2=0x7f0168101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:09.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.978+0000 7f016e80d700 1 --2- 172.21.15.150:0/2737728354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168101bd0 0x7f0168101fe0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0158009990 tx=0x7f0158009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.978+0000 7f016e80d700 1 -- 172.21.15.150:0/2737728354 shutdown_connections 2024-05-10T04:04:09.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.978+0000 7f016e80d700 1 --2- 172.21.15.150:0/2737728354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0168103690 0x7f0168109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.978+0000 7f016e80d700 1 --2- 172.21.15.150:0/2737728354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168101bd0 0x7f0168101fe0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.978+0000 7f016e80d700 1 --2- 172.21.15.150:0/2737728354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0168102cf0 0x7f0168103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.978+0000 7f016e80d700 1 -- 172.21.15.150:0/2737728354 >> 172.21.15.150:0/2737728354 conn(0x7f01680fd1e0 msgr2=0x7f01680ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:09.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.978+0000 7f016e80d700 1 -- 172.21.15.150:0/2737728354 shutdown_connections 2024-05-10T04:04:09.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.978+0000 7f016e80d700 1 -- 172.21.15.150:0/2737728354 wait complete. 2024-05-10T04:04:09.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.979+0000 7f016e80d700 1 Processor -- start 2024-05-10T04:04:09.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.979+0000 7f016e80d700 1 -- start start 2024-05-10T04:04:09.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.979+0000 7f016e80d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0168101bd0 0x7f0168198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:09.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.979+0000 7f016e80d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168102cf0 0x7f0168198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:09.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.979+0000 7f016e80d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0168103690 0x7f016819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:09.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.979+0000 7f016e80d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016819fb00 con 0x7f0168102cf0 2024-05-10T04:04:09.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.979+0000 7f016e80d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016819fc70 con 0x7f0168101bd0 2024-05-10T04:04:09.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.979+0000 7f016e80d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016819fe30 con 0x7f0168103690 2024-05-10T04:04:09.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f01677fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168102cf0 0x7f0168198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:09.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f01677fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168102cf0 0x7f0168198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50108/0 (socket says 172.21.15.150:50108) 2024-05-10T04:04:09.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f01677fe700 1 -- 172.21.15.150:0/1588777173 learned_addr learned my addr 172.21.15.150:0/1588777173 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:09.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f0167fff700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0168101bd0 0x7f0168198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f016cdaa700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0168103690 0x7f016819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f01677fe700 1 -- 172.21.15.150:0/1588777173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0168103690 msgr2=0x7f016819f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f01677fe700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0168103690 0x7f016819f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f01677fe700 1 -- 172.21.15.150:0/1588777173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0168101bd0 msgr2=0x7f0168198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f01677fe700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0168101bd0 0x7f0168198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f01677fe700 1 -- 172.21.15.150:0/1588777173 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01580095f0 con 0x7f0168102cf0 2024-05-10T04:04:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f01677fe700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168102cf0 0x7f0168198560 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f015000e9a0 tx=0x7f015000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f01657fa700 1 -- 172.21.15.150:0/1588777173 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150009790 con 0x7f0168102cf0 2024-05-10T04:04:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f01657fa700 1 -- 172.21.15.150:0/1588777173 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f015000cbd0 con 0x7f0168102cf0 2024-05-10T04:04:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f016e80d700 1 -- 172.21.15.150:0/1588777173 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01681a0110 con 0x7f0168102cf0 2024-05-10T04:04:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f01657fa700 1 -- 172.21.15.150:0/1588777173 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150018680 con 0x7f0168102cf0 2024-05-10T04:04:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.980+0000 7f016e80d700 1 -- 172.21.15.150:0/1588777173 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01681a0620 con 0x7f0168102cf0 2024-05-10T04:04:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.981+0000 7f016e80d700 1 -- 172.21.15.150:0/1588777173 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0168108de0 con 0x7f0168102cf0 2024-05-10T04:04:09.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.982+0000 7f01657fa700 1 -- 172.21.15.150:0/1588777173 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f015000cd40 con 0x7f0168102cf0 2024-05-10T04:04:09.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.983+0000 7f01657fa700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f01540a11b0 0x7f01540a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:09.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.983+0000 7f01657fa700 1 -- 172.21.15.150:0/1588777173 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0150014070 con 0x7f0168102cf0 2024-05-10T04:04:09.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.983+0000 7f0167fff700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f01540a11b0 0x7f01540a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:09.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.983+0000 7f0167fff700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f01540a11b0 0x7f01540a36c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0158006010 tx=0x7f015801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:09.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:09.987+0000 7f01657fa700 1 -- 172.21.15.150:0/1588777173 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0150096620 con 0x7f0168102cf0 2024-05-10T04:04:10.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.184+0000 7f016e80d700 1 -- 172.21.15.150:0/1588777173 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01680678b0 con 0x7f0168102cf0 2024-05-10T04:04:10.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.186+0000 7f01657fa700 1 -- 172.21.15.150:0/1588777173 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24554 (secure 0 0 0) 0x7f0150099c40 con 0x7f0168102cf0 2024-05-10T04:04:10.188 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:04:10.189 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":87,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:03:30.185138+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:04:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.188+0000 7f016e80d700 1 -- 172.21.15.150:0/1588777173 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f01540a11b0 msgr2=0x7f01540a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.188+0000 7f016e80d700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f01540a11b0 0x7f01540a36c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0158006010 tx=0x7f015801a040 comp rx=0 tx=0).stop 2024-05-10T04:04:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.188+0000 7f016e80d700 1 -- 172.21.15.150:0/1588777173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168102cf0 msgr2=0x7f0168198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.188+0000 7f016e80d700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168102cf0 0x7f0168198560 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f015000e9a0 tx=0x7f015000ecb0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.188+0000 7f016e80d700 1 -- 172.21.15.150:0/1588777173 shutdown_connections 2024-05-10T04:04:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.188+0000 7f016e80d700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f01540a11b0 0x7f01540a36c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.188+0000 7f016e80d700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0168103690 0x7f016819f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.188+0000 7f016e80d700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0168101bd0 0x7f0168198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.188+0000 7f016e80d700 1 --2- 172.21.15.150:0/1588777173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0168102cf0 0x7f0168198560 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.188+0000 7f016e80d700 1 -- 172.21.15.150:0/1588777173 >> 172.21.15.150:0/1588777173 conn(0x7f01680fd1e0 msgr2=0x7f0168105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.188+0000 7f016e80d700 1 -- 172.21.15.150:0/1588777173 shutdown_connections 2024-05-10T04:04:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.188+0000 7f016e80d700 1 -- 172.21.15.150:0/1588777173 wait complete. 2024-05-10T04:04:10.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:02:30.076694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:02:30.760710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:02:33.850714+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:04:10.205 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:04:10.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.338+0000 7f788e46d700 1 -- 172.21.15.150:0/3969371831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7888101110 msgr2=0x7f7888101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:10.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.338+0000 7f788e46d700 1 --2- 172.21.15.150:0/3969371831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7888101110 0x7f7888101520 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7878009990 tx=0x7f7878009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.338+0000 7f788e46d700 1 -- 172.21.15.150:0/3969371831 shutdown_connections 2024-05-10T04:04:10.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.338+0000 7f788e46d700 1 --2- 172.21.15.150:0/3969371831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7888102950 0x7f7888109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.338+0000 7f788e46d700 1 --2- 172.21.15.150:0/3969371831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7888101110 0x7f7888101520 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.338+0000 7f788e46d700 1 --2- 172.21.15.150:0/3969371831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7888102000 0x7f7888102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.338+0000 7f788e46d700 1 -- 172.21.15.150:0/3969371831 >> 172.21.15.150:0/3969371831 conn(0x7f78880fb8f0 msgr2=0x7f78880fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:10.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.338+0000 7f788e46d700 1 -- 172.21.15.150:0/3969371831 shutdown_connections 2024-05-10T04:04:10.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.338+0000 7f788e46d700 1 -- 172.21.15.150:0/3969371831 wait complete. 2024-05-10T04:04:10.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.339+0000 7f788e46d700 1 Processor -- start 2024-05-10T04:04:10.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.339+0000 7f788e46d700 1 -- start start 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.339+0000 7f788e46d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7888101110 0x7f78881007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.339+0000 7f788e46d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7888102000 0x7f788819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.339+0000 7f788e46d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7888102950 0x7f788819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.339+0000 7f788e46d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788819fcb0 con 0x7f7888102000 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.339+0000 7f788e46d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788819fe20 con 0x7f7888102950 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.339+0000 7f788e46d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788819ffe0 con 0x7f7888101110 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.339+0000 7f78877fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7888102000 0x7f788819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f78877fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7888102000 0x7f788819cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50114/0 (socket says 172.21.15.150:50114) 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f78877fe700 1 -- 172.21.15.150:0/2116144829 learned_addr learned my addr 172.21.15.150:0/2116144829 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f788ca0a700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7888102950 0x7f788819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f7887fff700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7888101110 0x7f78881007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f78877fe700 1 -- 172.21.15.150:0/2116144829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7888101110 msgr2=0x7f78881007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f78877fe700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7888101110 0x7f78881007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f78877fe700 1 -- 172.21.15.150:0/2116144829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7888102950 msgr2=0x7f788819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:10.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f78877fe700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7888102950 0x7f788819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f78877fe700 1 -- 172.21.15.150:0/2116144829 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78780095f0 con 0x7f7888102000 2024-05-10T04:04:10.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f788ca0a700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7888102950 0x7f788819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:10.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f7887fff700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7888101110 0x7f78881007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:10.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f78877fe700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7888102000 0x7f788819cfa0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f787000e9a0 tx=0x7f787000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:10.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f78857fa700 1 -- 172.21.15.150:0/2116144829 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870009790 con 0x7f7888102000 2024-05-10T04:04:10.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f78857fa700 1 -- 172.21.15.150:0/2116144829 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f787000cbd0 con 0x7f7888102000 2024-05-10T04:04:10.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f788e46d700 1 -- 172.21.15.150:0/2116144829 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78881a02c0 con 0x7f7888102000 2024-05-10T04:04:10.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f78857fa700 1 -- 172.21.15.150:0/2116144829 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870018680 con 0x7f7888102000 2024-05-10T04:04:10.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.340+0000 7f788e46d700 1 -- 172.21.15.150:0/2116144829 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78881a07d0 con 0x7f7888102000 2024-05-10T04:04:10.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.341+0000 7f788e46d700 1 -- 172.21.15.150:0/2116144829 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78881080f0 con 0x7f7888102000 2024-05-10T04:04:10.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.342+0000 7f78857fa700 1 -- 172.21.15.150:0/2116144829 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f787000cd40 con 0x7f7888102000 2024-05-10T04:04:10.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.343+0000 7f78857fa700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f78740a1260 0x7f78740a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:10.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.343+0000 7f78857fa700 1 -- 172.21.15.150:0/2116144829 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7870014070 con 0x7f7888102000 2024-05-10T04:04:10.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.343+0000 7f7887fff700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f78740a1260 0x7f78740a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:10.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.344+0000 7f7887fff700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f78740a1260 0x7f78740a3770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f787800b2e0 tx=0x7f7878005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:10.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.348+0000 7f78857fa700 1 -- 172.21.15.150:0/2116144829 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7870052d60 con 0x7f7888102000 2024-05-10T04:04:10.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:10 smithi150 ceph-mon[42477]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:04:10.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:10 smithi150 ceph-mon[42477]: pgmap v277: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T04:04:10.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:10 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1588777173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:04:10.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:10 smithi181 ceph-mon[46602]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:04:10.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:10 smithi181 ceph-mon[46602]: pgmap v277: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T04:04:10.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:10 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1588777173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:04:10.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:10 smithi161 ceph-mon[46918]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:04:10.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:10 smithi161 ceph-mon[46918]: pgmap v277: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T04:04:10.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:10 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1588777173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:04:10.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.543+0000 7f788e46d700 1 -- 172.21.15.150:0/2116144829 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78880681a0 con 0x7f7888102000 2024-05-10T04:04:10.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.544+0000 7f78857fa700 1 -- 172.21.15.150:0/2116144829 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7870099e60 con 0x7f7888102000 2024-05-10T04:04:10.546 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:04:10.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.546+0000 7f788e46d700 1 -- 172.21.15.150:0/2116144829 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f78740a1260 msgr2=0x7f78740a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:10.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.546+0000 7f788e46d700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f78740a1260 0x7f78740a3770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f787800b2e0 tx=0x7f7878005fb0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.546+0000 7f788e46d700 1 -- 172.21.15.150:0/2116144829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7888102000 msgr2=0x7f788819cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:10.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.546+0000 7f788e46d700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7888102000 0x7f788819cfa0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f787000e9a0 tx=0x7f787000ed60 comp rx=0 tx=0).stop 2024-05-10T04:04:10.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.546+0000 7f788e46d700 1 -- 172.21.15.150:0/2116144829 shutdown_connections 2024-05-10T04:04:10.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.546+0000 7f788e46d700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f78740a1260 0x7f78740a3770 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.546+0000 7f788e46d700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7888102950 0x7f788819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.546+0000 7f788e46d700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7888101110 0x7f78881007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.546+0000 7f788e46d700 1 --2- 172.21.15.150:0/2116144829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7888102000 0x7f788819cfa0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.546+0000 7f788e46d700 1 -- 172.21.15.150:0/2116144829 >> 172.21.15.150:0/2116144829 conn(0x7f78880fb8f0 msgr2=0x7f7888105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:10.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.546+0000 7f788e46d700 1 -- 172.21.15.150:0/2116144829 shutdown_connections 2024-05-10T04:04:10.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.546+0000 7f788e46d700 1 -- 172.21.15.150:0/2116144829 wait complete. 2024-05-10T04:04:10.562 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:04:10.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.695+0000 7f3ff32de700 1 -- 172.21.15.150:0/1063536672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fec101d80 msgr2=0x7f3fec108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:10.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.695+0000 7f3ff32de700 1 --2- 172.21.15.150:0/1063536672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fec101d80 0x7f3fec108630 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c210 tx=0x7f3fe400c520 comp rx=0 tx=0).stop 2024-05-10T04:04:10.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.695+0000 7f3ff32de700 1 -- 172.21.15.150:0/1063536672 shutdown_connections 2024-05-10T04:04:10.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.695+0000 7f3ff32de700 1 --2- 172.21.15.150:0/1063536672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fec101d80 0x7f3fec108630 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.695+0000 7f3ff32de700 1 --2- 172.21.15.150:0/1063536672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fec1002c0 0x7f3fec1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.695+0000 7f3ff32de700 1 --2- 172.21.15.150:0/1063536672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fec1013e0 0x7f3fec1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.695+0000 7f3ff32de700 1 -- 172.21.15.150:0/1063536672 >> 172.21.15.150:0/1063536672 conn(0x7f3fec0fb8f0 msgr2=0x7f3fec0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:10.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.696+0000 7f3ff32de700 1 -- 172.21.15.150:0/1063536672 shutdown_connections 2024-05-10T04:04:10.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.696+0000 7f3ff32de700 1 -- 172.21.15.150:0/1063536672 wait complete. 2024-05-10T04:04:10.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.696+0000 7f3ff32de700 1 Processor -- start 2024-05-10T04:04:10.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.696+0000 7f3ff32de700 1 -- start start 2024-05-10T04:04:10.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.696+0000 7f3ff32de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fec1002c0 0x7f3fec198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:10.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.696+0000 7f3ff32de700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fec1013e0 0x7f3fec1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:10.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.696+0000 7f3ff32de700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fec101d80 0x7f3fec19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:10.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.696+0000 7f3ff32de700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec19fb10 con 0x7f3fec101d80 2024-05-10T04:04:10.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.696+0000 7f3ff32de700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec19fc80 con 0x7f3fec1013e0 2024-05-10T04:04:10.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.696+0000 7f3ff32de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec19fe40 con 0x7f3fec1002c0 2024-05-10T04:04:10.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3ff187b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fec101d80 0x7f3fec19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:10.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3ff187b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fec101d80 0x7f3fec19f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50126/0 (socket says 172.21.15.150:50126) 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3ff187b700 1 -- 172.21.15.150:0/4058581285 learned_addr learned my addr 172.21.15.150:0/4058581285 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3ff107a700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fec1002c0 0x7f3fec198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3ff0879700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fec1013e0 0x7f3fec1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3ff187b700 1 -- 172.21.15.150:0/4058581285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fec1002c0 msgr2=0x7f3fec198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3ff187b700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fec1002c0 0x7f3fec198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3ff187b700 1 -- 172.21.15.150:0/4058581285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fec1013e0 msgr2=0x7f3fec1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3ff187b700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fec1013e0 0x7f3fec1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3ff187b700 1 -- 172.21.15.150:0/4058581285 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe400ae10 con 0x7f3fec101d80 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3ff187b700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fec101d80 0x7f3fec19f500 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c9a0 tx=0x7f3fe4004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3fe27fc700 1 -- 172.21.15.150:0/4058581285 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400f040 con 0x7f3fec101d80 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3fe27fc700 1 -- 172.21.15.150:0/4058581285 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fe400a590 con 0x7f3fec101d80 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3fe27fc700 1 -- 172.21.15.150:0/4058581285 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe401e480 con 0x7f3fec101d80 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3ff32de700 1 -- 172.21.15.150:0/4058581285 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fec1a00c0 con 0x7f3fec101d80 2024-05-10T04:04:10.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.697+0000 7f3ff32de700 1 -- 172.21.15.150:0/4058581285 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec1a0540 con 0x7f3fec101d80 2024-05-10T04:04:10.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.698+0000 7f3ff32de700 1 -- 172.21.15.150:0/4058581285 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fec1074d0 con 0x7f3fec101d80 2024-05-10T04:04:10.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.699+0000 7f3fe27fc700 1 -- 172.21.15.150:0/4058581285 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3fe40075a0 con 0x7f3fec101d80 2024-05-10T04:04:10.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.700+0000 7f3fe27fc700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3fd80a1360 0x7f3fd80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:10.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.700+0000 7f3fe27fc700 1 -- 172.21.15.150:0/4058581285 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3fe40ceca0 con 0x7f3fec101d80 2024-05-10T04:04:10.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.700+0000 7f3ff107a700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3fd80a1360 0x7f3fd80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:10.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.700+0000 7f3ff107a700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3fd80a1360 0x7f3fd80a3870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3fec1012e0 tx=0x7f3fdc00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:10.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.704+0000 7f3fe27fc700 1 -- 172.21.15.150:0/4058581285 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3fe4097ff0 con 0x7f3fec101d80 2024-05-10T04:04:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.901+0000 7f3ff32de700 1 -- 172.21.15.150:0/4058581285 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3fec10b3e0 con 0x7f3fec101d80 2024-05-10T04:04:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.901+0000 7f3fe27fc700 1 -- 172.21.15.150:0/4058581285 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3fe409b610 con 0x7f3fec101d80 2024-05-10T04:04:10.903 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:04:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.903+0000 7f3ff32de700 1 -- 172.21.15.150:0/4058581285 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3fd80a1360 msgr2=0x7f3fd80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.903+0000 7f3ff32de700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3fd80a1360 0x7f3fd80a3870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3fec1012e0 tx=0x7f3fdc00b5b0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.903+0000 7f3ff32de700 1 -- 172.21.15.150:0/4058581285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fec101d80 msgr2=0x7f3fec19f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.903+0000 7f3ff32de700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fec101d80 0x7f3fec19f500 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c9a0 tx=0x7f3fe4004d90 comp rx=0 tx=0).stop 2024-05-10T04:04:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.903+0000 7f3ff32de700 1 -- 172.21.15.150:0/4058581285 shutdown_connections 2024-05-10T04:04:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.903+0000 7f3ff32de700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3fd80a1360 0x7f3fd80a3870 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.903+0000 7f3ff32de700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fec101d80 0x7f3fec19f500 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.903+0000 7f3ff32de700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fec1002c0 0x7f3fec198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.903+0000 7f3ff32de700 1 --2- 172.21.15.150:0/4058581285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fec1013e0 0x7f3fec1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.903+0000 7f3ff32de700 1 -- 172.21.15.150:0/4058581285 >> 172.21.15.150:0/4058581285 conn(0x7f3fec0fb8f0 msgr2=0x7f3fec104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.903+0000 7f3ff32de700 1 -- 172.21.15.150:0/4058581285 shutdown_connections 2024-05-10T04:04:10.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:10.903+0000 7f3ff32de700 1 -- 172.21.15.150:0/4058581285 wait complete. 2024-05-10T04:04:10.918 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:04:11.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.051+0000 7fc74644b700 1 -- 172.21.15.150:0/2645320097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7401078b0 msgr2=0x7fc740109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:11.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.051+0000 7fc74644b700 1 --2- 172.21.15.150:0/2645320097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7401078b0 0x7fc740109d90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc73800b0a0 tx=0x7fc73800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.051+0000 7fc74644b700 1 -- 172.21.15.150:0/2645320097 shutdown_connections 2024-05-10T04:04:11.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.051+0000 7fc74644b700 1 --2- 172.21.15.150:0/2645320097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7401078b0 0x7fc740109d90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.051+0000 7fc74644b700 1 --2- 172.21.15.150:0/2645320097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc74006a490 0x7fc740104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.051+0000 7fc74644b700 1 --2- 172.21.15.150:0/2645320097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc740104d40 0x7fc740107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.051+0000 7fc74644b700 1 -- 172.21.15.150:0/2645320097 >> 172.21.15.150:0/2645320097 conn(0x7fc7400fc0c0 msgr2=0x7fc7400fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:11.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.051+0000 7fc74644b700 1 -- 172.21.15.150:0/2645320097 shutdown_connections 2024-05-10T04:04:11.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.051+0000 7fc74644b700 1 -- 172.21.15.150:0/2645320097 wait complete. 2024-05-10T04:04:11.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc74644b700 1 Processor -- start 2024-05-10T04:04:11.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc74644b700 1 -- start start 2024-05-10T04:04:11.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc74644b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc74006a490 0x7fc740101fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:11.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc74644b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc740104d40 0x7fc740100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:11.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc74644b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7401078b0 0x7fc740100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:11.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc74644b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc740101210 con 0x7fc7401078b0 2024-05-10T04:04:11.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc74644b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc740101380 con 0x7fc740104d40 2024-05-10T04:04:11.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc74644b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc740101540 con 0x7fc74006a490 2024-05-10T04:04:11.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc7449e8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7401078b0 0x7fc740100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:11.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc7449e8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7401078b0 0x7fc740100bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50140/0 (socket says 172.21.15.150:50140) 2024-05-10T04:04:11.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc7449e8700 1 -- 172.21.15.150:0/3536107414 learned_addr learned my addr 172.21.15.150:0/3536107414 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:11.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc73f7fe700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc740104d40 0x7fc740100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:11.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc73ffff700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc74006a490 0x7fc740101fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:11.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc7449e8700 1 -- 172.21.15.150:0/3536107414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc74006a490 msgr2=0x7fc740101fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:11.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc7449e8700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc74006a490 0x7fc740101fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc7449e8700 1 -- 172.21.15.150:0/3536107414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc740104d40 msgr2=0x7fc740100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:11.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc7449e8700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc740104d40 0x7fc740100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.052+0000 7fc7449e8700 1 -- 172.21.15.150:0/3536107414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc738009d00 con 0x7fc7401078b0 2024-05-10T04:04:11.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.053+0000 7fc7449e8700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7401078b0 0x7fc740100bb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc738006130 tx=0x7fc738004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:11.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.053+0000 7fc73d7fa700 1 -- 172.21.15.150:0/3536107414 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73800e040 con 0x7fc7401078b0 2024-05-10T04:04:11.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.053+0000 7fc73d7fa700 1 -- 172.21.15.150:0/3536107414 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc73800cec0 con 0x7fc7401078b0 2024-05-10T04:04:11.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.053+0000 7fc74644b700 1 -- 172.21.15.150:0/3536107414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7401a21b0 con 0x7fc7401078b0 2024-05-10T04:04:11.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.053+0000 7fc73d7fa700 1 -- 172.21.15.150:0/3536107414 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73801dea0 con 0x7fc7401078b0 2024-05-10T04:04:11.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.053+0000 7fc74644b700 1 -- 172.21.15.150:0/3536107414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7401a2630 con 0x7fc7401078b0 2024-05-10T04:04:11.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.054+0000 7fc74644b700 1 -- 172.21.15.150:0/3536107414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7400fdc70 con 0x7fc7401078b0 2024-05-10T04:04:11.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.055+0000 7fc73d7fa700 1 -- 172.21.15.150:0/3536107414 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc73801d690 con 0x7fc7401078b0 2024-05-10T04:04:11.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.056+0000 7fc73d7fa700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc72c0a12e0 0x7fc72c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:11.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.056+0000 7fc73ffff700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc72c0a12e0 0x7fc72c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:11.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.056+0000 7fc73d7fa700 1 -- 172.21.15.150:0/3536107414 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc7380cef70 con 0x7fc7401078b0 2024-05-10T04:04:11.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.056+0000 7fc73ffff700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc72c0a12e0 0x7fc72c0a37f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc730005fd0 tx=0x7fc73000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:11.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.060+0000 7fc73d7fa700 1 -- 172.21.15.150:0/3536107414 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc738098320 con 0x7fc7401078b0 2024-05-10T04:04:11.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.267+0000 7fc74644b700 1 -- 172.21.15.150:0/3536107414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc74010d3b0 con 0x7fc7401078b0 2024-05-10T04:04:11.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.267+0000 7fc73d7fa700 1 -- 172.21.15.150:0/3536107414 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc73809b940 con 0x7fc7401078b0 2024-05-10T04:04:11.270 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:04:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.269+0000 7fc74644b700 1 -- 172.21.15.150:0/3536107414 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc72c0a12e0 msgr2=0x7fc72c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.269+0000 7fc74644b700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc72c0a12e0 0x7fc72c0a37f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc730005fd0 tx=0x7fc73000a560 comp rx=0 tx=0).stop 2024-05-10T04:04:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.269+0000 7fc74644b700 1 -- 172.21.15.150:0/3536107414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7401078b0 msgr2=0x7fc740100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.269+0000 7fc74644b700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7401078b0 0x7fc740100bb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc738006130 tx=0x7fc738004d00 comp rx=0 tx=0).stop 2024-05-10T04:04:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.269+0000 7fc74644b700 1 -- 172.21.15.150:0/3536107414 shutdown_connections 2024-05-10T04:04:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.269+0000 7fc74644b700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc72c0a12e0 0x7fc72c0a37f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.269+0000 7fc74644b700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7401078b0 0x7fc740100bb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.269+0000 7fc74644b700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc74006a490 0x7fc740101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.269+0000 7fc74644b700 1 --2- 172.21.15.150:0/3536107414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc740104d40 0x7fc740100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.269+0000 7fc74644b700 1 -- 172.21.15.150:0/3536107414 >> 172.21.15.150:0/3536107414 conn(0x7fc7400fc0c0 msgr2=0x7fc740102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.270+0000 7fc74644b700 1 -- 172.21.15.150:0/3536107414 shutdown_connections 2024-05-10T04:04:11.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.270+0000 7fc74644b700 1 -- 172.21.15.150:0/3536107414 wait complete. 2024-05-10T04:04:11.286 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:04:11.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.420+0000 7f172b8f6700 1 -- 172.21.15.150:0/3311744924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1724102000 msgr2=0x7f1724102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:11.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.420+0000 7f172b8f6700 1 --2- 172.21.15.150:0/3311744924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1724102000 0x7f1724102410 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1720009990 tx=0x7f1720009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.420+0000 7f172b8f6700 1 -- 172.21.15.150:0/3311744924 shutdown_connections 2024-05-10T04:04:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.420+0000 7f172b8f6700 1 --2- 172.21.15.150:0/3311744924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1724102950 0x7f1724109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.420+0000 7f172b8f6700 1 --2- 172.21.15.150:0/3311744924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1724101110 0x7f1724101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.420+0000 7f172b8f6700 1 --2- 172.21.15.150:0/3311744924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1724102000 0x7f1724102410 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.420+0000 7f172b8f6700 1 -- 172.21.15.150:0/3311744924 >> 172.21.15.150:0/3311744924 conn(0x7f17240fb8f0 msgr2=0x7f17240fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.421+0000 7f172b8f6700 1 -- 172.21.15.150:0/3311744924 shutdown_connections 2024-05-10T04:04:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.421+0000 7f172b8f6700 1 -- 172.21.15.150:0/3311744924 wait complete. 2024-05-10T04:04:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.421+0000 7f172b8f6700 1 Processor -- start 2024-05-10T04:04:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.421+0000 7f172b8f6700 1 -- start start 2024-05-10T04:04:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.421+0000 7f172b8f6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1724101110 0x7f17241007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.421+0000 7f172b8f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1724102000 0x7f172419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.421+0000 7f172b8f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1724102950 0x7f172419f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.421+0000 7f172b8f6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172419fb50 con 0x7f1724102950 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.421+0000 7f172b8f6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172419fcc0 con 0x7f1724101110 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.421+0000 7f172b8f6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172419fe80 con 0x7f1724102000 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f1729e93700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1724102950 0x7f172419f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f1729e93700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1724102950 0x7f172419f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50152/0 (socket says 172.21.15.150:50152) 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f1729e93700 1 -- 172.21.15.150:0/2704802827 learned_addr learned my addr 172.21.15.150:0/2704802827 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f1729692700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1724101110 0x7f17241007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f1728e91700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1724102000 0x7f172419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f1729e93700 1 -- 172.21.15.150:0/2704802827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1724102000 msgr2=0x7f172419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f1729e93700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1724102000 0x7f172419cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f1729e93700 1 -- 172.21.15.150:0/2704802827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1724101110 msgr2=0x7f17241007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f1729e93700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1724101110 0x7f17241007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f1729e93700 1 -- 172.21.15.150:0/2704802827 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17200095f0 con 0x7f1724102950 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f1729e93700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1724102950 0x7f172419f450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f171c007ee0 tx=0x7f171c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f17167fc700 1 -- 172.21.15.150:0/2704802827 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c017070 con 0x7f1724102950 2024-05-10T04:04:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f17167fc700 1 -- 172.21.15.150:0/2704802827 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f171c00f040 con 0x7f1724102950 2024-05-10T04:04:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f17167fc700 1 -- 172.21.15.150:0/2704802827 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c005740 con 0x7f1724102950 2024-05-10T04:04:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f172b8f6700 1 -- 172.21.15.150:0/2704802827 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17241a0160 con 0x7f1724102950 2024-05-10T04:04:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.422+0000 7f172b8f6700 1 -- 172.21.15.150:0/2704802827 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17241a0620 con 0x7f1724102950 2024-05-10T04:04:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.423+0000 7f172b8f6700 1 -- 172.21.15.150:0/2704802827 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17241080f0 con 0x7f1724102950 2024-05-10T04:04:11.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.424+0000 7f17167fc700 1 -- 172.21.15.150:0/2704802827 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f171c00dc60 con 0x7f1724102950 2024-05-10T04:04:11.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.425+0000 7f17167fc700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f17100a1260 0x7f17100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:11.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.425+0000 7f17167fc700 1 -- 172.21.15.150:0/2704802827 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f171c0ccb80 con 0x7f1724102950 2024-05-10T04:04:11.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.425+0000 7f1729692700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f17100a1260 0x7f17100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:11.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.425+0000 7f1729692700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f17100a1260 0x7f17100a3770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1724101f90 tx=0x7f171800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:11.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.429+0000 7f17167fc700 1 -- 172.21.15.150:0/2704802827 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f171c095ea0 con 0x7f1724102950 2024-05-10T04:04:11.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:11 smithi150 ceph-mon[42477]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:04:11.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:11 smithi150 ceph-mon[42477]: from='client.15576 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:04:11.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:11 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2116144829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:04:11.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:11 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4058581285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:04:11.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:11 smithi181 ceph-mon[46602]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:04:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:11 smithi181 ceph-mon[46602]: from='client.15576 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:04:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:11 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2116144829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:04:11.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:11 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4058581285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:04:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:11 smithi161 ceph-mon[46918]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:04:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:11 smithi161 ceph-mon[46918]: from='client.15576 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:04:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:11 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2116144829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:04:11.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:11 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4058581285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:04:11.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.630+0000 7f172b8f6700 1 -- 172.21.15.150:0/2704802827 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f17240681a0 con 0x7f1724102950 2024-05-10T04:04:11.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.630+0000 7f17167fc700 1 -- 172.21.15.150:0/2704802827 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f171c0994c0 con 0x7f1724102950 2024-05-10T04:04:11.632 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:04:11.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.632+0000 7f172b8f6700 1 -- 172.21.15.150:0/2704802827 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f17100a1260 msgr2=0x7f17100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:11.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.632+0000 7f172b8f6700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f17100a1260 0x7f17100a3770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1724101f90 tx=0x7f171800a380 comp rx=0 tx=0).stop 2024-05-10T04:04:11.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.632+0000 7f172b8f6700 1 -- 172.21.15.150:0/2704802827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1724102950 msgr2=0x7f172419f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:11.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.632+0000 7f172b8f6700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1724102950 0x7f172419f450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f171c007ee0 tx=0x7f171c00d3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.633+0000 7f172b8f6700 1 -- 172.21.15.150:0/2704802827 shutdown_connections 2024-05-10T04:04:11.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.633+0000 7f172b8f6700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f17100a1260 0x7f17100a3770 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.633+0000 7f172b8f6700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1724102950 0x7f172419f450 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.633+0000 7f172b8f6700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1724101110 0x7f17241007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.633+0000 7f172b8f6700 1 --2- 172.21.15.150:0/2704802827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1724102000 0x7f172419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.633+0000 7f172b8f6700 1 -- 172.21.15.150:0/2704802827 >> 172.21.15.150:0/2704802827 conn(0x7f17240fb8f0 msgr2=0x7f1724105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:11.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.633+0000 7f172b8f6700 1 -- 172.21.15.150:0/2704802827 shutdown_connections 2024-05-10T04:04:11.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.633+0000 7f172b8f6700 1 -- 172.21.15.150:0/2704802827 wait complete. 2024-05-10T04:04:11.648 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:04:11.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.780+0000 7f8f6f11c700 1 -- 172.21.15.150:0/1752805309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f680ffc20 msgr2=0x7f8f68100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:11.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.780+0000 7f8f6f11c700 1 --2- 172.21.15.150:0/1752805309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f680ffc20 0x7f8f68100030 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009990 tx=0x7f8f54009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.780+0000 7f8f6f11c700 1 -- 172.21.15.150:0/1752805309 shutdown_connections 2024-05-10T04:04:11.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.780+0000 7f8f6f11c700 1 --2- 172.21.15.150:0/1752805309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f68100570 0x7f8f6810a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.780+0000 7f8f6f11c700 1 --2- 172.21.15.150:0/1752805309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f681031e0 0x7f8f681035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.780+0000 7f8f6f11c700 1 --2- 172.21.15.150:0/1752805309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f680ffc20 0x7f8f68100030 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.780+0000 7f8f6f11c700 1 -- 172.21.15.150:0/1752805309 >> 172.21.15.150:0/1752805309 conn(0x7f8f680fb8f0 msgr2=0x7f8f680fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:11.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.780+0000 7f8f6f11c700 1 -- 172.21.15.150:0/1752805309 shutdown_connections 2024-05-10T04:04:11.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.780+0000 7f8f6f11c700 1 -- 172.21.15.150:0/1752805309 wait complete. 2024-05-10T04:04:11.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f6f11c700 1 Processor -- start 2024-05-10T04:04:11.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f6f11c700 1 -- start start 2024-05-10T04:04:11.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f6f11c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f680ffc20 0x7f8f68197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:11.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f6f11c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f68100570 0x7f8f68198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:11.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f6f11c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f681031e0 0x7f8f6819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:11.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f6f11c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819fab0 con 0x7f8f681031e0 2024-05-10T04:04:11.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f6f11c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819fc20 con 0x7f8f680ffc20 2024-05-10T04:04:11.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f6f11c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819fde0 con 0x7f8f68100570 2024-05-10T04:04:11.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f6d6b9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f681031e0 0x7f8f6819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:11.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f6d6b9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f681031e0 0x7f8f6819f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50182/0 (socket says 172.21.15.150:50182) 2024-05-10T04:04:11.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f6d6b9700 1 -- 172.21.15.150:0/3738552420 learned_addr learned my addr 172.21.15.150:0/3738552420 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:11.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f6ceb8700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f680ffc20 0x7f8f68197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:11.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f67fff700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f68100570 0x7f8f68198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:11.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.781+0000 7f8f6d6b9700 1 -- 172.21.15.150:0/3738552420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f68100570 msgr2=0x7f8f68198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:11.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.782+0000 7f8f6d6b9700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f68100570 0x7f8f68198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.782+0000 7f8f6d6b9700 1 -- 172.21.15.150:0/3738552420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f680ffc20 msgr2=0x7f8f68197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:11.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.782+0000 7f8f6d6b9700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f680ffc20 0x7f8f68197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:11.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.782+0000 7f8f6d6b9700 1 -- 172.21.15.150:0/3738552420 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f540095f0 con 0x7f8f681031e0 2024-05-10T04:04:11.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.782+0000 7f8f6d6b9700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f681031e0 0x7f8f6819f4a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000ef30 tx=0x7f8f6000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:11.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.782+0000 7f8f65ffb700 1 -- 172.21.15.150:0/3738552420 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60018070 con 0x7f8f681031e0 2024-05-10T04:04:11.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.782+0000 7f8f65ffb700 1 -- 172.21.15.150:0/3738552420 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f6000f040 con 0x7f8f681031e0 2024-05-10T04:04:11.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.782+0000 7f8f6f11c700 1 -- 172.21.15.150:0/3738552420 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f681a0060 con 0x7f8f681031e0 2024-05-10T04:04:11.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.782+0000 7f8f65ffb700 1 -- 172.21.15.150:0/3738552420 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000cdb0 con 0x7f8f681031e0 2024-05-10T04:04:11.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.782+0000 7f8f6f11c700 1 -- 172.21.15.150:0/3738552420 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f681a0570 con 0x7f8f681031e0 2024-05-10T04:04:11.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.783+0000 7f8f6f11c700 1 -- 172.21.15.150:0/3738552420 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f68109030 con 0x7f8f681031e0 2024-05-10T04:04:11.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.784+0000 7f8f65ffb700 1 -- 172.21.15.150:0/3738552420 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f8f60003a00 con 0x7f8f681031e0 2024-05-10T04:04:11.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.785+0000 7f8f65ffb700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8f580a11b0 0x7f8f580a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:11.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.785+0000 7f8f6ceb8700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8f580a11b0 0x7f8f580a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:11.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.785+0000 7f8f65ffb700 1 -- 172.21.15.150:0/3738552420 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8f600cc940 con 0x7f8f681031e0 2024-05-10T04:04:11.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.785+0000 7f8f6ceb8700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8f580a11b0 0x7f8f580a36c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8f68068120 tx=0x7f8f5c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:11.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:11.789+0000 7f8f65ffb700 1 -- 172.21.15.150:0/3738552420 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8f60095cf0 con 0x7f8f681031e0 2024-05-10T04:04:12.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.034+0000 7f8f6f11c700 1 -- 172.21.15.150:0/3738552420 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f680678b0 con 0x7f8f681031e0 2024-05-10T04:04:12.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.035+0000 7f8f65ffb700 1 -- 172.21.15.150:0/3738552420 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10886 (secure 0 0 0) 0x7f8f60099310 con 0x7f8f681031e0 2024-05-10T04:04:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:04:12.037 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:04:04.157819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:04:12.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.037+0000 7f8f6f11c700 1 -- 172.21.15.150:0/3738552420 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8f580a11b0 msgr2=0x7f8f580a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.037+0000 7f8f6f11c700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8f580a11b0 0x7f8f580a36c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8f68068120 tx=0x7f8f5c007400 comp rx=0 tx=0).stop 2024-05-10T04:04:12.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.037+0000 7f8f6f11c700 1 -- 172.21.15.150:0/3738552420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f681031e0 msgr2=0x7f8f6819f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.037+0000 7f8f6f11c700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f681031e0 0x7f8f6819f4a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000ef30 tx=0x7f8f6000c3a0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.037+0000 7f8f6f11c700 1 -- 172.21.15.150:0/3738552420 shutdown_connections 2024-05-10T04:04:12.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.037+0000 7f8f6f11c700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f8f580a11b0 0x7f8f580a36c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.037+0000 7f8f6f11c700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f681031e0 0x7f8f6819f4a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.037+0000 7f8f6f11c700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f680ffc20 0x7f8f68197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.037+0000 7f8f6f11c700 1 --2- 172.21.15.150:0/3738552420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f68100570 0x7f8f68198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.037+0000 7f8f6f11c700 1 -- 172.21.15.150:0/3738552420 >> 172.21.15.150:0/3738552420 conn(0x7f8f680fb8f0 msgr2=0x7f8f68106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.037+0000 7f8f6f11c700 1 -- 172.21.15.150:0/3738552420 shutdown_connections 2024-05-10T04:04:12.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.037+0000 7f8f6f11c700 1 -- 172.21.15.150:0/3738552420 wait complete. 2024-05-10T04:04:12.040 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 25 2024-05-10T04:04:12.053 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:04:12.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.184+0000 7fee42043700 1 -- 172.21.15.150:0/2945233364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee3c101be0 msgr2=0x7fee3c101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.184+0000 7fee42043700 1 --2- 172.21.15.150:0/2945233364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee3c101be0 0x7fee3c101ff0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fee24009990 tx=0x7fee24009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.184+0000 7fee42043700 1 -- 172.21.15.150:0/2945233364 shutdown_connections 2024-05-10T04:04:12.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.184+0000 7fee42043700 1 --2- 172.21.15.150:0/2945233364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c102580 0x7fee3c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.184+0000 7fee42043700 1 --2- 172.21.15.150:0/2945233364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3c100ac0 0x7fee3c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.184+0000 7fee42043700 1 --2- 172.21.15.150:0/2945233364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee3c101be0 0x7fee3c101ff0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.184+0000 7fee42043700 1 -- 172.21.15.150:0/2945233364 >> 172.21.15.150:0/2945233364 conn(0x7fee3c0fc0d0 msgr2=0x7fee3c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.184+0000 7fee42043700 1 -- 172.21.15.150:0/2945233364 shutdown_connections 2024-05-10T04:04:12.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.184+0000 7fee42043700 1 -- 172.21.15.150:0/2945233364 wait complete. 2024-05-10T04:04:12.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.185+0000 7fee42043700 1 Processor -- start 2024-05-10T04:04:12.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.185+0000 7fee42043700 1 -- start start 2024-05-10T04:04:12.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.185+0000 7fee42043700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee3c100ac0 0x7fee3c10b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.185+0000 7fee42043700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3c101be0 0x7fee3c1099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.185+0000 7fee42043700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c102580 0x7fee3c109f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.185+0000 7fee42043700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c10a580 con 0x7fee3c100ac0 2024-05-10T04:04:12.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.185+0000 7fee42043700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c10a6f0 con 0x7fee3c101be0 2024-05-10T04:04:12.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.185+0000 7fee42043700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c10a8b0 con 0x7fee3c102580 2024-05-10T04:04:12.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee3b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee3c100ac0 0x7fee3c10b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee3b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee3c100ac0 0x7fee3c10b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50196/0 (socket says 172.21.15.150:50196) 2024-05-10T04:04:12.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee3b7fe700 1 -- 172.21.15.150:0/1732794535 learned_addr learned my addr 172.21.15.150:0/1732794535 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:12.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee3affd700 1 --2- 172.21.15.150:0/1732794535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3c101be0 0x7fee3c1099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee3b7fe700 1 -- 172.21.15.150:0/1732794535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c102580 msgr2=0x7fee3c109f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee3b7fe700 1 --2- 172.21.15.150:0/1732794535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c102580 0x7fee3c109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee3b7fe700 1 -- 172.21.15.150:0/1732794535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3c101be0 msgr2=0x7fee3c1099e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee3b7fe700 1 --2- 172.21.15.150:0/1732794535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3c101be0 0x7fee3c1099e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee3b7fe700 1 -- 172.21.15.150:0/1732794535 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee240095f0 con 0x7fee3c100ac0 2024-05-10T04:04:12.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee3b7fe700 1 --2- 172.21.15.150:0/1732794535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee3c100ac0 0x7fee3c10b340 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00d950 tx=0x7fee2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee38ff9700 1 -- 172.21.15.150:0/1732794535 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c004220 con 0x7fee3c100ac0 2024-05-10T04:04:12.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee42043700 1 -- 172.21.15.150:0/1732794535 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c1a2200 con 0x7fee3c100ac0 2024-05-10T04:04:12.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee38ff9700 1 -- 172.21.15.150:0/1732794535 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee2c004380 con 0x7fee3c100ac0 2024-05-10T04:04:12.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee38ff9700 1 -- 172.21.15.150:0/1732794535 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c003bf0 con 0x7fee3c100ac0 2024-05-10T04:04:12.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.186+0000 7fee42043700 1 -- 172.21.15.150:0/1732794535 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3c1a2710 con 0x7fee3c100ac0 2024-05-10T04:04:12.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.187+0000 7fee42043700 1 -- 172.21.15.150:0/1732794535 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3c107cd0 con 0x7fee3c100ac0 2024-05-10T04:04:12.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.188+0000 7fee38ff9700 1 -- 172.21.15.150:0/1732794535 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fee2c003d50 con 0x7fee3c100ac0 2024-05-10T04:04:12.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.189+0000 7fee38ff9700 1 --2- 172.21.15.150:0/1732794535 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fee280a11b0 0x7fee280a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.189+0000 7fee3affd700 1 --2- 172.21.15.150:0/1732794535 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fee280a11b0 0x7fee280a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.189+0000 7fee38ff9700 1 -- 172.21.15.150:0/1732794535 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fee2c0cd010 con 0x7fee3c100ac0 2024-05-10T04:04:12.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.190+0000 7fee3affd700 1 --2- 172.21.15.150:0/1732794535 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fee280a11b0 0x7fee280a36c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fee2400b5c0 tx=0x7fee2401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.193+0000 7fee38ff9700 1 -- 172.21.15.150:0/1732794535 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fee2c096330 con 0x7fee3c100ac0 2024-05-10T04:04:12.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.393+0000 7fee42043700 1 -- 172.21.15.150:0/1732794535 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee3c0694d0 con 0x7fee3c100ac0 2024-05-10T04:04:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.395+0000 7fee38ff9700 1 -- 172.21.15.150:0/1732794535 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24554 (secure 0 0 0) 0x7fee2c017020 con 0x7fee3c100ac0 2024-05-10T04:04:12.397 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:04:12.397 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":87,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:03:30.185138+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:04:12.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.397+0000 7fee42043700 1 -- 172.21.15.150:0/1732794535 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fee280a11b0 msgr2=0x7fee280a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.397+0000 7fee42043700 1 --2- 172.21.15.150:0/1732794535 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fee280a11b0 0x7fee280a36c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fee2400b5c0 tx=0x7fee2401a040 comp rx=0 tx=0).stop 2024-05-10T04:04:12.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.397+0000 7fee42043700 1 -- 172.21.15.150:0/1732794535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee3c100ac0 msgr2=0x7fee3c10b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.397+0000 7fee42043700 1 --2- 172.21.15.150:0/1732794535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee3c100ac0 0x7fee3c10b340 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00d950 tx=0x7fee2c00dc60 comp rx=0 tx=0).stop 2024-05-10T04:04:12.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.397+0000 7fee42043700 1 -- 172.21.15.150:0/1732794535 shutdown_connections 2024-05-10T04:04:12.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.397+0000 7fee42043700 1 --2- 172.21.15.150:0/1732794535 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fee280a11b0 0x7fee280a36c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.397+0000 7fee42043700 1 --2- 172.21.15.150:0/1732794535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c102580 0x7fee3c109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.397+0000 7fee42043700 1 --2- 172.21.15.150:0/1732794535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee3c100ac0 0x7fee3c10b340 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.397+0000 7fee42043700 1 --2- 172.21.15.150:0/1732794535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3c101be0 0x7fee3c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.397+0000 7fee42043700 1 -- 172.21.15.150:0/1732794535 >> 172.21.15.150:0/1732794535 conn(0x7fee3c0fc0d0 msgr2=0x7fee3c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.397+0000 7fee42043700 1 -- 172.21.15.150:0/1732794535 shutdown_connections 2024-05-10T04:04:12.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.397+0000 7fee42043700 1 -- 172.21.15.150:0/1732794535 wait complete. 2024-05-10T04:04:12.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:12 smithi150 ceph-mon[42477]: pgmap v278: 129 pgs: 129 active+clean; 649 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-10T04:04:12.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:12 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3536107414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:04:12.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:12 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2704802827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:04:12.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:12 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3738552420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:04:12.413 DEBUG:tasks.fwd_scrub:Yielding 2024-05-10T04:04:12.413 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-10T04:04:12.423 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:04:12.423 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-10T04:04:12.424 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-10T04:04:12.424 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T04:04:12.424 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T04:04:12.426 INFO:teuthology.run_tasks:Running task workunit... 2024-05-10T04:04:12.469 INFO:tasks.workunit:Pulling workunits from ref 50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:04:12.469 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-10T04:04:12.469 INFO:tasks.workunit:timeout=3h 2024-05-10T04:04:12.469 INFO:tasks.workunit:cleanup=True 2024-05-10T04:04:12.470 DEBUG:teuthology.orchestra.run.smithi150:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-10T04:04:12.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:12 smithi181 ceph-mon[46602]: pgmap v278: 129 pgs: 129 active+clean; 649 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-10T04:04:12.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:12 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3536107414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:04:12.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:12 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2704802827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:04:12.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:12 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3738552420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:04:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-10T04:04:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-10T04:04:12.489 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2024-05-10T04:04:12.489 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:04:12.489 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:cephfs_t:s0 2024-05-10T04:04:12.489 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-10 04:02:58.821330834 +0000 2024-05-10T04:04:12.489 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-10 04:02:58.821330834 +0000 2024-05-10T04:04:12.489 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-10 04:04:02.576450104 +0000 2024-05-10T04:04:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-10 04:02:58.821330834 +0000 2024-05-10T04:04:12.489 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-10T04:04:12.489 DEBUG:teuthology.orchestra.run.smithi150:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-10T04:04:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:12 smithi161 ceph-mon[46918]: pgmap v278: 129 pgs: 129 active+clean; 649 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-10T04:04:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:12 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3536107414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:04:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:12 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2704802827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:04:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:12 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3738552420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:04:12.565 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:04:12.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.564+0000 7f0fa7ce4700 1 -- 172.21.15.150:0/2673819933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fa01002c0 msgr2=0x7f0fa01006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.564+0000 7f0fa7ce4700 1 --2- 172.21.15.150:0/2673819933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fa01002c0 0x7f0fa01006d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00aaa0 tx=0x7f0f9c00adb0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.564+0000 7f0fa7ce4700 1 -- 172.21.15.150:0/2673819933 shutdown_connections 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.564+0000 7f0fa7ce4700 1 --2- 172.21.15.150:0/2673819933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fa0101d80 0x7f0fa0108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.564+0000 7f0fa7ce4700 1 --2- 172.21.15.150:0/2673819933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fa01002c0 0x7f0fa01006d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.564+0000 7f0fa7ce4700 1 --2- 172.21.15.150:0/2673819933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fa01013e0 0x7f0fa01017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.564+0000 7f0fa7ce4700 1 -- 172.21.15.150:0/2673819933 >> 172.21.15.150:0/2673819933 conn(0x7f0fa00fb8f0 msgr2=0x7f0fa00fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.564+0000 7f0fa7ce4700 1 -- 172.21.15.150:0/2673819933 shutdown_connections 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.564+0000 7f0fa7ce4700 1 -- 172.21.15.150:0/2673819933 wait complete. 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.564+0000 7f0fa7ce4700 1 Processor -- start 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.564+0000 7f0fa7ce4700 1 -- start start 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa7ce4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fa01002c0 0x7f0fa0197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa7ce4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fa01013e0 0x7f0fa0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa7ce4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fa0101d80 0x7f0fa019f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa7ce4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa019f9d0 con 0x7f0fa0101d80 2024-05-10T04:04:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa7ce4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa019fb40 con 0x7f0fa01002c0 2024-05-10T04:04:12.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa7ce4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa019fd00 con 0x7f0fa01013e0 2024-05-10T04:04:12.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa6281700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fa0101d80 0x7f0fa019f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa6281700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fa0101d80 0x7f0fa019f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50216/0 (socket says 172.21.15.150:50216) 2024-05-10T04:04:12.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa6281700 1 -- 172.21.15.150:0/928432824 learned_addr learned my addr 172.21.15.150:0/928432824 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:12.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa5a80700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fa01002c0 0x7f0fa0197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa527f700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fa01013e0 0x7f0fa0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa6281700 1 -- 172.21.15.150:0/928432824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fa01013e0 msgr2=0x7f0fa0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa6281700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fa01013e0 0x7f0fa0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa6281700 1 -- 172.21.15.150:0/928432824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fa01002c0 msgr2=0x7f0fa0197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa6281700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fa01002c0 0x7f0fa0197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa6281700 1 -- 172.21.15.150:0/928432824 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c00a700 con 0x7f0fa0101d80 2024-05-10T04:04:12.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0fa6281700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fa0101d80 0x7f0fa019f3c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0f98007ee0 tx=0x7f0f9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0f96ffd700 1 -- 172.21.15.150:0/928432824 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98017070 con 0x7f0fa0101d80 2024-05-10T04:04:12.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.565+0000 7f0f96ffd700 1 -- 172.21.15.150:0/928432824 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f9800f040 con 0x7f0fa0101d80 2024-05-10T04:04:12.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.566+0000 7f0f96ffd700 1 -- 172.21.15.150:0/928432824 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98005580 con 0x7f0fa0101d80 2024-05-10T04:04:12.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.566+0000 7f0fa7ce4700 1 -- 172.21.15.150:0/928432824 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa019ffe0 con 0x7f0fa0101d80 2024-05-10T04:04:12.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.566+0000 7f0fa7ce4700 1 -- 172.21.15.150:0/928432824 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa01a0350 con 0x7f0fa0101d80 2024-05-10T04:04:12.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.568+0000 7f0f96ffd700 1 -- 172.21.15.150:0/928432824 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0f9800db20 con 0x7f0fa0101d80 2024-05-10T04:04:12.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.568+0000 7f0f96ffd700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0f8c0a1260 0x7f0f8c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.569+0000 7f0f96ffd700 1 -- 172.21.15.150:0/928432824 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0f980cc440 con 0x7f0fa0101d80 2024-05-10T04:04:12.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.569+0000 7f0fa5a80700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0f8c0a1260 0x7f0f8c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.569+0000 7f0fa5a80700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0f8c0a1260 0x7f0f8c0a3770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c006010 tx=0x7f0f9c003cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.569+0000 7f0fb8028b80 1 Processor -- start 2024-05-10T04:04:12.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.569+0000 7f0fb8028b80 1 -- start start 2024-05-10T04:04:12.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fb8028b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c39720 0x5595b7c39b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fb8028b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c3a5f0 0x5595b7c3aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fb8028b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 0x5595b7c41840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fb8028b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b7b601d0 con 0x5595b7c3af40 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fb8028b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b7b5db90 con 0x5595b7c3a5f0 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fb8028b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b7b59480 con 0x5595b7c39720 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fa6281700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 0x5595b7c41840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fa6281700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 0x5595b7c41840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50218/0 (socket says 172.21.15.150:50218) 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fa6281700 1 -- 172.21.15.150:0/794764656 learned_addr learned my addr 172.21.15.150:0/794764656 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fa5a80700 1 --2- 172.21.15.150:0/794764656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c39720 0x5595b7c39b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fa527f700 1 --2- 172.21.15.150:0/794764656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c3a5f0 0x5595b7c3aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fa6281700 1 -- 172.21.15.150:0/794764656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c39720 msgr2=0x5595b7c39b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fa6281700 1 --2- 172.21.15.150:0/794764656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c39720 0x5595b7c39b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fa6281700 1 -- 172.21.15.150:0/794764656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c3a5f0 msgr2=0x5595b7c3aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fa6281700 1 --2- 172.21.15.150:0/794764656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c3a5f0 0x5595b7c3aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fa6281700 1 -- 172.21.15.150:0/794764656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b79da430 con 0x5595b7c3af40 2024-05-10T04:04:12.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fa527f700 1 --2- 172.21.15.150:0/794764656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c3a5f0 0x5595b7c3aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:04:12.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0fa6281700 1 --2- 172.21.15.150:0/794764656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 0x5595b7c41840 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0f98095e90 tx=0x7f0f98004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b506327c7fbe84c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.570+0000 7f0f8bfff700 1 -- 172.21.15.150:0/794764656 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9809d250 con 0x5595b7c3af40 2024-05-10T04:04:12.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.571+0000 7f0f8bfff700 1 -- 172.21.15.150:0/794764656 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f980a18a0 con 0x5595b7c3af40 2024-05-10T04:04:12.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.571+0000 7f0f8bfff700 1 -- 172.21.15.150:0/794764656 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f980c9d30 con 0x5595b7c3af40 2024-05-10T04:04:12.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.571+0000 7f0fb8028b80 1 -- 172.21.15.150:0/794764656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 msgr2=0x5595b7c41840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.571+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/794764656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 0x5595b7c41840 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0f98095e90 tx=0x7f0f98004b80 comp rx=0 tx=0).stop 2024-05-10T04:04:12.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.571+0000 7f0fb8028b80 1 -- 172.21.15.150:0/794764656 shutdown_connections 2024-05-10T04:04:12.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.571+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/794764656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 0x5595b7c41840 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.571+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/794764656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c39720 0x5595b7c39b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.571+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/794764656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c3a5f0 0x5595b7c3aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.571+0000 7f0fb8028b80 1 -- 172.21.15.150:0/794764656 >> 172.21.15.150:0/794764656 conn(0x5595b7b9bae0 msgr2=0x5595b7c34060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.572+0000 7f0fb8028b80 1 -- 172.21.15.150:0/794764656 shutdown_connections 2024-05-10T04:04:12.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.572+0000 7f0fb8028b80 1 -- 172.21.15.150:0/794764656 wait complete. 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.572+0000 7f0fb8028b80 1 -- start start 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.572+0000 7f0fb8028b80 1 Processor -- start 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.572+0000 7f0fb8028b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c39720 0x5595b7c437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.572+0000 7f0fb8028b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c3a5f0 0x5595b7c43cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.572+0000 7f0fb8028b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 0x5595b7c4ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.572+0000 7f0fb8028b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b7af55e0 con 0x5595b7c3af40 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.572+0000 7f0fb8028b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b7af5280 con 0x5595b7c39720 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.572+0000 7f0fb8028b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b79dc720 con 0x5595b7c3a5f0 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0fb8028b80 10 client.0 _ll_register_callbacks cb 0x5595b7afebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0fa6281700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 0x5595b7c4ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0fa6281700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 0x5595b7c4ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50230/0 (socket says 172.21.15.150:50230) 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0fa6281700 1 -- 172.21.15.150:0/1501711228 learned_addr learned my addr 172.21.15.150:0/1501711228 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0fa5a80700 1 --2- 172.21.15.150:0/1501711228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c39720 0x5595b7c437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0fa527f700 1 --2- 172.21.15.150:0/1501711228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c3a5f0 0x5595b7c43cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0fa6281700 1 -- 172.21.15.150:0/1501711228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c3a5f0 msgr2=0x5595b7c43cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0fa6281700 1 --2- 172.21.15.150:0/1501711228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c3a5f0 0x5595b7c43cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0fa6281700 1 -- 172.21.15.150:0/1501711228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c39720 msgr2=0x5595b7c437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0fa6281700 1 --2- 172.21.15.150:0/1501711228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c39720 0x5595b7c437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0fa6281700 1 -- 172.21.15.150:0/1501711228 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9000b8e0 con 0x5595b7c3af40 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0fa6281700 1 --2- 172.21.15.150:0/1501711228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 0x5595b7c4ac50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0f98099120 tx=0x7f0f98004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3eda6ab1f252d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0f897fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0f897fa700 1 -- 172.21.15.150:0/1501711228 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98097560 con 0x5595b7c3af40 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0f897fa700 1 -- 172.21.15.150:0/1501711228 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0f9809d250 con 0x5595b7c3af40 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.573+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1501711228 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595b7ab22f0 con 0x5595b7c3af40 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.574+0000 7f0f897fa700 1 -- 172.21.15.150:0/1501711228 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f98099680 con 0x5595b7c3af40 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.574+0000 7f0fb8028b80 10 client.15630 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.574+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1501711228 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f0fa01a0350 con 0x5595b7c3af40 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.574+0000 7f0f897fa700 1 -- 172.21.15.150:0/1501711228 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f9807e020 con 0x5595b7c3af40 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.574+0000 7f0f897fa700 1 -- 172.21.15.150:0/1501711228 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0f98098a30 con 0x5595b7c3af40 2024-05-10T04:04:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.574+0000 7f0fb8028b80 10 client.15630 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.574+0000 7f0fb8028b80 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.574+0000 7f0fb8028b80 20 client.15630 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.574+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/1501711228 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5595b7c4f020 0x5595b7c51470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.574+0000 7f0fb8028b80 4 client.15630 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.574+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1501711228 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595b7c08800 con 0x5595b7c4f020 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.574+0000 7f0fa5a80700 1 --2- 172.21.15.150:0/1501711228 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5595b7c4f020 0x5595b7c51470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.575+0000 7f0f897fa700 10 client.15630 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.575+0000 7f0fa5a80700 1 --2- 172.21.15.150:0/1501711228 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5595b7c4f020 0x5595b7c51470 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282586094e700614 server_cookie=4f725dc7fd0dcbe7 in_seq=0 out_seq=0 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.575+0000 7f0f897fa700 1 -- 172.21.15.150:0/1501711228 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5595b7c08800 con 0x5595b7c4f020 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.575+0000 7f0f897fa700 10 client.15630 handle_command_reply: tid=0 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.576+0000 7f0fb8028b80 1 client.15630 shutdown 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.576+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1501711228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 msgr2=0x5595b7c4ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.576+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/1501711228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 0x5595b7c4ac50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0f98099120 tx=0x7f0f98004e70 comp rx=0 tx=0).stop 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.576+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1501711228 shutdown_connections 2024-05-10T04:04:12.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.576+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/1501711228 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5595b7c4f020 0x5595b7c51470 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.576+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/1501711228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3af40 0x5595b7c4ac50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.576+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/1501711228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c39720 0x5595b7c437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.576+0000 7f0f897fa700 0 client.15630 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:12.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.576+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/1501711228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c3a5f0 0x5595b7c43cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.576+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1501711228 >> 172.21.15.150:0/1501711228 conn(0x5595b7b9bae0 msgr2=0x5595b7c34030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.576+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1501711228 shutdown_connections 2024-05-10T04:04:12.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.576+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1501711228 wait complete. 2024-05-10T04:04:12.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.576+0000 7f0fb8028b80 20 client.15630 trim_cache size 0 max 16384 2024-05-10T04:04:12.584 INFO:tasks.workunit.client.0.smithi150.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-10T04:04:12.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.594+0000 7f0fb8028b80 1 Processor -- start 2024-05-10T04:04:12.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.594+0000 7f0fb8028b80 1 -- start start 2024-05-10T04:04:12.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fb8028b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7b9bae0 0x5595b7c3e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fb8028b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c39720 0x5595b7c4ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fb8028b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 0x5595b7c3e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fb8028b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b7c4eda0 con 0x5595b7c3a5f0 2024-05-10T04:04:12.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fb8028b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b7c443b0 con 0x5595b7c39720 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fb8028b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b7c39fd0 con 0x5595b7b9bae0 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fa6281700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 0x5595b7c3e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fa6281700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 0x5595b7c3e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50236/0 (socket says 172.21.15.150:50236) 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fa6281700 1 -- 172.21.15.150:0/2439096389 learned_addr learned my addr 172.21.15.150:0/2439096389 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fa527f700 1 --2- 172.21.15.150:0/2439096389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c39720 0x5595b7c4ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fa5a80700 1 --2- 172.21.15.150:0/2439096389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7b9bae0 0x5595b7c3e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fa6281700 1 -- 172.21.15.150:0/2439096389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7b9bae0 msgr2=0x5595b7c3e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fa6281700 1 --2- 172.21.15.150:0/2439096389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7b9bae0 0x5595b7c3e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fa6281700 1 -- 172.21.15.150:0/2439096389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c39720 msgr2=0x5595b7c4ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fa6281700 1 --2- 172.21.15.150:0/2439096389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c39720 0x5595b7c4ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fa6281700 1 -- 172.21.15.150:0/2439096389 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b7c34300 con 0x5595b7c3a5f0 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fa5a80700 1 --2- 172.21.15.150:0/2439096389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7b9bae0 0x5595b7c3e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0fa6281700 1 --2- 172.21.15.150:0/2439096389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 0x5595b7c3e9b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800d700 tx=0x7f0f980950a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fe49c21b424f89 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.595+0000 7f0f94ff9700 1 -- 172.21.15.150:0/2439096389 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98099b20 con 0x5595b7c3a5f0 2024-05-10T04:04:12.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.596+0000 7f0f94ff9700 1 -- 172.21.15.150:0/2439096389 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f9807e030 con 0x5595b7c3a5f0 2024-05-10T04:04:12.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.596+0000 7f0f94ff9700 1 -- 172.21.15.150:0/2439096389 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f980977c0 con 0x5595b7c3a5f0 2024-05-10T04:04:12.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.596+0000 7f0fb8028b80 1 -- 172.21.15.150:0/2439096389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 msgr2=0x5595b7c3e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.596+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/2439096389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 0x5595b7c3e9b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800d700 tx=0x7f0f980950a0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.596+0000 7f0fb8028b80 1 -- 172.21.15.150:0/2439096389 shutdown_connections 2024-05-10T04:04:12.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.596+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/2439096389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 0x5595b7c3e9b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.596+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/2439096389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7b9bae0 0x5595b7c3e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.596+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/2439096389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7c39720 0x5595b7c4ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.596+0000 7f0fb8028b80 1 -- 172.21.15.150:0/2439096389 >> 172.21.15.150:0/2439096389 conn(0x5595b7c304d0 msgr2=0x5595b7c42d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.596+0000 7f0fb8028b80 1 -- 172.21.15.150:0/2439096389 shutdown_connections 2024-05-10T04:04:12.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.596+0000 7f0fb8028b80 1 -- 172.21.15.150:0/2439096389 wait complete. 2024-05-10T04:04:12.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.597+0000 7f0fb8028b80 1 -- start start 2024-05-10T04:04:12.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.597+0000 7f0fb8028b80 1 Processor -- start 2024-05-10T04:04:12.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.597+0000 7f0fb8028b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7b9bae0 0x5595b7c4a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.597+0000 7f0fb8028b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c39720 0x5595b7c48e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fb8028b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 0x5595b7c493a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fb8028b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b7c4afc0 con 0x5595b7c3a5f0 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fb8028b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b7c498e0 con 0x5595b7b9bae0 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fb8028b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b7c49aa0 con 0x5595b7c39720 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fb8028b80 10 client.0 _ll_register_callbacks cb 0x5595b7afebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fa6281700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 0x5595b7c493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fa5a80700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7b9bae0 0x5595b7c4a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fa5a80700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7b9bae0 0x5595b7c4a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:57596/0 (socket says 172.21.15.150:57596) 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fa6281700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 0x5595b7c493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50238/0 (socket says 172.21.15.150:50238) 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fa5a80700 1 -- 172.21.15.150:0/1161804261 learned_addr learned my addr 172.21.15.150:0/1161804261 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fa527f700 1 --2- 172.21.15.150:0/1161804261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c39720 0x5595b7c48e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fa5a80700 1 -- 172.21.15.150:0/1161804261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c39720 msgr2=0x5595b7c48e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fa5a80700 1 --2- 172.21.15.150:0/1161804261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c39720 0x5595b7c48e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fa5a80700 1 -- 172.21.15.150:0/1161804261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 msgr2=0x5595b7c493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fa5a80700 1 --2- 172.21.15.150:0/1161804261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 0x5595b7c493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fa5a80700 1 -- 172.21.15.150:0/1161804261 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa01a0350 con 0x5595b7b9bae0 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fa6281700 1 --2- 172.21.15.150:0/1161804261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 0x5595b7c493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fa5a80700 1 --2- 172.21.15.150:0/1161804261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7b9bae0 0x5595b7c4a7f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00fffb0 tx=0x7f0f9c00a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=534fcd09eafebef2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0f897fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0f897fa700 1 -- 172.21.15.150:0/1161804261 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c034210 con 0x5595b7b9bae0 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0f897fa700 1 -- 172.21.15.150:0/1161804261 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0f9c012070 con 0x5595b7b9bae0 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.598+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1161804261 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f980a18a0 con 0x5595b7b9bae0 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.599+0000 7f0f897fa700 1 -- 172.21.15.150:0/1161804261 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f9c034370 con 0x5595b7b9bae0 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.599+0000 7f0fb8028b80 10 client.25358 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.599+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1161804261 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5595b7986be0 con 0x5595b7b9bae0 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.599+0000 7f0f897fa700 1 -- 172.21.15.150:0/1161804261 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f980a18a0 con 0x5595b7b9bae0 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.599+0000 7f0f897fa700 1 -- 172.21.15.150:0/1161804261 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0f9c022a90 con 0x5595b7b9bae0 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.599+0000 7f0fb8028b80 10 client.25358 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.599+0000 7f0fb8028b80 10 client.25358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.599+0000 7f0fb8028b80 20 client.25358 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.599+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/1161804261 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5595b7c3af40 0x5595b7ca8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.599+0000 7f0fb8028b80 4 client.25358 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.599+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1161804261 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5595b7b51f40 con 0x5595b7c3af40 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.599+0000 7f0fa527f700 1 --2- 172.21.15.150:0/1161804261 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5595b7c3af40 0x5595b7ca8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.600+0000 7f0f897fa700 10 client.25358 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.600+0000 7f0fa527f700 1 --2- 172.21.15.150:0/1161804261 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5595b7c3af40 0x5595b7ca8ab0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41eb7b108e4163c server_cookie=3662ff5ae14a1c64 in_seq=0 out_seq=0 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0f897fa700 1 -- 172.21.15.150:0/1161804261 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5595b7b51f40 con 0x5595b7c3af40 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0f897fa700 10 client.25358 handle_command_reply: tid=0 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0fb8028b80 1 client.25358 shutdown 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1161804261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7b9bae0 msgr2=0x5595b7c4a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/1161804261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7b9bae0 0x5595b7c4a7f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00fffb0 tx=0x7f0f9c00a420 comp rx=0 tx=0).stop 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1161804261 shutdown_connections 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/1161804261 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5595b7c3af40 0x5595b7ca8ab0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/1161804261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5595b7c3a5f0 0x5595b7c493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/1161804261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595b7b9bae0 0x5595b7c4a7f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0fb8028b80 1 --2- 172.21.15.150:0/1161804261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595b7c39720 0x5595b7c48e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1161804261 >> 172.21.15.150:0/1161804261 conn(0x5595b7c304d0 msgr2=0x5595b7c41aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0f897fa700 0 client.25358 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:12.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1161804261 shutdown_connections 2024-05-10T04:04:12.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0fb8028b80 1 -- 172.21.15.150:0/1161804261 wait complete. 2024-05-10T04:04:12.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.601+0000 7f0fb8028b80 20 client.25358 trim_cache size 0 max 16384 2024-05-10T04:04:12.609 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:04:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: "return_code": 0, 2024-05-10T04:04:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_tag": "26aa8d37-1074-44cf-9164-91e15d03e612", 2024-05-10T04:04:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": "asynchronous" 2024-05-10T04:04:12.610 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:04:12.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.602+0000 7f0fa7ce4700 1 -- 172.21.15.150:0/928432824 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0f8c0a1260 msgr2=0x7f0f8c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.602+0000 7f0fa7ce4700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0f8c0a1260 0x7f0f8c0a3770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c006010 tx=0x7f0f9c003cf0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.602+0000 7f0fa7ce4700 1 -- 172.21.15.150:0/928432824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fa0101d80 msgr2=0x7f0fa019f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.602+0000 7f0fa7ce4700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fa0101d80 0x7f0fa019f3c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0f98007ee0 tx=0x7f0f9800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.602+0000 7f0fa7ce4700 1 -- 172.21.15.150:0/928432824 shutdown_connections 2024-05-10T04:04:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.602+0000 7f0fa7ce4700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0f8c0a1260 0x7f0f8c0a3770 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.602+0000 7f0fa7ce4700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fa0101d80 0x7f0fa019f3c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.602+0000 7f0fa7ce4700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fa01002c0 0x7f0fa0197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.602+0000 7f0fa7ce4700 1 --2- 172.21.15.150:0/928432824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fa01013e0 0x7f0fa0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.602+0000 7f0fa7ce4700 1 -- 172.21.15.150:0/928432824 >> 172.21.15.150:0/928432824 conn(0x7f0fa00fb8f0 msgr2=0x7f0fa0104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.602+0000 7f0fa7ce4700 1 -- 172.21.15.150:0/928432824 shutdown_connections 2024-05-10T04:04:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.602+0000 7f0fa7ce4700 1 -- 172.21.15.150:0/928432824 wait complete. 2024-05-10T04:04:12.617 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:04:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.800+0000 7feb7ac23700 1 -- 172.21.15.150:0/1288949297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb74101be0 msgr2=0x7feb74101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.800+0000 7feb7ac23700 1 --2- 172.21.15.150:0/1288949297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb74101be0 0x7feb74101ff0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7feb60009990 tx=0x7feb60009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.800+0000 7feb7ac23700 1 -- 172.21.15.150:0/1288949297 shutdown_connections 2024-05-10T04:04:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.800+0000 7feb7ac23700 1 --2- 172.21.15.150:0/1288949297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb74102580 0x7feb74108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.800+0000 7feb7ac23700 1 --2- 172.21.15.150:0/1288949297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb74100ac0 0x7feb74100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.800+0000 7feb7ac23700 1 --2- 172.21.15.150:0/1288949297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb74101be0 0x7feb74101ff0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.800+0000 7feb7ac23700 1 -- 172.21.15.150:0/1288949297 >> 172.21.15.150:0/1288949297 conn(0x7feb740fc0d0 msgr2=0x7feb740fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.800+0000 7feb7ac23700 1 -- 172.21.15.150:0/1288949297 shutdown_connections 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.800+0000 7feb7ac23700 1 -- 172.21.15.150:0/1288949297 wait complete. 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb7ac23700 1 Processor -- start 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb7ac23700 1 -- start start 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb7ac23700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb74100ac0 0x7feb74196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb7ac23700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb74101be0 0x7feb741972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb7ac23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb74102580 0x7feb7419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb7ac23700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7419e870 con 0x7feb74101be0 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb7ac23700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7419e9e0 con 0x7feb74100ac0 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb7ac23700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7419eba0 con 0x7feb74102580 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb73fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb74101be0 0x7feb741972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb73fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb74101be0 0x7feb741972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50258/0 (socket says 172.21.15.150:50258) 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb73fff700 1 -- 172.21.15.150:0/572536339 learned_addr learned my addr 172.21.15.150:0/572536339 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb789bf700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb74100ac0 0x7feb74196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb791c0700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb74102580 0x7feb7419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb73fff700 1 -- 172.21.15.150:0/572536339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb74102580 msgr2=0x7feb7419e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb73fff700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb74102580 0x7feb7419e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb73fff700 1 -- 172.21.15.150:0/572536339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb74100ac0 msgr2=0x7feb74196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb73fff700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb74100ac0 0x7feb74196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.801+0000 7feb73fff700 1 -- 172.21.15.150:0/572536339 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600095f0 con 0x7feb74101be0 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.802+0000 7feb73fff700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb74101be0 0x7feb741972d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feb600098e0 tx=0x7feb60009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.802+0000 7feb71ffb700 1 -- 172.21.15.150:0/572536339 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6001d070 con 0x7feb74101be0 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.802+0000 7feb71ffb700 1 -- 172.21.15.150:0/572536339 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb60021410 con 0x7feb74101be0 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.802+0000 7feb7ac23700 1 -- 172.21.15.150:0/572536339 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7419ee20 con 0x7feb74101be0 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.802+0000 7feb71ffb700 1 -- 172.21.15.150:0/572536339 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60017650 con 0x7feb74101be0 2024-05-10T04:04:12.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.802+0000 7feb7ac23700 1 -- 172.21.15.150:0/572536339 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7419f190 con 0x7feb74101be0 2024-05-10T04:04:12.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.804+0000 7feb71ffb700 1 -- 172.21.15.150:0/572536339 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7feb600177b0 con 0x7feb74101be0 2024-05-10T04:04:12.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.805+0000 7feb71ffb700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb640a1260 0x7feb640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.805+0000 7feb71ffb700 1 -- 172.21.15.150:0/572536339 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb600ce150 con 0x7feb74101be0 2024-05-10T04:04:12.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.805+0000 7feb789bf700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb640a1260 0x7feb640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.805+0000 7feb8af67b80 1 Processor -- start 2024-05-10T04:04:12.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.805+0000 7feb8af67b80 1 -- start start 2024-05-10T04:04:12.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb789bf700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb640a1260 0x7feb640a3770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feb74101ae0 tx=0x7feb68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb8af67b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d86a0 0x55c5255d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb8af67b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d9570 0x55c5255d9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb8af67b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d9ec0 0x55c5255e07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb8af67b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5254ff150 con 0x55c5255d86a0 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb8af67b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5254fcb10 con 0x55c5255d9ec0 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb8af67b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5254f8400 con 0x55c5255d9570 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb789bf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d86a0 0x55c5255d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb789bf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d86a0 0x55c5255d8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50260/0 (socket says 172.21.15.150:50260) 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb789bf700 1 -- 172.21.15.150:0/1967475560 learned_addr learned my addr 172.21.15.150:0/1967475560 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb73fff700 1 --2- 172.21.15.150:0/1967475560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d9570 0x55c5255d9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb791c0700 1 --2- 172.21.15.150:0/1967475560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d9ec0 0x55c5255e07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb789bf700 1 -- 172.21.15.150:0/1967475560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d9570 msgr2=0x55c5255d9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb789bf700 1 --2- 172.21.15.150:0/1967475560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d9570 0x55c5255d9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb789bf700 1 -- 172.21.15.150:0/1967475560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d9ec0 msgr2=0x55c5255e07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb789bf700 1 --2- 172.21.15.150:0/1967475560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d9ec0 0x55c5255e07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb789bf700 1 -- 172.21.15.150:0/1967475560 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5253793b0 con 0x55c5255d86a0 2024-05-10T04:04:12.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.806+0000 7feb789bf700 1 --2- 172.21.15.150:0/1967475560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d86a0 0x55c5255d8af0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7feb68011d20 tx=0x7feb68014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76985e2a403e43a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.807+0000 7feb5effd700 1 -- 172.21.15.150:0/1967475560 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6800ed20 con 0x55c5255d86a0 2024-05-10T04:04:12.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.807+0000 7feb5effd700 1 -- 172.21.15.150:0/1967475560 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb68021070 con 0x55c5255d86a0 2024-05-10T04:04:12.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.807+0000 7feb5effd700 1 -- 172.21.15.150:0/1967475560 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6801d670 con 0x55c5255d86a0 2024-05-10T04:04:12.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.807+0000 7feb8af67b80 1 -- 172.21.15.150:0/1967475560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d86a0 msgr2=0x55c5255d8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.807+0000 7feb8af67b80 1 --2- 172.21.15.150:0/1967475560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d86a0 0x55c5255d8af0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7feb68011d20 tx=0x7feb68014000 comp rx=0 tx=0).stop 2024-05-10T04:04:12.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.807+0000 7feb8af67b80 1 -- 172.21.15.150:0/1967475560 shutdown_connections 2024-05-10T04:04:12.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.807+0000 7feb8af67b80 1 --2- 172.21.15.150:0/1967475560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d9ec0 0x55c5255e07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.807+0000 7feb8af67b80 1 --2- 172.21.15.150:0/1967475560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d86a0 0x55c5255d8af0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.807+0000 7feb8af67b80 1 --2- 172.21.15.150:0/1967475560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d9570 0x55c5255d9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.807+0000 7feb8af67b80 1 -- 172.21.15.150:0/1967475560 >> 172.21.15.150:0/1967475560 conn(0x55c52553aa60 msgr2=0x55c5255d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.808+0000 7feb8af67b80 1 -- 172.21.15.150:0/1967475560 shutdown_connections 2024-05-10T04:04:12.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.808+0000 7feb8af67b80 1 -- 172.21.15.150:0/1967475560 wait complete. 2024-05-10T04:04:12.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.808+0000 7feb8af67b80 1 -- start start 2024-05-10T04:04:12.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.808+0000 7feb8af67b80 1 Processor -- start 2024-05-10T04:04:12.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.808+0000 7feb8af67b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d86a0 0x55c5255e26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.808+0000 7feb8af67b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d9570 0x55c5255e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.808+0000 7feb8af67b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9ec0 0x55c5255e9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.808+0000 7feb8af67b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c525494560 con 0x55c5255d9ec0 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.808+0000 7feb8af67b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c525494200 con 0x55c5255d86a0 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.808+0000 7feb8af67b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c52537b6a0 con 0x55c5255d9570 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.808+0000 7feb8af67b80 10 client.0 _ll_register_callbacks cb 0x55c52549db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.808+0000 7feb791c0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9ec0 0x55c5255e9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb791c0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9ec0 0x55c5255e9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50264/0 (socket says 172.21.15.150:50264) 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb791c0700 1 -- 172.21.15.150:0/4012788393 learned_addr learned my addr 172.21.15.150:0/4012788393 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb789bf700 1 --2- 172.21.15.150:0/4012788393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d86a0 0x55c5255e26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb73fff700 1 --2- 172.21.15.150:0/4012788393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d9570 0x55c5255e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb791c0700 1 -- 172.21.15.150:0/4012788393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d9570 msgr2=0x55c5255e2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb791c0700 1 --2- 172.21.15.150:0/4012788393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d9570 0x55c5255e2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb791c0700 1 -- 172.21.15.150:0/4012788393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d86a0 msgr2=0x55c5255e26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb791c0700 1 --2- 172.21.15.150:0/4012788393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d86a0 0x55c5255e26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb791c0700 1 -- 172.21.15.150:0/4012788393 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb68017070 con 0x55c5255d9ec0 2024-05-10T04:04:12.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb791c0700 1 --2- 172.21.15.150:0/4012788393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9ec0 0x55c5255e9b60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7feb6c004320 tx=0x7feb6c004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74591d8ebbe477ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb53fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:12.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb53fff700 1 -- 172.21.15.150:0/4012788393 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c003b10 con 0x55c5255d9ec0 2024-05-10T04:04:12.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb53fff700 1 -- 172.21.15.150:0/4012788393 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb6c007910 con 0x55c5255d9ec0 2024-05-10T04:04:12.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb8af67b80 1 -- 172.21.15.150:0/4012788393 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c525451270 con 0x55c5255d9ec0 2024-05-10T04:04:12.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb53fff700 1 -- 172.21.15.150:0/4012788393 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb6c0048e0 con 0x55c5255d9ec0 2024-05-10T04:04:12.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb8af67b80 10 client.15648 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:12.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.809+0000 7feb8af67b80 1 -- 172.21.15.150:0/4012788393 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c525325b60 con 0x55c5255d9ec0 2024-05-10T04:04:12.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.810+0000 7feb53fff700 1 -- 172.21.15.150:0/4012788393 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c525451270 con 0x55c5255d9ec0 2024-05-10T04:04:12.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.810+0000 7feb53fff700 1 -- 172.21.15.150:0/4012788393 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feb6c016cf0 con 0x55c5255d9ec0 2024-05-10T04:04:12.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.810+0000 7feb8af67b80 10 client.15648 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:12.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.810+0000 7feb8af67b80 10 client.15648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:12.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.810+0000 7feb8af67b80 20 client.15648 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.810+0000 7feb8af67b80 1 --2- 172.21.15.150:0/4012788393 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c5255ee370 0x55c5255f07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.810+0000 7feb8af67b80 4 client.15648 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.810+0000 7feb8af67b80 1 -- 172.21.15.150:0/4012788393 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5255a7780 con 0x55c5255ee370 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.810+0000 7feb789bf700 1 --2- 172.21.15.150:0/4012788393 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c5255ee370 0x55c5255f07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.810+0000 7feb789bf700 1 --2- 172.21.15.150:0/4012788393 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c5255ee370 0x55c5255f07c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f22e6b2bc2464946 server_cookie=50b6a7c54ca7e8fb in_seq=0 out_seq=0 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.810+0000 7feb53fff700 10 client.15648 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb53fff700 1 -- 172.21.15.150:0/4012788393 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5255a7780 con 0x55c5255ee370 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb53fff700 10 client.15648 handle_command_reply: tid=0 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb8af67b80 1 client.15648 shutdown 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb8af67b80 1 -- 172.21.15.150:0/4012788393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9ec0 msgr2=0x55c5255e9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb8af67b80 1 --2- 172.21.15.150:0/4012788393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9ec0 0x55c5255e9b60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7feb6c004320 tx=0x7feb6c004350 comp rx=0 tx=0).stop 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb8af67b80 1 -- 172.21.15.150:0/4012788393 shutdown_connections 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb8af67b80 1 --2- 172.21.15.150:0/4012788393 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c5255ee370 0x55c5255f07c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb8af67b80 1 --2- 172.21.15.150:0/4012788393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9ec0 0x55c5255e9b60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb53fff700 0 client.15648 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb8af67b80 1 --2- 172.21.15.150:0/4012788393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d86a0 0x55c5255e26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb8af67b80 1 --2- 172.21.15.150:0/4012788393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d9570 0x55c5255e2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb8af67b80 1 -- 172.21.15.150:0/4012788393 >> 172.21.15.150:0/4012788393 conn(0x55c52553aa60 msgr2=0x55c5255d2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb8af67b80 1 -- 172.21.15.150:0/4012788393 shutdown_connections 2024-05-10T04:04:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.811+0000 7feb8af67b80 1 -- 172.21.15.150:0/4012788393 wait complete. 2024-05-10T04:04:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.812+0000 7feb8af67b80 20 client.15648 trim_cache size 0 max 16384 2024-05-10T04:04:12.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.829+0000 7feb8af67b80 1 Processor -- start 2024-05-10T04:04:12.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.829+0000 7feb8af67b80 1 -- start start 2024-05-10T04:04:12.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.829+0000 7feb8af67b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c52553aa60 0x55c5255e1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.829+0000 7feb8af67b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d86a0 0x55c5255dd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.829+0000 7feb8af67b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9570 0x55c5255dd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.829+0000 7feb8af67b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5255e06d0 con 0x55c5255d9570 2024-05-10T04:04:12.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.829+0000 7feb8af67b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5255e0a50 con 0x55c52553aa60 2024-05-10T04:04:12.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.829+0000 7feb8af67b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5255d2f90 con 0x55c5255d86a0 2024-05-10T04:04:12.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.829+0000 7feb789bf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c52553aa60 0x55c5255e1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.829+0000 7feb789bf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c52553aa60 0x55c5255e1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:57648/0 (socket says 172.21.15.150:57648) 2024-05-10T04:04:12.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.829+0000 7feb791c0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9570 0x55c5255dd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.829+0000 7feb789bf700 1 -- 172.21.15.150:0/1177705899 learned_addr learned my addr 172.21.15.150:0/1177705899 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:12.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.830+0000 7feb73fff700 1 --2- 172.21.15.150:0/1177705899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d86a0 0x55c5255dd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.830+0000 7feb789bf700 1 -- 172.21.15.150:0/1177705899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d86a0 msgr2=0x55c5255dd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.830+0000 7feb789bf700 1 --2- 172.21.15.150:0/1177705899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d86a0 0x55c5255dd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.830+0000 7feb789bf700 1 -- 172.21.15.150:0/1177705899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9570 msgr2=0x55c5255dd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.830+0000 7feb789bf700 1 --2- 172.21.15.150:0/1177705899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9570 0x55c5255dd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.830+0000 7feb789bf700 1 -- 172.21.15.150:0/1177705899 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5254be560 con 0x55c52553aa60 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.830+0000 7feb73fff700 1 --2- 172.21.15.150:0/1177705899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d86a0 0x55c5255dd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.830+0000 7feb789bf700 1 --2- 172.21.15.150:0/1177705899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c52553aa60 0x55c5255e1ca0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55c5255d8380 tx=0x7feb6800ea20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dbc053acf081acd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.830+0000 7feb5f7fe700 1 -- 172.21.15.150:0/1177705899 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6800ed80 con 0x55c52553aa60 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.830+0000 7feb5f7fe700 1 -- 172.21.15.150:0/1177705899 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb6801cc90 con 0x55c52553aa60 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.830+0000 7feb5f7fe700 1 -- 172.21.15.150:0/1177705899 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68005540 con 0x55c52553aa60 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.831+0000 7feb8af67b80 1 -- 172.21.15.150:0/1177705899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c52553aa60 msgr2=0x55c5255e1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.831+0000 7feb8af67b80 1 --2- 172.21.15.150:0/1177705899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c52553aa60 0x55c5255e1ca0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55c5255d8380 tx=0x7feb6800ea20 comp rx=0 tx=0).stop 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.831+0000 7feb8af67b80 1 -- 172.21.15.150:0/1177705899 shutdown_connections 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.831+0000 7feb8af67b80 1 --2- 172.21.15.150:0/1177705899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9570 0x55c5255dd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.831+0000 7feb8af67b80 1 --2- 172.21.15.150:0/1177705899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c52553aa60 0x55c5255e1ca0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.831+0000 7feb8af67b80 1 --2- 172.21.15.150:0/1177705899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c5255d86a0 0x55c5255dd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.831+0000 7feb8af67b80 1 -- 172.21.15.150:0/1177705899 >> 172.21.15.150:0/1177705899 conn(0x55c5255cf450 msgr2=0x55c5255d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.831+0000 7feb8af67b80 1 -- 172.21.15.150:0/1177705899 shutdown_connections 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.831+0000 7feb8af67b80 1 -- 172.21.15.150:0/1177705899 wait complete. 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.831+0000 7feb8af67b80 1 -- start start 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.831+0000 7feb8af67b80 1 Processor -- start 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb8af67b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c52553aa60 0x55c52563ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb8af67b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d86a0 0x55c52563efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb8af67b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9570 0x55c525645c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb8af67b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c525646140 con 0x55c5255d9570 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb8af67b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c525646280 con 0x55c5255d86a0 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb8af67b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c525646410 con 0x55c52553aa60 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb8af67b80 10 client.0 _ll_register_callbacks cb 0x55c52549db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb791c0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9570 0x55c525645c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb791c0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9570 0x55c525645c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50278/0 (socket says 172.21.15.150:50278) 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb791c0700 1 -- 172.21.15.150:0/2949282608 learned_addr learned my addr 172.21.15.150:0/2949282608 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:12.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb73fff700 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d86a0 0x55c52563efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb789bf700 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c52553aa60 0x55c52563ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb791c0700 1 -- 172.21.15.150:0/2949282608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c52553aa60 msgr2=0x55c52563ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb791c0700 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c52553aa60 0x55c52563ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb791c0700 1 -- 172.21.15.150:0/2949282608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d86a0 msgr2=0x55c52563efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb791c0700 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d86a0 0x55c52563efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb791c0700 1 -- 172.21.15.150:0/2949282608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6802a040 con 0x55c5255d9570 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb73fff700 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d86a0 0x55c52563efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb789bf700 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c52553aa60 0x55c52563ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb791c0700 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9570 0x55c525645c00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55c5255d36c0 tx=0x7feb6c004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1799b6707d3093 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.832+0000 7feb53fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb53fff700 1 -- 172.21.15.150:0/2949282608 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c01ecf0 con 0x55c5255d9570 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb53fff700 1 -- 172.21.15.150:0/2949282608 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb6c01e6a0 con 0x55c5255d9570 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb8af67b80 1 -- 172.21.15.150:0/2949282608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb68021070 con 0x55c5255d9570 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb53fff700 1 -- 172.21.15.150:0/2949282608 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb6c0091e0 con 0x55c5255d9570 2024-05-10T04:04:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb8af67b80 10 client.15654 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:12.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb8af67b80 1 -- 172.21.15.150:0/2949282608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c52537dcd0 con 0x55c5255d9570 2024-05-10T04:04:12.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb53fff700 1 -- 172.21.15.150:0/2949282608 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb68021070 con 0x55c5255d9570 2024-05-10T04:04:12.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb53fff700 1 -- 172.21.15.150:0/2949282608 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feb6c01f5e0 con 0x55c5255d9570 2024-05-10T04:04:12.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb8af67b80 10 client.15654 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:12.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb8af67b80 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:12.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb8af67b80 20 client.15654 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:12.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb8af67b80 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c5255d9ec0 0x55c525649d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:12.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb8af67b80 4 client.15654 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:04:12.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb8af67b80 1 -- 172.21.15.150:0/2949282608 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c5254f0ec0 con 0x55c5255d9ec0 2024-05-10T04:04:12.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.833+0000 7feb789bf700 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c5255d9ec0 0x55c525649d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:12.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb53fff700 10 client.15654 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:12.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb789bf700 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c5255d9ec0 0x55c525649d40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74e2fccff138811 server_cookie=a027920e8b305529 in_seq=0 out_seq=0 2024-05-10T04:04:12.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb53fff700 1 -- 172.21.15.150:0/2949282608 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c5254f0ec0 con 0x55c5255d9ec0 2024-05-10T04:04:12.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb53fff700 10 client.15654 handle_command_reply: tid=0 2024-05-10T04:04:12.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb8af67b80 1 client.15654 shutdown 2024-05-10T04:04:12.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb8af67b80 1 -- 172.21.15.150:0/2949282608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9570 msgr2=0x55c525645c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb8af67b80 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9570 0x55c525645c00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55c5255d36c0 tx=0x7feb6c004b10 comp rx=0 tx=0).stop 2024-05-10T04:04:12.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb8af67b80 1 -- 172.21.15.150:0/2949282608 shutdown_connections 2024-05-10T04:04:12.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb8af67b80 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c5255d9ec0 0x55c525649d40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb8af67b80 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5255d9570 0x55c525645c00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb8af67b80 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c52553aa60 0x55c52563ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb53fff700 0 client.15654 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:12.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb8af67b80 1 --2- 172.21.15.150:0/2949282608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5255d86a0 0x55c52563efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb8af67b80 1 -- 172.21.15.150:0/2949282608 >> 172.21.15.150:0/2949282608 conn(0x55c5255cf450 msgr2=0x55c5255d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb8af67b80 1 -- 172.21.15.150:0/2949282608 shutdown_connections 2024-05-10T04:04:12.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.834+0000 7feb8af67b80 1 -- 172.21.15.150:0/2949282608 wait complete. 2024-05-10T04:04:12.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.835+0000 7feb8af67b80 20 client.15654 trim_cache size 0 max 16384 2024-05-10T04:04:12.844 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:04:12.844 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T04:04:12.844 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": { 2024-05-10T04:04:12.844 INFO:teuthology.orchestra.run.smithi150.stdout: "26aa8d37-1074-44cf-9164-91e15d03e612": { 2024-05-10T04:04:12.844 INFO:teuthology.orchestra.run.smithi150.stdout: "path": "/", 2024-05-10T04:04:12.844 INFO:teuthology.orchestra.run.smithi150.stdout: "tag": "26aa8d37-1074-44cf-9164-91e15d03e612", 2024-05-10T04:04:12.844 INFO:teuthology.orchestra.run.smithi150.stdout: "options": "recursive,force" 2024-05-10T04:04:12.844 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:04:12.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:04:12.845 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:04:12.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.835+0000 7feb7ac23700 1 -- 172.21.15.150:0/572536339 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb640a1260 msgr2=0x7feb640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.835+0000 7feb7ac23700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb640a1260 0x7feb640a3770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feb74101ae0 tx=0x7feb68009260 comp rx=0 tx=0).stop 2024-05-10T04:04:12.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.835+0000 7feb7ac23700 1 -- 172.21.15.150:0/572536339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb74101be0 msgr2=0x7feb741972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:12.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.835+0000 7feb7ac23700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb74101be0 0x7feb741972d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feb600098e0 tx=0x7feb60009580 comp rx=0 tx=0).stop 2024-05-10T04:04:12.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.835+0000 7feb7ac23700 1 -- 172.21.15.150:0/572536339 shutdown_connections 2024-05-10T04:04:12.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.835+0000 7feb7ac23700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7feb640a1260 0x7feb640a3770 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.835+0000 7feb7ac23700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb74102580 0x7feb7419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.835+0000 7feb7ac23700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb74100ac0 0x7feb74196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.835+0000 7feb7ac23700 1 --2- 172.21.15.150:0/572536339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb74101be0 0x7feb741972d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:12.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.835+0000 7feb7ac23700 1 -- 172.21.15.150:0/572536339 >> 172.21.15.150:0/572536339 conn(0x7feb740fc0d0 msgr2=0x7feb74104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:12.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.835+0000 7feb7ac23700 1 -- 172.21.15.150:0/572536339 shutdown_connections 2024-05-10T04:04:12.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:12.835+0000 7feb7ac23700 1 -- 172.21.15.150:0/572536339 wait complete. 2024-05-10T04:04:12.850 INFO:tasks.cephfs.filesystem:scrub status for tag:26aa8d37-1074-44cf-9164-91e15d03e612 - {'path': '/', 'tag': '26aa8d37-1074-44cf-9164-91e15d03e612', 'options': 'recursive,force'} 2024-05-10T04:04:13.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:13 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1732794535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:04:13.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:13 smithi150 ceph-mon[42477]: scrub queued for path: / 2024-05-10T04:04:13.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:13 smithi150 ceph-mon[42477]: scrub summary: idle+waiting paths [/] 2024-05-10T04:04:13.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:13 smithi150 ceph-mon[42477]: scrub summary: active paths [/] 2024-05-10T04:04:13.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:13 smithi150 ceph-mon[42477]: scrub summary: idle+waiting paths [/] 2024-05-10T04:04:13.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:13 smithi150 ceph-mon[42477]: pgmap v279: 129 pgs: 129 active+clean; 649 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-10T04:04:13.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:13 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1732794535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:04:13.798 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:13 smithi181 ceph-mon[46602]: scrub queued for path: / 2024-05-10T04:04:13.799 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:13 smithi181 ceph-mon[46602]: scrub summary: idle+waiting paths [/] 2024-05-10T04:04:13.799 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:13 smithi181 ceph-mon[46602]: scrub summary: active paths [/] 2024-05-10T04:04:13.799 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:13 smithi181 ceph-mon[46602]: scrub summary: idle+waiting paths [/] 2024-05-10T04:04:13.799 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:13 smithi181 ceph-mon[46602]: pgmap v279: 129 pgs: 129 active+clean; 649 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-10T04:04:13.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:13 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1732794535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:04:13.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:13 smithi161 ceph-mon[46918]: scrub queued for path: / 2024-05-10T04:04:13.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:13 smithi161 ceph-mon[46918]: scrub summary: idle+waiting paths [/] 2024-05-10T04:04:13.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:13 smithi161 ceph-mon[46918]: scrub summary: active paths [/] 2024-05-10T04:04:13.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:13 smithi161 ceph-mon[46918]: scrub summary: idle+waiting paths [/] 2024-05-10T04:04:13.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:13 smithi161 ceph-mon[46918]: pgmap v279: 129 pgs: 129 active+clean; 649 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-10T04:04:16.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:16 smithi150 ceph-mon[42477]: pgmap v280: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-10T04:04:16.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:16 smithi181 ceph-mon[46602]: pgmap v280: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-10T04:04:16.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:16 smithi161 ceph-mon[46918]: pgmap v280: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-10T04:04:18.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:18 smithi150 ceph-mon[42477]: pgmap v281: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-10T04:04:18.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:18 smithi181 ceph-mon[46602]: pgmap v281: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-10T04:04:18.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:18 smithi161 ceph-mon[46918]: pgmap v281: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-10T04:04:20.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:20 smithi150 ceph-mon[42477]: pgmap v282: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-10T04:04:20.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:20 smithi181 ceph-mon[46602]: pgmap v282: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-10T04:04:20.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:20 smithi161 ceph-mon[46918]: pgmap v282: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-10T04:04:22.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:22 smithi150 ceph-mon[42477]: pgmap v283: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-10T04:04:22.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:22 smithi181 ceph-mon[46602]: pgmap v283: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-10T04:04:22.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:22 smithi161 ceph-mon[46918]: pgmap v283: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-10T04:04:24.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:24 smithi150 ceph-mon[42477]: pgmap v284: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-10T04:04:24.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:24 smithi181 ceph-mon[46602]: pgmap v284: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-10T04:04:24.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:24 smithi161 ceph-mon[46918]: pgmap v284: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-10T04:04:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.603+0000 7f635a2ba700 1 -- 172.21.15.150:0/2573097701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63541025a0 msgr2=0x7f6354108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.603+0000 7f635a2ba700 1 --2- 172.21.15.150:0/2573097701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63541025a0 0x7f6354108e50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f634c00b0a0 tx=0x7f634c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.603+0000 7f635a2ba700 1 -- 172.21.15.150:0/2573097701 shutdown_connections 2024-05-10T04:04:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.603+0000 7f635a2ba700 1 --2- 172.21.15.150:0/2573097701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63541025a0 0x7f6354108e50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.603+0000 7f635a2ba700 1 --2- 172.21.15.150:0/2573097701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6354100ae0 0x7f6354100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.603+0000 7f635a2ba700 1 --2- 172.21.15.150:0/2573097701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6354101c00 0x7f6354102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.603+0000 7f635a2ba700 1 -- 172.21.15.150:0/2573097701 >> 172.21.15.150:0/2573097701 conn(0x7f63540fc0f0 msgr2=0x7f63540fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.603+0000 7f635a2ba700 1 -- 172.21.15.150:0/2573097701 shutdown_connections 2024-05-10T04:04:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.603+0000 7f635a2ba700 1 -- 172.21.15.150:0/2573097701 wait complete. 2024-05-10T04:04:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.604+0000 7f635a2ba700 1 Processor -- start 2024-05-10T04:04:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.604+0000 7f635a2ba700 1 -- start start 2024-05-10T04:04:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.604+0000 7f635a2ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6354100ae0 0x7f6354198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.604+0000 7f635a2ba700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6354101c00 0x7f6354198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.604+0000 7f635a2ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63541025a0 0x7f635419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.604+0000 7f635a2ba700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635419fbe0 con 0x7f6354100ae0 2024-05-10T04:04:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.604+0000 7f635a2ba700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635419fd20 con 0x7f6354101c00 2024-05-10T04:04:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.604+0000 7f635a2ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635419feb0 con 0x7f63541025a0 2024-05-10T04:04:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.604+0000 7f6353fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6354100ae0 0x7f6354198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f6353fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6354100ae0 0x7f6354198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43894/0 (socket says 172.21.15.150:43894) 2024-05-10T04:04:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f6353fff700 1 -- 172.21.15.150:0/2344506105 learned_addr learned my addr 172.21.15.150:0/2344506105 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f63537fe700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6354101c00 0x7f6354198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f6358857700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63541025a0 0x7f635419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f6353fff700 1 -- 172.21.15.150:0/2344506105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63541025a0 msgr2=0x7f635419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f6353fff700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63541025a0 0x7f635419f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f6353fff700 1 -- 172.21.15.150:0/2344506105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6354101c00 msgr2=0x7f6354198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f6353fff700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6354101c00 0x7f6354198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f6353fff700 1 -- 172.21.15.150:0/2344506105 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c009d00 con 0x7f6354100ae0 2024-05-10T04:04:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f6353fff700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6354100ae0 0x7f6354198140 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f634400d980 tx=0x7f634400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f63517fa700 1 -- 172.21.15.150:0/2344506105 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344004220 con 0x7f6354100ae0 2024-05-10T04:04:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f635a2ba700 1 -- 172.21.15.150:0/2344506105 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63541a0190 con 0x7f6354100ae0 2024-05-10T04:04:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f63517fa700 1 -- 172.21.15.150:0/2344506105 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6344004380 con 0x7f6354100ae0 2024-05-10T04:04:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f63517fa700 1 -- 172.21.15.150:0/2344506105 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344003bf0 con 0x7f6354100ae0 2024-05-10T04:04:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.605+0000 7f635a2ba700 1 -- 172.21.15.150:0/2344506105 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63541a06a0 con 0x7f6354100ae0 2024-05-10T04:04:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.608+0000 7f63517fa700 1 -- 172.21.15.150:0/2344506105 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6344003d50 con 0x7f6354100ae0 2024-05-10T04:04:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.609+0000 7f63517fa700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f633c0a11b0 0x7f633c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.609+0000 7f63517fa700 1 -- 172.21.15.150:0/2344506105 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f63440cc980 con 0x7f6354100ae0 2024-05-10T04:04:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.609+0000 7f63537fe700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f633c0a11b0 0x7f633c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f636a5feb80 1 Processor -- start 2024-05-10T04:04:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f636a5feb80 1 -- start start 2024-05-10T04:04:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f63537fe700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f633c0a11b0 0x7f633c0a36c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f63541991e0 tx=0x7f634800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f636a5feb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c1180 0x5602461c15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f636a5feb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c2050 0x5602461c2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f636a5feb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c29a0 0x5602461c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f636a5feb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602460e80f0 con 0x5602461c2050 2024-05-10T04:04:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f636a5feb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602460e5ab0 con 0x5602461c1180 2024-05-10T04:04:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f636a5feb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602460e13a0 con 0x5602461c29a0 2024-05-10T04:04:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f63537fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c2050 0x5602461c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f63537fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c2050 0x5602461c2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43908/0 (socket says 172.21.15.150:43908) 2024-05-10T04:04:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f63537fe700 1 -- 172.21.15.150:0/1040680909 learned_addr learned my addr 172.21.15.150:0/1040680909 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f6353fff700 1 --2- 172.21.15.150:0/1040680909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c1180 0x5602461c15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f6358857700 1 --2- 172.21.15.150:0/1040680909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c29a0 0x5602461c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f63537fe700 1 -- 172.21.15.150:0/1040680909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c29a0 msgr2=0x5602461c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f63537fe700 1 --2- 172.21.15.150:0/1040680909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c29a0 0x5602461c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f63537fe700 1 -- 172.21.15.150:0/1040680909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c1180 msgr2=0x5602461c15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f63537fe700 1 --2- 172.21.15.150:0/1040680909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c1180 0x5602461c15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f63537fe700 1 -- 172.21.15.150:0/1040680909 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560245f62350 con 0x5602461c2050 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f6353fff700 1 --2- 172.21.15.150:0/1040680909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c1180 0x5602461c15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.610+0000 7f63537fe700 1 --2- 172.21.15.150:0/1040680909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c2050 0x5602461c2460 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6348010f70 tx=0x7f6348013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98709183ab3fcac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.611+0000 7f633a7fc700 1 -- 172.21.15.150:0/1040680909 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634801e620 con 0x5602461c2050 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.611+0000 7f633a7fc700 1 -- 172.21.15.150:0/1040680909 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6348015070 con 0x5602461c2050 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.611+0000 7f633a7fc700 1 -- 172.21.15.150:0/1040680909 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634801c9c0 con 0x5602461c2050 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.611+0000 7f636a5feb80 1 -- 172.21.15.150:0/1040680909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c2050 msgr2=0x5602461c2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.611+0000 7f636a5feb80 1 --2- 172.21.15.150:0/1040680909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c2050 0x5602461c2460 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6348010f70 tx=0x7f6348013000 comp rx=0 tx=0).stop 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.612+0000 7f636a5feb80 1 -- 172.21.15.150:0/1040680909 shutdown_connections 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.612+0000 7f636a5feb80 1 --2- 172.21.15.150:0/1040680909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c29a0 0x5602461c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.612+0000 7f636a5feb80 1 --2- 172.21.15.150:0/1040680909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c1180 0x5602461c15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.612+0000 7f636a5feb80 1 --2- 172.21.15.150:0/1040680909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c2050 0x5602461c2460 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.612+0000 7f636a5feb80 1 -- 172.21.15.150:0/1040680909 >> 172.21.15.150:0/1040680909 conn(0x560246123a00 msgr2=0x5602461bba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.612+0000 7f636a5feb80 1 -- 172.21.15.150:0/1040680909 shutdown_connections 2024-05-10T04:04:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.612+0000 7f636a5feb80 1 -- 172.21.15.150:0/1040680909 wait complete. 2024-05-10T04:04:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.612+0000 7f636a5feb80 1 -- start start 2024-05-10T04:04:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.612+0000 7f636a5feb80 1 Processor -- start 2024-05-10T04:04:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.612+0000 7f636a5feb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c1180 0x5602461cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.612+0000 7f636a5feb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c2050 0x5602461cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f636a5feb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c29a0 0x5602461d25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f636a5feb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56024607d500 con 0x5602461c29a0 2024-05-10T04:04:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f636a5feb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56024607d1a0 con 0x5602461c2050 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f636a5feb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560245f64640 con 0x5602461c1180 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f636a5feb80 10 client.0 _ll_register_callbacks cb 0x560246086ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f6358857700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c29a0 0x5602461d25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f6358857700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c29a0 0x5602461d25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43922/0 (socket says 172.21.15.150:43922) 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f6358857700 1 -- 172.21.15.150:0/1603562884 learned_addr learned my addr 172.21.15.150:0/1603562884 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f63537fe700 1 --2- 172.21.15.150:0/1603562884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c2050 0x5602461cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f6353fff700 1 --2- 172.21.15.150:0/1603562884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c1180 0x5602461cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f6358857700 1 -- 172.21.15.150:0/1603562884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c1180 msgr2=0x5602461cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f6358857700 1 --2- 172.21.15.150:0/1603562884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c1180 0x5602461cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f6358857700 1 -- 172.21.15.150:0/1603562884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c2050 msgr2=0x5602461cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f6358857700 1 --2- 172.21.15.150:0/1603562884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c2050 0x5602461cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f6358857700 1 -- 172.21.15.150:0/1603562884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6348010c20 con 0x5602461c29a0 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f6358857700 1 --2- 172.21.15.150:0/1603562884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c29a0 0x5602461d25f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f634c004c80 tx=0x7f634c004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9204d2592d0aba0c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f63337fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f63337fe700 1 -- 172.21.15.150:0/1603562884 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c00e040 con 0x5602461c29a0 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f63337fe700 1 -- 172.21.15.150:0/1603562884 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f634c01c440 con 0x5602461c29a0 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f636a5feb80 1 -- 172.21.15.150:0/1603562884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56024603a210 con 0x5602461c29a0 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.613+0000 7f63337fe700 1 -- 172.21.15.150:0/1603562884 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f634c009780 con 0x5602461c29a0 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.614+0000 7f636a5feb80 10 client.15666 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.614+0000 7f636a5feb80 1 -- 172.21.15.150:0/1603562884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560245f0eb00 con 0x5602461c29a0 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.614+0000 7f63337fe700 1 -- 172.21.15.150:0/1603562884 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56024603a210 con 0x5602461c29a0 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.614+0000 7f63337fe700 1 -- 172.21.15.150:0/1603562884 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f634c025060 con 0x5602461c29a0 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.614+0000 7f636a5feb80 10 client.15666 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.614+0000 7f636a5feb80 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.614+0000 7f636a5feb80 20 client.15666 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.614+0000 7f636a5feb80 1 --2- 172.21.15.150:0/1603562884 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602461d6e20 0x5602461d9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.614+0000 7f636a5feb80 4 client.15666 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:04:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.614+0000 7f636a5feb80 1 -- 172.21.15.150:0/1603562884 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560246190720 con 0x5602461d6e20 2024-05-10T04:04:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.614+0000 7f6353fff700 1 --2- 172.21.15.150:0/1603562884 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602461d6e20 0x5602461d9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.615+0000 7f63337fe700 10 client.15666 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.615+0000 7f6353fff700 1 --2- 172.21.15.150:0/1603562884 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602461d6e20 0x5602461d9270 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=208835cf9ae96126 server_cookie=6bec3e11960e9c81 in_seq=0 out_seq=0 2024-05-10T04:04:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.615+0000 7f63337fe700 1 -- 172.21.15.150:0/1603562884 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560246190720 con 0x5602461d6e20 2024-05-10T04:04:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.615+0000 7f63337fe700 10 client.15666 handle_command_reply: tid=0 2024-05-10T04:04:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.615+0000 7f636a5feb80 1 client.15666 shutdown 2024-05-10T04:04:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.616+0000 7f636a5feb80 1 -- 172.21.15.150:0/1603562884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c29a0 msgr2=0x5602461d25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.616+0000 7f636a5feb80 1 --2- 172.21.15.150:0/1603562884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c29a0 0x5602461d25f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f634c004c80 tx=0x7f634c004cb0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.616+0000 7f636a5feb80 1 -- 172.21.15.150:0/1603562884 shutdown_connections 2024-05-10T04:04:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.616+0000 7f636a5feb80 1 --2- 172.21.15.150:0/1603562884 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602461d6e20 0x5602461d9270 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.616+0000 7f636a5feb80 1 --2- 172.21.15.150:0/1603562884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c29a0 0x5602461d25f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.616+0000 7f636a5feb80 1 --2- 172.21.15.150:0/1603562884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c1180 0x5602461cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.616+0000 7f63337fe700 0 client.15666 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.616+0000 7f636a5feb80 1 --2- 172.21.15.150:0/1603562884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c2050 0x5602461cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.616+0000 7f636a5feb80 1 -- 172.21.15.150:0/1603562884 >> 172.21.15.150:0/1603562884 conn(0x560246123a00 msgr2=0x5602461bba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.616+0000 7f636a5feb80 1 -- 172.21.15.150:0/1603562884 shutdown_connections 2024-05-10T04:04:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.616+0000 7f636a5feb80 1 -- 172.21.15.150:0/1603562884 wait complete. 2024-05-10T04:04:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.616+0000 7f636a5feb80 20 client.15666 trim_cache size 0 max 16384 2024-05-10T04:04:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f636a5feb80 1 Processor -- start 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f636a5feb80 1 -- start start 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f636a5feb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 0x5602461ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f636a5feb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c2050 0x5602461d25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f636a5feb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c29a0 0x5602461c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f636a5feb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602461d9980 con 0x5602461c1180 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f636a5feb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602461bd200 con 0x5602461c29a0 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f636a5feb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602461c1a10 con 0x5602461c2050 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f6353fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 0x5602461ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f6353fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 0x5602461ddb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43938/0 (socket says 172.21.15.150:43938) 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f6353fff700 1 -- 172.21.15.150:0/2343301428 learned_addr learned my addr 172.21.15.150:0/2343301428 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f63537fe700 1 --2- 172.21.15.150:0/2343301428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c2050 0x5602461d25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f6358857700 1 --2- 172.21.15.150:0/2343301428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c29a0 0x5602461c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f6353fff700 1 -- 172.21.15.150:0/2343301428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c2050 msgr2=0x5602461d25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f6353fff700 1 --2- 172.21.15.150:0/2343301428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c2050 0x5602461d25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f6353fff700 1 -- 172.21.15.150:0/2343301428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c29a0 msgr2=0x5602461c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f6353fff700 1 --2- 172.21.15.150:0/2343301428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c29a0 0x5602461c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f6353fff700 1 -- 172.21.15.150:0/2343301428 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602460a7500 con 0x5602461c1180 2024-05-10T04:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f6358857700 1 --2- 172.21.15.150:0/2343301428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c29a0 0x5602461c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.631+0000 7f6353fff700 1 --2- 172.21.15.150:0/2343301428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 0x5602461ddb80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6344095f50 tx=0x7f63440b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804021a2e3c22f91 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.632+0000 7f633affd700 1 -- 172.21.15.150:0/2343301428 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63440b22b0 con 0x5602461c1180 2024-05-10T04:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.632+0000 7f633affd700 1 -- 172.21.15.150:0/2343301428 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6344097df0 con 0x5602461c1180 2024-05-10T04:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.632+0000 7f633affd700 1 -- 172.21.15.150:0/2343301428 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63440a1b50 con 0x5602461c1180 2024-05-10T04:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.632+0000 7f636a5feb80 1 -- 172.21.15.150:0/2343301428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 msgr2=0x5602461ddb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.632+0000 7f636a5feb80 1 --2- 172.21.15.150:0/2343301428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 0x5602461ddb80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6344095f50 tx=0x7f63440b9040 comp rx=0 tx=0).stop 2024-05-10T04:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.632+0000 7f636a5feb80 1 -- 172.21.15.150:0/2343301428 shutdown_connections 2024-05-10T04:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.632+0000 7f636a5feb80 1 --2- 172.21.15.150:0/2343301428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c29a0 0x5602461c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.632+0000 7f636a5feb80 1 --2- 172.21.15.150:0/2343301428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 0x5602461ddb80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.632+0000 7f636a5feb80 1 --2- 172.21.15.150:0/2343301428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c2050 0x5602461d25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.632+0000 7f636a5feb80 1 -- 172.21.15.150:0/2343301428 >> 172.21.15.150:0/2343301428 conn(0x560246123a00 msgr2=0x5602461d9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.632+0000 7f636a5feb80 1 -- 172.21.15.150:0/2343301428 shutdown_connections 2024-05-10T04:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.632+0000 7f636a5feb80 1 -- 172.21.15.150:0/2343301428 wait complete. 2024-05-10T04:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.633+0000 7f636a5feb80 1 -- start start 2024-05-10T04:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.633+0000 7f636a5feb80 1 Processor -- start 2024-05-10T04:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.633+0000 7f636a5feb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 0x5602461cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.633+0000 7f636a5feb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c2050 0x5602461cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.633+0000 7f636a5feb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c29a0 0x56024622b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.633+0000 7f636a5feb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602461d26c0 con 0x5602461c1180 2024-05-10T04:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.633+0000 7f636a5feb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602461d2830 con 0x5602461c29a0 2024-05-10T04:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.633+0000 7f636a5feb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56024622b820 con 0x5602461c2050 2024-05-10T04:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.633+0000 7f636a5feb80 10 client.0 _ll_register_callbacks cb 0x560246086ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.633+0000 7f6353fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 0x5602461cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.633+0000 7f6353fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 0x5602461cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43954/0 (socket says 172.21.15.150:43954) 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.633+0000 7f6353fff700 1 -- 172.21.15.150:0/4279528633 learned_addr learned my addr 172.21.15.150:0/4279528633 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f6358857700 1 --2- 172.21.15.150:0/4279528633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c29a0 0x56024622b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f63537fe700 1 --2- 172.21.15.150:0/4279528633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c2050 0x5602461cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f6353fff700 1 -- 172.21.15.150:0/4279528633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c2050 msgr2=0x5602461cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f6353fff700 1 --2- 172.21.15.150:0/4279528633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c2050 0x5602461cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f6353fff700 1 -- 172.21.15.150:0/4279528633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c29a0 msgr2=0x56024622b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f6353fff700 1 --2- 172.21.15.150:0/4279528633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c29a0 0x56024622b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f6353fff700 1 -- 172.21.15.150:0/4279528633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63541a06a0 con 0x5602461c1180 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f6353fff700 1 --2- 172.21.15.150:0/4279528633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 0x5602461cf290 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f634400d950 tx=0x7f63440992d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cba8b42c32fe97d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f63337fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f63337fe700 1 -- 172.21.15.150:0/4279528633 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344064070 con 0x5602461c1180 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f63337fe700 1 -- 172.21.15.150:0/4279528633 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f63440a18f0 con 0x5602461c1180 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f636a5feb80 1 -- 172.21.15.150:0/4279528633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6348015070 con 0x5602461c1180 2024-05-10T04:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f63337fe700 1 -- 172.21.15.150:0/4279528633 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63440a1af0 con 0x5602461c1180 2024-05-10T04:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f636a5feb80 10 client.15672 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f636a5feb80 1 -- 172.21.15.150:0/4279528633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560245f66c70 con 0x5602461c1180 2024-05-10T04:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f63337fe700 1 -- 172.21.15.150:0/4279528633 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6348015070 con 0x5602461c1180 2024-05-10T04:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f63337fe700 1 -- 172.21.15.150:0/4279528633 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6344098b40 con 0x5602461c1180 2024-05-10T04:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f636a5feb80 10 client.15672 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f636a5feb80 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.634+0000 7f636a5feb80 20 client.15672 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.635+0000 7f636a5feb80 1 --2- 172.21.15.150:0/4279528633 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602461d6e20 0x56024622ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.635+0000 7f636a5feb80 4 client.15672 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.635+0000 7f636a5feb80 1 -- 172.21.15.150:0/4279528633 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5602460d9e60 con 0x5602461d6e20 2024-05-10T04:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.635+0000 7f63537fe700 1 --2- 172.21.15.150:0/4279528633 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602461d6e20 0x56024622ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.635+0000 7f63337fe700 10 client.15672 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.635+0000 7f63537fe700 1 --2- 172.21.15.150:0/4279528633 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602461d6e20 0x56024622ecc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbdcf7f6e0c5e04a server_cookie=66ce1afdaf653779 in_seq=0 out_seq=0 2024-05-10T04:04:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f63337fe700 1 -- 172.21.15.150:0/4279528633 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5602460d9e60 con 0x5602461d6e20 2024-05-10T04:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f63337fe700 10 client.15672 handle_command_reply: tid=0 2024-05-10T04:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f636a5feb80 1 client.15672 shutdown 2024-05-10T04:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f636a5feb80 1 -- 172.21.15.150:0/4279528633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 msgr2=0x5602461cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f636a5feb80 1 --2- 172.21.15.150:0/4279528633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 0x5602461cf290 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f634400d950 tx=0x7f63440992d0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f636a5feb80 1 -- 172.21.15.150:0/4279528633 shutdown_connections 2024-05-10T04:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f636a5feb80 1 --2- 172.21.15.150:0/4279528633 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602461d6e20 0x56024622ecc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f636a5feb80 1 --2- 172.21.15.150:0/4279528633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602461c29a0 0x56024622b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f636a5feb80 1 --2- 172.21.15.150:0/4279528633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602461c1180 0x5602461cf290 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f636a5feb80 1 --2- 172.21.15.150:0/4279528633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602461c2050 0x5602461cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f63337fe700 0 client.15672 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f636a5feb80 1 -- 172.21.15.150:0/4279528633 >> 172.21.15.150:0/4279528633 conn(0x560246123a00 msgr2=0x5602461d9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f636a5feb80 1 -- 172.21.15.150:0/4279528633 shutdown_connections 2024-05-10T04:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f636a5feb80 1 -- 172.21.15.150:0/4279528633 wait complete. 2024-05-10T04:04:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.649+0000 7f636a5feb80 20 client.15672 trim_cache size 0 max 16384 2024-05-10T04:04:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:04:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:04:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:04:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.650+0000 7f635a2ba700 1 -- 172.21.15.150:0/2344506105 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f633c0a11b0 msgr2=0x7f633c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.650+0000 7f635a2ba700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f633c0a11b0 0x7f633c0a36c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f63541991e0 tx=0x7f634800b410 comp rx=0 tx=0).stop 2024-05-10T04:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.650+0000 7f635a2ba700 1 -- 172.21.15.150:0/2344506105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6354100ae0 msgr2=0x7f6354198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.650+0000 7f635a2ba700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6354100ae0 0x7f6354198140 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f634400d980 tx=0x7f634400dc90 comp rx=0 tx=0).stop 2024-05-10T04:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.650+0000 7f635a2ba700 1 -- 172.21.15.150:0/2344506105 shutdown_connections 2024-05-10T04:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.650+0000 7f635a2ba700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f633c0a11b0 0x7f633c0a36c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.650+0000 7f635a2ba700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63541025a0 0x7f635419f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.650+0000 7f635a2ba700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6354100ae0 0x7f6354198140 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.650+0000 7f635a2ba700 1 --2- 172.21.15.150:0/2344506105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6354101c00 0x7f6354198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.650+0000 7f635a2ba700 1 -- 172.21.15.150:0/2344506105 >> 172.21.15.150:0/2344506105 conn(0x7f63540fc0f0 msgr2=0x7f63540fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.650+0000 7f635a2ba700 1 -- 172.21.15.150:0/2344506105 shutdown_connections 2024-05-10T04:04:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:25.650+0000 7f635a2ba700 1 -- 172.21.15.150:0/2344506105 wait complete. 2024-05-10T04:04:26.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:26 smithi150 ceph-mon[42477]: pgmap v285: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T04:04:26.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:26 smithi150 ceph-mon[42477]: scrub summary: idle 2024-05-10T04:04:26.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:26 smithi181 ceph-mon[46602]: pgmap v285: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T04:04:26.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:26 smithi181 ceph-mon[46602]: scrub summary: idle 2024-05-10T04:04:26.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:26 smithi161 ceph-mon[46918]: pgmap v285: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T04:04:26.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:26 smithi161 ceph-mon[46918]: scrub summary: idle 2024-05-10T04:04:28.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:28 smithi150 ceph-mon[42477]: pgmap v286: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-10T04:04:28.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:28 smithi181 ceph-mon[46602]: pgmap v286: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-10T04:04:28.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:28 smithi161 ceph-mon[46918]: pgmap v286: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-10T04:04:30.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:30 smithi150 ceph-mon[42477]: pgmap v287: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-10T04:04:30.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:30 smithi181 ceph-mon[46602]: pgmap v287: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-10T04:04:30.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:30 smithi161 ceph-mon[46918]: pgmap v287: 129 pgs: 129 active+clean; 654 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-10T04:04:32.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:32 smithi150 ceph-mon[42477]: pgmap v288: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-10T04:04:32.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:32 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:04:32.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:32 smithi181 ceph-mon[46602]: pgmap v288: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-10T04:04:32.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:32 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:04:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:32 smithi161 ceph-mon[46918]: pgmap v288: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-10T04:04:32.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:32 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:04:34.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:33 smithi150 ceph-mon[42477]: pgmap v289: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-10T04:04:34.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:04:34.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:04:34.464 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:33 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:33 smithi181 ceph-mon[46602]: pgmap v289: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-10T04:04:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:04:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:04:34.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:33 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:33 smithi161 ceph-mon[46918]: pgmap v289: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-10T04:04:34.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:34.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:04:34.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:04:34.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:33 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:04:36.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:36 smithi150 ceph-mon[42477]: pgmap v290: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-10T04:04:36.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:36 smithi181 ceph-mon[46602]: pgmap v290: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-10T04:04:36.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:36 smithi161 ceph-mon[46918]: pgmap v290: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-10T04:04:38.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:38 smithi150 ceph-mon[42477]: pgmap v291: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-10T04:04:38.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:38 smithi181 ceph-mon[46602]: pgmap v291: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-10T04:04:38.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:38 smithi161 ceph-mon[46918]: pgmap v291: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-10T04:04:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.779+0000 7fbc8aefc700 1 -- 172.21.15.150:0/3692008329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc84103170 msgr2=0x7fbc84103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.779+0000 7fbc8aefc700 1 --2- 172.21.15.150:0/3692008329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc84103170 0x7fbc84103580 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009990 tx=0x7fbc78009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.779+0000 7fbc8aefc700 1 -- 172.21.15.150:0/3692008329 shutdown_connections 2024-05-10T04:04:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.779+0000 7fbc8aefc700 1 --2- 172.21.15.150:0/3692008329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc84100500 0x7fbc8410a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.779+0000 7fbc8aefc700 1 --2- 172.21.15.150:0/3692008329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc84103170 0x7fbc84103580 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.779+0000 7fbc8aefc700 1 --2- 172.21.15.150:0/3692008329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc840ffbb0 0x7fbc840fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.779+0000 7fbc8aefc700 1 -- 172.21.15.150:0/3692008329 >> 172.21.15.150:0/3692008329 conn(0x7fbc840fb840 msgr2=0x7fbc840fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.779+0000 7fbc8aefc700 1 -- 172.21.15.150:0/3692008329 shutdown_connections 2024-05-10T04:04:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.779+0000 7fbc8aefc700 1 -- 172.21.15.150:0/3692008329 wait complete. 2024-05-10T04:04:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc8aefc700 1 Processor -- start 2024-05-10T04:04:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc8aefc700 1 -- start start 2024-05-10T04:04:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc8aefc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc840ffbb0 0x7fbc84102740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc8aefc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc84100500 0x7fbc84100de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc8aefc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84103170 0x7fbc84101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc8aefc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84101980 con 0x7fbc840ffbb0 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc8aefc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84101af0 con 0x7fbc84100500 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc8aefc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84101cb0 con 0x7fbc84103170 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc88c98700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc840ffbb0 0x7fbc84102740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc88c98700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc840ffbb0 0x7fbc84102740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35326/0 (socket says 172.21.15.150:35326) 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc88c98700 1 -- 172.21.15.150:0/3536285818 learned_addr learned my addr 172.21.15.150:0/3536285818 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc89499700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84103170 0x7fbc84101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc88c98700 1 -- 172.21.15.150:0/3536285818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84103170 msgr2=0x7fbc84101320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc83fff700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc84100500 0x7fbc84100de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc88c98700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84103170 0x7fbc84101320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc88c98700 1 -- 172.21.15.150:0/3536285818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc84100500 msgr2=0x7fbc84100de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc88c98700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc84100500 0x7fbc84100de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc88c98700 1 -- 172.21.15.150:0/3536285818 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc780095f0 con 0x7fbc840ffbb0 2024-05-10T04:04:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc89499700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84103170 0x7fbc84101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc83fff700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc84100500 0x7fbc84100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.780+0000 7fbc88c98700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc840ffbb0 0x7fbc84102740 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbc780098e0 tx=0x7fbc78005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.781+0000 7fbc81ffb700 1 -- 172.21.15.150:0/3536285818 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7801d070 con 0x7fbc840ffbb0 2024-05-10T04:04:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.781+0000 7fbc81ffb700 1 -- 172.21.15.150:0/3536285818 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc78021410 con 0x7fbc840ffbb0 2024-05-10T04:04:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.781+0000 7fbc8aefc700 1 -- 172.21.15.150:0/3536285818 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc841a2200 con 0x7fbc840ffbb0 2024-05-10T04:04:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.781+0000 7fbc81ffb700 1 -- 172.21.15.150:0/3536285818 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78017650 con 0x7fbc840ffbb0 2024-05-10T04:04:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.781+0000 7fbc8aefc700 1 -- 172.21.15.150:0/3536285818 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc841a2620 con 0x7fbc840ffbb0 2024-05-10T04:04:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.784+0000 7fbc81ffb700 1 -- 172.21.15.150:0/3536285818 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fbc780177b0 con 0x7fbc840ffbb0 2024-05-10T04:04:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.785+0000 7fbc81ffb700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc740a11b0 0x7fbc740a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.785+0000 7fbc81ffb700 1 -- 172.21.15.150:0/3536285818 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbc780ce200 con 0x7fbc840ffbb0 2024-05-10T04:04:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.785+0000 7fbc83fff700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc740a11b0 0x7fbc740a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.785+0000 7fbc9b240b80 1 Processor -- start 2024-05-10T04:04:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.785+0000 7fbc9b240b80 1 -- start start 2024-05-10T04:04:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.785+0000 7fbc83fff700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc740a11b0 0x7fbc740a36c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbc841020e0 tx=0x7fbc70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.785+0000 7fbc9b240b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd7180 0x562051dd75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.785+0000 7fbc9b240b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051dd8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.785+0000 7fbc9b240b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd89a0 0x562051ddf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.785+0000 7fbc9b240b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051cfe0f0 con 0x562051dd8050 2024-05-10T04:04:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.785+0000 7fbc9b240b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051cfbab0 con 0x562051dd89a0 2024-05-10T04:04:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.785+0000 7fbc9b240b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051cf73a0 con 0x562051dd7180 2024-05-10T04:04:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051dd8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051dd8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35330/0 (socket says 172.21.15.150:35330) 2024-05-10T04:04:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc83fff700 1 -- 172.21.15.150:0/2273561263 learned_addr learned my addr 172.21.15.150:0/2273561263 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc88c98700 1 --2- 172.21.15.150:0/2273561263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd7180 0x562051dd75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc89499700 1 --2- 172.21.15.150:0/2273561263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd89a0 0x562051ddf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc83fff700 1 -- 172.21.15.150:0/2273561263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd7180 msgr2=0x562051dd75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc83fff700 1 --2- 172.21.15.150:0/2273561263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd7180 0x562051dd75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc83fff700 1 -- 172.21.15.150:0/2273561263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd89a0 msgr2=0x562051ddf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc83fff700 1 --2- 172.21.15.150:0/2273561263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd89a0 0x562051ddf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc83fff700 1 -- 172.21.15.150:0/2273561263 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562051b78350 con 0x562051dd8050 2024-05-10T04:04:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc83fff700 1 --2- 172.21.15.150:0/2273561263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051dd8460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbc70004480 tx=0x7fbc70013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7935545dd39841bf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc6effd700 1 -- 172.21.15.150:0/2273561263 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70018040 con 0x562051dd8050 2024-05-10T04:04:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc6effd700 1 -- 172.21.15.150:0/2273561263 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc70003d40 con 0x562051dd8050 2024-05-10T04:04:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.786+0000 7fbc6effd700 1 -- 172.21.15.150:0/2273561263 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70020ce0 con 0x562051dd8050 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.787+0000 7fbc9b240b80 1 -- 172.21.15.150:0/2273561263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 msgr2=0x562051dd8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.787+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/2273561263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051dd8460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbc70004480 tx=0x7fbc70013000 comp rx=0 tx=0).stop 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.787+0000 7fbc9b240b80 1 -- 172.21.15.150:0/2273561263 shutdown_connections 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.787+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/2273561263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd89a0 0x562051ddf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.787+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/2273561263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd7180 0x562051dd75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.787+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/2273561263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051dd8460 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.787+0000 7fbc9b240b80 1 -- 172.21.15.150:0/2273561263 >> 172.21.15.150:0/2273561263 conn(0x562051d39a00 msgr2=0x562051dd1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.787+0000 7fbc9b240b80 1 -- 172.21.15.150:0/2273561263 shutdown_connections 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.787+0000 7fbc9b240b80 1 -- 172.21.15.150:0/2273561263 wait complete. 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.787+0000 7fbc9b240b80 1 -- start start 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc9b240b80 1 Processor -- start 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc9b240b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd7180 0x562051de11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc9b240b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc9b240b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd89a0 0x562051de8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc9b240b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051c93500 con 0x562051dd8050 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc9b240b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051c931a0 con 0x562051dd7180 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc9b240b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051b7a640 con 0x562051dd89a0 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc9b240b80 10 client.0 _ll_register_callbacks cb 0x562051c9cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35340/0 (socket says 172.21.15.150:35340) 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc83fff700 1 -- 172.21.15.150:0/3968893242 learned_addr learned my addr 172.21.15.150:0/3968893242 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc89499700 1 --2- 172.21.15.150:0/3968893242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd89a0 0x562051de8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc88c98700 1 --2- 172.21.15.150:0/3968893242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd7180 0x562051de11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc83fff700 1 -- 172.21.15.150:0/3968893242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd89a0 msgr2=0x562051de8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc83fff700 1 --2- 172.21.15.150:0/3968893242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd89a0 0x562051de8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc83fff700 1 -- 172.21.15.150:0/3968893242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd7180 msgr2=0x562051de11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc83fff700 1 --2- 172.21.15.150:0/3968893242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd7180 0x562051de11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.788+0000 7fbc83fff700 1 -- 172.21.15.150:0/3968893242 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc70004130 con 0x562051dd8050 2024-05-10T04:04:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc83fff700 1 --2- 172.21.15.150:0/3968893242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de16f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009fd0 tx=0x7fbc700206e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7c53cec6d383a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc63fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc63fff700 1 -- 172.21.15.150:0/3968893242 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70003880 con 0x562051dd8050 2024-05-10T04:04:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc9b240b80 1 -- 172.21.15.150:0/3968893242 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562051c50210 con 0x562051dd8050 2024-05-10T04:04:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc63fff700 1 -- 172.21.15.150:0/3968893242 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbc700039e0 con 0x562051dd8050 2024-05-10T04:04:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc63fff700 1 -- 172.21.15.150:0/3968893242 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc70021ee0 con 0x562051dd8050 2024-05-10T04:04:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc9b240b80 10 client.15684 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc9b240b80 1 -- 172.21.15.150:0/3968893242 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562051b24b00 con 0x562051dd8050 2024-05-10T04:04:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc63fff700 1 -- 172.21.15.150:0/3968893242 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562051c50210 con 0x562051dd8050 2024-05-10T04:04:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc63fff700 1 -- 172.21.15.150:0/3968893242 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbc7001e040 con 0x562051dd8050 2024-05-10T04:04:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc9b240b80 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc9b240b80 10 client.15684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc9b240b80 20 client.15684 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/3968893242 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x562051dece00 0x562051def250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc9b240b80 4 client.15684 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:04:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.789+0000 7fbc9b240b80 1 -- 172.21.15.150:0/3968893242 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562051da6720 con 0x562051dece00 2024-05-10T04:04:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.790+0000 7fbc88c98700 1 --2- 172.21.15.150:0/3968893242 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x562051dece00 0x562051def250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.790+0000 7fbc88c98700 1 --2- 172.21.15.150:0/3968893242 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x562051dece00 0x562051def250 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=638d2cdd04c5bd3e server_cookie=b986fcb8b45a83f9 in_seq=0 out_seq=0 2024-05-10T04:04:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.790+0000 7fbc63fff700 10 client.15684 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc63fff700 1 -- 172.21.15.150:0/3968893242 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562051da6720 con 0x562051dece00 2024-05-10T04:04:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc63fff700 10 client.15684 handle_command_reply: tid=0 2024-05-10T04:04:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc9b240b80 1 client.15684 shutdown 2024-05-10T04:04:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc9b240b80 1 -- 172.21.15.150:0/3968893242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 msgr2=0x562051de16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/3968893242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de16f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009fd0 tx=0x7fbc700206e0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc9b240b80 1 -- 172.21.15.150:0/3968893242 shutdown_connections 2024-05-10T04:04:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/3968893242 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x562051dece00 0x562051def250 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/3968893242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd89a0 0x562051de8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/3968893242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd7180 0x562051de11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc63fff700 0 client.15684 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/3968893242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de16f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc9b240b80 1 -- 172.21.15.150:0/3968893242 >> 172.21.15.150:0/3968893242 conn(0x562051d39a00 msgr2=0x562051dd1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc9b240b80 1 -- 172.21.15.150:0/3968893242 shutdown_connections 2024-05-10T04:04:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc9b240b80 1 -- 172.21.15.150:0/3968893242 wait complete. 2024-05-10T04:04:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.791+0000 7fbc9b240b80 20 client.15684 trim_cache size 0 max 16384 2024-05-10T04:04:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.806+0000 7fbc9b240b80 1 Processor -- start 2024-05-10T04:04:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.806+0000 7fbc9b240b80 1 -- start start 2024-05-10T04:04:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.806+0000 7fbc9b240b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd7180 0x562051de8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.806+0000 7fbc9b240b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.806+0000 7fbc9b240b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd89a0 0x562051de1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.806+0000 7fbc9b240b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051ddf1e0 con 0x562051dd8050 2024-05-10T04:04:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.806+0000 7fbc9b240b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051ddcab0 con 0x562051dd89a0 2024-05-10T04:04:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.806+0000 7fbc9b240b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051decb80 con 0x562051dd7180 2024-05-10T04:04:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.806+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.806+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35356/0 (socket says 172.21.15.150:35356) 2024-05-10T04:04:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.806+0000 7fbc83fff700 1 -- 172.21.15.150:0/3666899155 learned_addr learned my addr 172.21.15.150:0/3666899155 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.807+0000 7fbc88c98700 1 --2- 172.21.15.150:0/3666899155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd7180 0x562051de8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.807+0000 7fbc83fff700 1 -- 172.21.15.150:0/3666899155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd7180 msgr2=0x562051de8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.807+0000 7fbc83fff700 1 --2- 172.21.15.150:0/3666899155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd7180 0x562051de8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.807+0000 7fbc83fff700 1 -- 172.21.15.150:0/3666899155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd89a0 msgr2=0x562051de1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.807+0000 7fbc89499700 1 --2- 172.21.15.150:0/3666899155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd89a0 0x562051de1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.807+0000 7fbc83fff700 1 --2- 172.21.15.150:0/3666899155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd89a0 0x562051de1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.807+0000 7fbc83fff700 1 -- 172.21.15.150:0/3666899155 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562051cbd500 con 0x562051dd8050 2024-05-10T04:04:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.807+0000 7fbc83fff700 1 --2- 172.21.15.150:0/3666899155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de8610 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbc70004100 tx=0x7fbc700344a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f29ca88a4b96ab1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.807+0000 7fbc6f7fe700 1 -- 172.21.15.150:0/3666899155 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc700107b0 con 0x562051dd8050 2024-05-10T04:04:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.807+0000 7fbc6f7fe700 1 -- 172.21.15.150:0/3666899155 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc70003b40 con 0x562051dd8050 2024-05-10T04:04:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.807+0000 7fbc6f7fe700 1 -- 172.21.15.150:0/3666899155 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70034a10 con 0x562051dd8050 2024-05-10T04:04:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.808+0000 7fbc9b240b80 1 -- 172.21.15.150:0/3666899155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 msgr2=0x562051de8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.808+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/3666899155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de8610 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbc70004100 tx=0x7fbc700344a0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.808+0000 7fbc9b240b80 1 -- 172.21.15.150:0/3666899155 shutdown_connections 2024-05-10T04:04:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.808+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/3666899155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd89a0 0x562051de1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.808+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/3666899155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd7180 0x562051de8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.808+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/3666899155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de8610 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.808+0000 7fbc9b240b80 1 -- 172.21.15.150:0/3666899155 >> 172.21.15.150:0/3666899155 conn(0x562051d39a00 msgr2=0x562051de0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.808+0000 7fbc9b240b80 1 -- 172.21.15.150:0/3666899155 shutdown_connections 2024-05-10T04:04:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.808+0000 7fbc9b240b80 1 -- 172.21.15.150:0/3666899155 wait complete. 2024-05-10T04:04:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.808+0000 7fbc9b240b80 1 -- start start 2024-05-10T04:04:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.808+0000 7fbc9b240b80 1 Processor -- start 2024-05-10T04:04:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc9b240b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd7180 0x562051de5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc9b240b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc9b240b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd89a0 0x562051de6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc9b240b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051de71c0 con 0x562051dd8050 2024-05-10T04:04:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc9b240b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051de7330 con 0x562051dd7180 2024-05-10T04:04:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc9b240b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562051de74f0 con 0x562051dd89a0 2024-05-10T04:04:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc9b240b80 10 client.0 _ll_register_callbacks cb 0x562051c9cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35358/0 (socket says 172.21.15.150:35358) 2024-05-10T04:04:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc83fff700 1 -- 172.21.15.150:0/2059993971 learned_addr learned my addr 172.21.15.150:0/2059993971 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc89499700 1 --2- 172.21.15.150:0/2059993971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd89a0 0x562051de6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc88c98700 1 --2- 172.21.15.150:0/2059993971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd7180 0x562051de5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc83fff700 1 -- 172.21.15.150:0/2059993971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd89a0 msgr2=0x562051de6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc83fff700 1 --2- 172.21.15.150:0/2059993971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd89a0 0x562051de6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc83fff700 1 -- 172.21.15.150:0/2059993971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd7180 msgr2=0x562051de5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc83fff700 1 --2- 172.21.15.150:0/2059993971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd7180 0x562051de5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc83fff700 1 -- 172.21.15.150:0/2059993971 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562051b24b00 con 0x562051dd8050 2024-05-10T04:04:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc83fff700 1 --2- 172.21.15.150:0/2059993971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de5d50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbc70003d10 tx=0x7fbc700107b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1430460909cb2ff6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc63fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc63fff700 1 -- 172.21.15.150:0/2059993971 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70003880 con 0x562051dd8050 2024-05-10T04:04:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc63fff700 1 -- 172.21.15.150:0/2059993971 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbc700347b0 con 0x562051dd8050 2024-05-10T04:04:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.809+0000 7fbc9b240b80 1 -- 172.21.15.150:0/2059993971 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc70003d40 con 0x562051dd8050 2024-05-10T04:04:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.810+0000 7fbc63fff700 1 -- 172.21.15.150:0/2059993971 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc700039e0 con 0x562051dd8050 2024-05-10T04:04:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.810+0000 7fbc9b240b80 10 client.15690 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.810+0000 7fbc9b240b80 1 -- 172.21.15.150:0/2059993971 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562051b7cc70 con 0x562051dd8050 2024-05-10T04:04:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.810+0000 7fbc63fff700 1 -- 172.21.15.150:0/2059993971 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc70003d40 con 0x562051dd8050 2024-05-10T04:04:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.810+0000 7fbc63fff700 1 -- 172.21.15.150:0/2059993971 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbc70010ca0 con 0x562051dd8050 2024-05-10T04:04:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.810+0000 7fbc9b240b80 10 client.15690 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.810+0000 7fbc9b240b80 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.810+0000 7fbc9b240b80 20 client.15690 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.810+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/2059993971 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x562051e432b0 0x562051e456d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.810+0000 7fbc9b240b80 4 client.15690 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:04:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.810+0000 7fbc9b240b80 1 -- 172.21.15.150:0/2059993971 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562051cefe60 con 0x562051e432b0 2024-05-10T04:04:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.810+0000 7fbc88c98700 1 --2- 172.21.15.150:0/2059993971 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x562051e432b0 0x562051e456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.811+0000 7fbc63fff700 10 client.15690 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.811+0000 7fbc88c98700 1 --2- 172.21.15.150:0/2059993971 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x562051e432b0 0x562051e456d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=700e4647b218a25a server_cookie=cc85af4287571300 in_seq=0 out_seq=0 2024-05-10T04:04:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc63fff700 1 -- 172.21.15.150:0/2059993971 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562051cefe60 con 0x562051e432b0 2024-05-10T04:04:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc63fff700 10 client.15690 handle_command_reply: tid=0 2024-05-10T04:04:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc9b240b80 1 client.15690 shutdown 2024-05-10T04:04:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc9b240b80 1 -- 172.21.15.150:0/2059993971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 msgr2=0x562051de5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/2059993971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de5d50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbc70003d10 tx=0x7fbc700107b0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc9b240b80 1 -- 172.21.15.150:0/2059993971 shutdown_connections 2024-05-10T04:04:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/2059993971 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x562051e432b0 0x562051e456d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/2059993971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562051dd89a0 0x562051de6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc63fff700 0 client.15690 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/2059993971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562051dd7180 0x562051de5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc9b240b80 1 --2- 172.21.15.150:0/2059993971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562051dd8050 0x562051de5d50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc9b240b80 1 -- 172.21.15.150:0/2059993971 >> 172.21.15.150:0/2059993971 conn(0x562051d39a00 msgr2=0x562051ddf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc9b240b80 1 -- 172.21.15.150:0/2059993971 shutdown_connections 2024-05-10T04:04:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc9b240b80 1 -- 172.21.15.150:0/2059993971 wait complete. 2024-05-10T04:04:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.817+0000 7fbc9b240b80 20 client.15690 trim_cache size 0 max 16384 2024-05-10T04:04:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:04:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:04:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:04:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:04:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.818+0000 7fbc8aefc700 1 -- 172.21.15.150:0/3536285818 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc740a11b0 msgr2=0x7fbc740a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.818+0000 7fbc8aefc700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc740a11b0 0x7fbc740a36c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbc841020e0 tx=0x7fbc70009260 comp rx=0 tx=0).stop 2024-05-10T04:04:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.818+0000 7fbc8aefc700 1 -- 172.21.15.150:0/3536285818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc840ffbb0 msgr2=0x7fbc84102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.818+0000 7fbc8aefc700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc840ffbb0 0x7fbc84102740 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbc780098e0 tx=0x7fbc78005b00 comp rx=0 tx=0).stop 2024-05-10T04:04:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.818+0000 7fbc8aefc700 1 -- 172.21.15.150:0/3536285818 shutdown_connections 2024-05-10T04:04:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.818+0000 7fbc8aefc700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbc740a11b0 0x7fbc740a36c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.818+0000 7fbc8aefc700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84103170 0x7fbc84101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.818+0000 7fbc8aefc700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc840ffbb0 0x7fbc84102740 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.818+0000 7fbc8aefc700 1 --2- 172.21.15.150:0/3536285818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc84100500 0x7fbc84100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.818+0000 7fbc8aefc700 1 -- 172.21.15.150:0/3536285818 >> 172.21.15.150:0/3536285818 conn(0x7fbc840fb840 msgr2=0x7fbc84106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.818+0000 7fbc8aefc700 1 -- 172.21.15.150:0/3536285818 shutdown_connections 2024-05-10T04:04:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:38.818+0000 7fbc8aefc700 1 -- 172.21.15.150:0/3536285818 wait complete. 2024-05-10T04:04:40.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:40 smithi150 ceph-mon[42477]: pgmap v292: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T04:04:40.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:40 smithi181 ceph-mon[46602]: pgmap v292: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T04:04:40.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:40 smithi161 ceph-mon[46918]: pgmap v292: 129 pgs: 129 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T04:04:42.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:42 smithi150 ceph-mon[42477]: pgmap v293: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T04:04:42.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:42 smithi181 ceph-mon[46602]: pgmap v293: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T04:04:42.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:42 smithi161 ceph-mon[46918]: pgmap v293: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T04:04:42.851 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:04:43.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.000+0000 7f53c10b5700 1 -- 172.21.15.150:0/1849000598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53bc102580 msgr2=0x7f53bc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.000+0000 7f53c10b5700 1 --2- 172.21.15.150:0/1849000598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53bc102580 0x7f53bc108e30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f53b400b0a0 tx=0x7f53b400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.000+0000 7f53c10b5700 1 -- 172.21.15.150:0/1849000598 shutdown_connections 2024-05-10T04:04:43.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.000+0000 7f53c10b5700 1 --2- 172.21.15.150:0/1849000598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53bc102580 0x7f53bc108e30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.000+0000 7f53c10b5700 1 --2- 172.21.15.150:0/1849000598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53bc100ac0 0x7f53bc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.000+0000 7f53c10b5700 1 --2- 172.21.15.150:0/1849000598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc101be0 0x7f53bc101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.000+0000 7f53c10b5700 1 -- 172.21.15.150:0/1849000598 >> 172.21.15.150:0/1849000598 conn(0x7f53bc0fc0d0 msgr2=0x7f53bc0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.000+0000 7f53c10b5700 1 -- 172.21.15.150:0/1849000598 shutdown_connections 2024-05-10T04:04:43.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.000+0000 7f53c10b5700 1 -- 172.21.15.150:0/1849000598 wait complete. 2024-05-10T04:04:43.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.001+0000 7f53c10b5700 1 Processor -- start 2024-05-10T04:04:43.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.001+0000 7f53c10b5700 1 -- start start 2024-05-10T04:04:43.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.001+0000 7f53c10b5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53bc100ac0 0x7f53bc196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.001+0000 7f53c10b5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53bc101be0 0x7f53bc197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.001+0000 7f53c10b5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc102580 0x7f53bc19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.001+0000 7f53c10b5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc19ea00 con 0x7f53bc100ac0 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.001+0000 7f53c10b5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc19eb70 con 0x7f53bc101be0 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.001+0000 7f53c10b5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc19ed30 con 0x7f53bc102580 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.001+0000 7f53bad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53bc100ac0 0x7f53bc196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53bad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53bc100ac0 0x7f53bc196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38286/0 (socket says 172.21.15.150:38286) 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53bad9d700 1 -- 172.21.15.150:0/170485436 learned_addr learned my addr 172.21.15.150:0/170485436 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53ba59c700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53bc101be0 0x7f53bc197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53bb59e700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc102580 0x7f53bc19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53bad9d700 1 -- 172.21.15.150:0/170485436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc102580 msgr2=0x7f53bc19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53bad9d700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc102580 0x7f53bc19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53bad9d700 1 -- 172.21.15.150:0/170485436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53bc101be0 msgr2=0x7f53bc197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53bad9d700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53bc101be0 0x7f53bc197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53bad9d700 1 -- 172.21.15.150:0/170485436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b4009d00 con 0x7f53bc100ac0 2024-05-10T04:04:43.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53ba59c700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53bc101be0 0x7f53bc197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:43.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53bad9d700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53bc100ac0 0x7f53bc196f20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00c970 tx=0x7f53ac00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53a3fff700 1 -- 172.21.15.150:0/170485436 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac004220 con 0x7f53bc100ac0 2024-05-10T04:04:43.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53a3fff700 1 -- 172.21.15.150:0/170485436 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53ac007dc0 con 0x7f53bc100ac0 2024-05-10T04:04:43.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53c10b5700 1 -- 172.21.15.150:0/170485436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53bc19efb0 con 0x7f53bc100ac0 2024-05-10T04:04:43.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53a3fff700 1 -- 172.21.15.150:0/170485436 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac003d00 con 0x7f53bc100ac0 2024-05-10T04:04:43.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.002+0000 7f53c10b5700 1 -- 172.21.15.150:0/170485436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc19f4e0 con 0x7f53bc100ac0 2024-05-10T04:04:43.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.005+0000 7f53a3fff700 1 -- 172.21.15.150:0/170485436 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f53ac00f450 con 0x7f53bc100ac0 2024-05-10T04:04:43.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.005+0000 7f53a3fff700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f53a40a1260 0x7f53a40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.005+0000 7f53a3fff700 1 -- 172.21.15.150:0/170485436 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f53ac0cca40 con 0x7f53bc100ac0 2024-05-10T04:04:43.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.005+0000 7f53ba59c700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f53a40a1260 0x7f53a40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.006+0000 7f53ba59c700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f53a40a1260 0x7f53a40a3770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f53bc197fc0 tx=0x7f53b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.006+0000 7f53d13f9b80 1 Processor -- start 2024-05-10T04:04:43.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.006+0000 7f53d13f9b80 1 -- start start 2024-05-10T04:04:43.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.006+0000 7f53d13f9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635775806a0 0x563577580af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.006+0000 7f53d13f9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563577581570 0x563577581980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.006+0000 7f53d13f9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x5635775887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.006+0000 7f53d13f9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635774a7150 con 0x563577581ec0 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.006+0000 7f53d13f9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635774a4b10 con 0x563577581570 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.006+0000 7f53d13f9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635774a0400 con 0x5635775806a0 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53bb59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x5635775887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53bb59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x5635775887c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38290/0 (socket says 172.21.15.150:38290) 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53bb59e700 1 -- 172.21.15.150:0/3845474131 learned_addr learned my addr 172.21.15.150:0/3845474131 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53bad9d700 1 --2- 172.21.15.150:0/3845474131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635775806a0 0x563577580af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53ba59c700 1 --2- 172.21.15.150:0/3845474131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563577581570 0x563577581980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53bb59e700 1 -- 172.21.15.150:0/3845474131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635775806a0 msgr2=0x563577580af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53bb59e700 1 --2- 172.21.15.150:0/3845474131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635775806a0 0x563577580af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53bb59e700 1 -- 172.21.15.150:0/3845474131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563577581570 msgr2=0x563577581980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53bb59e700 1 --2- 172.21.15.150:0/3845474131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563577581570 0x563577581980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53bb59e700 1 -- 172.21.15.150:0/3845474131 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635773213b0 con 0x563577581ec0 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53ba59c700 1 --2- 172.21.15.150:0/3845474131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563577581570 0x563577581980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53bb59e700 1 --2- 172.21.15.150:0/3845474131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x5635775887c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009c40 tx=0x7f53b4009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03dd3a9b532412d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53a17fa700 1 -- 172.21.15.150:0/3845474131 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b400e040 con 0x563577581ec0 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53a17fa700 1 -- 172.21.15.150:0/3845474131 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53b4004a50 con 0x563577581ec0 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.007+0000 7f53a17fa700 1 -- 172.21.15.150:0/3845474131 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b401bbc0 con 0x563577581ec0 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.008+0000 7f53d13f9b80 1 -- 172.21.15.150:0/3845474131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 msgr2=0x5635775887c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.008+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/3845474131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x5635775887c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009c40 tx=0x7f53b4009c70 comp rx=0 tx=0).stop 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.008+0000 7f53d13f9b80 1 -- 172.21.15.150:0/3845474131 shutdown_connections 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.008+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/3845474131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x5635775887c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.008+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/3845474131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635775806a0 0x563577580af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.008+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/3845474131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563577581570 0x563577581980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.008+0000 7f53d13f9b80 1 -- 172.21.15.150:0/3845474131 >> 172.21.15.150:0/3845474131 conn(0x5635774e2a60 msgr2=0x56357757afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.008+0000 7f53d13f9b80 1 -- 172.21.15.150:0/3845474131 shutdown_connections 2024-05-10T04:04:43.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.008+0000 7f53d13f9b80 1 -- 172.21.15.150:0/3845474131 wait complete. 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.008+0000 7f53d13f9b80 1 -- start start 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.008+0000 7f53d13f9b80 1 Processor -- start 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53d13f9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635775806a0 0x56357758c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53d13f9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581570 0x56357758cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53d13f9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x563577593cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53d13f9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357743c560 con 0x563577581ec0 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53d13f9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357743c200 con 0x5635775806a0 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53d13f9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635773236a0 con 0x563577581570 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53d13f9b80 10 client.0 _ll_register_callbacks cb 0x563577445b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53bb59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x563577593cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53bb59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x563577593cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38298/0 (socket says 172.21.15.150:38298) 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53bb59e700 1 -- 172.21.15.150:0/3558885617 learned_addr learned my addr 172.21.15.150:0/3558885617 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53ba59c700 1 --2- 172.21.15.150:0/3558885617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581570 0x56357758cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53bad9d700 1 --2- 172.21.15.150:0/3558885617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635775806a0 0x56357758c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53bb59e700 1 -- 172.21.15.150:0/3558885617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581570 msgr2=0x56357758cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53bb59e700 1 --2- 172.21.15.150:0/3558885617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581570 0x56357758cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53bb59e700 1 -- 172.21.15.150:0/3558885617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635775806a0 msgr2=0x56357758c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53bb59e700 1 --2- 172.21.15.150:0/3558885617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635775806a0 0x56357758c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53bb59e700 1 -- 172.21.15.150:0/3558885617 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc1098c0 con 0x563577581ec0 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f53bb59e700 1 --2- 172.21.15.150:0/3558885617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x563577593cc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f53b4006130 tx=0x7f53b401b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbecb1b51171dc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.009+0000 7f539eff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f539eff5700 1 -- 172.21.15.150:0/3558885617 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b400e040 con 0x563577581ec0 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f53d13f9b80 1 -- 172.21.15.150:0/3558885617 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635773f9270 con 0x563577581ec0 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f539eff5700 1 -- 172.21.15.150:0/3558885617 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f53b401bc30 con 0x563577581ec0 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f539eff5700 1 -- 172.21.15.150:0/3558885617 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53b401be10 con 0x563577581ec0 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f53d13f9b80 10 client.15702 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f53d13f9b80 1 -- 172.21.15.150:0/3558885617 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5635772cdb60 con 0x563577581ec0 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f539eff5700 1 -- 172.21.15.150:0/3558885617 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635773f9270 con 0x563577581ec0 2024-05-10T04:04:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f539eff5700 1 -- 172.21.15.150:0/3558885617 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f53b4019040 con 0x563577581ec0 2024-05-10T04:04:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f53d13f9b80 10 client.15702 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f53d13f9b80 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f53d13f9b80 20 client.15702 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/3558885617 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563577597e50 0x56357759a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f53d13f9b80 4 client.15702 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:04:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.010+0000 7f53d13f9b80 1 -- 172.21.15.150:0/3558885617 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56357754f780 con 0x563577597e50 2024-05-10T04:04:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.011+0000 7f53bad9d700 1 --2- 172.21.15.150:0/3558885617 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563577597e50 0x56357759a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.011+0000 7f53bad9d700 1 --2- 172.21.15.150:0/3558885617 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563577597e50 0x56357759a270 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45198ca08c5a155f server_cookie=e5bbdc5a0a137dcd in_seq=0 out_seq=0 2024-05-10T04:04:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.011+0000 7f539eff5700 10 client.15702 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:43.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f539eff5700 1 -- 172.21.15.150:0/3558885617 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56357754f780 con 0x563577597e50 2024-05-10T04:04:43.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f539eff5700 10 client.15702 handle_command_reply: tid=0 2024-05-10T04:04:43.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f53d13f9b80 1 client.15702 shutdown 2024-05-10T04:04:43.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f53d13f9b80 1 -- 172.21.15.150:0/3558885617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 msgr2=0x563577593cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/3558885617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x563577593cc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f53b4006130 tx=0x7f53b401b440 comp rx=0 tx=0).stop 2024-05-10T04:04:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f53d13f9b80 1 -- 172.21.15.150:0/3558885617 shutdown_connections 2024-05-10T04:04:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/3558885617 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563577597e50 0x56357759a270 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/3558885617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x563577593cc0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/3558885617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635775806a0 0x56357758c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f539eff5700 0 client.15702 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/3558885617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581570 0x56357758cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f53d13f9b80 1 -- 172.21.15.150:0/3558885617 >> 172.21.15.150:0/3558885617 conn(0x5635774e2a60 msgr2=0x56357757afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f53d13f9b80 1 -- 172.21.15.150:0/3558885617 shutdown_connections 2024-05-10T04:04:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f53d13f9b80 1 -- 172.21.15.150:0/3558885617 wait complete. 2024-05-10T04:04:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.012+0000 7f53d13f9b80 20 client.15702 trim_cache size 0 max 16384 2024-05-10T04:04:43.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53d13f9b80 1 Processor -- start 2024-05-10T04:04:43.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53d13f9b80 1 -- start start 2024-05-10T04:04:43.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53d13f9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635775806a0 0x563577588d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53d13f9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581570 0x563577589240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53d13f9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x5635775852b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53d13f9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357758d6e0 con 0x563577581ec0 2024-05-10T04:04:43.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53d13f9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563577594070 con 0x5635775806a0 2024-05-10T04:04:43.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53d13f9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357757c770 con 0x563577581570 2024-05-10T04:04:43.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53bb59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x5635775852b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53bb59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x5635775852b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38312/0 (socket says 172.21.15.150:38312) 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53bb59e700 1 -- 172.21.15.150:0/571516990 learned_addr learned my addr 172.21.15.150:0/571516990 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53ba59c700 1 --2- 172.21.15.150:0/571516990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581570 0x563577589240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53bad9d700 1 --2- 172.21.15.150:0/571516990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635775806a0 0x563577588d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53bb59e700 1 -- 172.21.15.150:0/571516990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581570 msgr2=0x563577589240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53bb59e700 1 --2- 172.21.15.150:0/571516990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581570 0x563577589240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53bb59e700 1 -- 172.21.15.150:0/571516990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635775806a0 msgr2=0x563577588d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53bb59e700 1 --2- 172.21.15.150:0/571516990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635775806a0 0x563577588d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53bb59e700 1 -- 172.21.15.150:0/571516990 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563577466560 con 0x563577581ec0 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53ba59c700 1 --2- 172.21.15.150:0/571516990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581570 0x563577589240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.029+0000 7f53bad9d700 1 --2- 172.21.15.150:0/571516990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635775806a0 0x563577588d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.030+0000 7f53bb59e700 1 --2- 172.21.15.150:0/571516990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x5635775852b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f53b40124f0 tx=0x7f53b400b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ee9490eafdd233 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.030+0000 7f53a1ffb700 1 -- 172.21.15.150:0/571516990 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b400e070 con 0x563577581ec0 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.030+0000 7f53a1ffb700 1 -- 172.21.15.150:0/571516990 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53b401b7d0 con 0x563577581ec0 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.030+0000 7f53a1ffb700 1 -- 172.21.15.150:0/571516990 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b402c870 con 0x563577581ec0 2024-05-10T04:04:43.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.030+0000 7f53d13f9b80 1 -- 172.21.15.150:0/571516990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 msgr2=0x5635775852b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.030+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/571516990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x5635775852b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f53b40124f0 tx=0x7f53b400b060 comp rx=0 tx=0).stop 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.030+0000 7f53d13f9b80 1 -- 172.21.15.150:0/571516990 shutdown_connections 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.030+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/571516990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563577581ec0 0x5635775852b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.030+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/571516990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635775806a0 0x563577588d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.030+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/571516990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581570 0x563577589240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.030+0000 7f53d13f9b80 1 -- 172.21.15.150:0/571516990 >> 172.21.15.150:0/571516990 conn(0x5635774e2a60 msgr2=0x5635775847a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.030+0000 7f53d13f9b80 1 -- 172.21.15.150:0/571516990 shutdown_connections 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.031+0000 7f53d13f9b80 1 -- 172.21.15.150:0/571516990 wait complete. 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.031+0000 7f53d13f9b80 1 -- start start 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.031+0000 7f53d13f9b80 1 Processor -- start 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.031+0000 7f53d13f9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635775806a0 0x5635775ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.031+0000 7f53d13f9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563577581570 0x5635775ec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.031+0000 7f53d13f9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581ec0 0x5635775eed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.031+0000 7f53d13f9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635775ef2e0 con 0x5635775806a0 2024-05-10T04:04:43.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.031+0000 7f53d13f9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635775ef450 con 0x563577581570 2024-05-10T04:04:43.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.031+0000 7f53d13f9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635775ef610 con 0x563577581ec0 2024-05-10T04:04:43.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.031+0000 7f53d13f9b80 10 client.0 _ll_register_callbacks cb 0x563577445b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:43.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.031+0000 7f53bad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635775806a0 0x5635775ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.031+0000 7f53bad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635775806a0 0x5635775ea2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38316/0 (socket says 172.21.15.150:38316) 2024-05-10T04:04:43.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.031+0000 7f53bad9d700 1 -- 172.21.15.150:0/122894132 learned_addr learned my addr 172.21.15.150:0/122894132 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f53bb59e700 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581ec0 0x5635775eed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f53ba59c700 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563577581570 0x5635775ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f53bad9d700 1 -- 172.21.15.150:0/122894132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581ec0 msgr2=0x5635775eed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f53bad9d700 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581ec0 0x5635775eed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f53bad9d700 1 -- 172.21.15.150:0/122894132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563577581570 msgr2=0x5635775ec810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f53bad9d700 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563577581570 0x5635775ec810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f53bad9d700 1 -- 172.21.15.150:0/122894132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635772cdb60 con 0x5635775806a0 2024-05-10T04:04:43.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f53ba59c700 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563577581570 0x5635775ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:43.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f53bad9d700 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635775806a0 0x5635775ea2c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00c940 tx=0x7f53ac007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea88a029b58d590 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f539eff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:43.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f539eff5700 1 -- 172.21.15.150:0/122894132 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0b1070 con 0x5635775806a0 2024-05-10T04:04:43.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f539eff5700 1 -- 172.21.15.150:0/122894132 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f53ac0a1ea0 con 0x5635775806a0 2024-05-10T04:04:43.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f53d13f9b80 1 -- 172.21.15.150:0/122894132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53b4004a50 con 0x5635775806a0 2024-05-10T04:04:43.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f539eff5700 1 -- 172.21.15.150:0/122894132 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53ac0ca330 con 0x5635775806a0 2024-05-10T04:04:43.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f53d13f9b80 10 client.15708 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:43.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f53d13f9b80 1 -- 172.21.15.150:0/122894132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563577325cd0 con 0x5635775806a0 2024-05-10T04:04:43.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f539eff5700 1 -- 172.21.15.150:0/122894132 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53b4004a50 con 0x5635775806a0 2024-05-10T04:04:43.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.032+0000 7f539eff5700 1 -- 172.21.15.150:0/122894132 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f53ac099c80 con 0x5635775806a0 2024-05-10T04:04:43.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f53d13f9b80 10 client.15708 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f53d13f9b80 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f53d13f9b80 20 client.15708 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5635775f0ec0 0x5635775f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f53d13f9b80 4 client.15708 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f53d13f9b80 1 -- 172.21.15.150:0/122894132 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563577498ec0 con 0x5635775f0ec0 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f53ba59c700 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5635775f0ec0 0x5635775f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f539eff5700 10 client.15708 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f53ba59c700 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5635775f0ec0 0x5635775f32e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f5743d01723776 server_cookie=ba290505cd7051e1 in_seq=0 out_seq=0 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f539eff5700 1 -- 172.21.15.150:0/122894132 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563577498ec0 con 0x5635775f0ec0 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f539eff5700 10 client.15708 handle_command_reply: tid=0 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f53d13f9b80 1 client.15708 shutdown 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f53d13f9b80 1 -- 172.21.15.150:0/122894132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635775806a0 msgr2=0x5635775ea2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635775806a0 0x5635775ea2c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00c940 tx=0x7f53ac007f30 comp rx=0 tx=0).stop 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.033+0000 7f53d13f9b80 1 -- 172.21.15.150:0/122894132 shutdown_connections 2024-05-10T04:04:43.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5635775f0ec0 0x5635775f32e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563577581ec0 0x5635775eed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f539eff5700 0 client.15708 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:43.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635775806a0 0x5635775ea2c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53d13f9b80 1 --2- 172.21.15.150:0/122894132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563577581570 0x5635775ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53d13f9b80 1 -- 172.21.15.150:0/122894132 >> 172.21.15.150:0/122894132 conn(0x5635774e2a60 msgr2=0x563577588c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53d13f9b80 1 -- 172.21.15.150:0/122894132 shutdown_connections 2024-05-10T04:04:43.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53d13f9b80 1 -- 172.21.15.150:0/122894132 wait complete. 2024-05-10T04:04:43.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53d13f9b80 20 client.15708 trim_cache size 0 max 16384 2024-05-10T04:04:43.043 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:04:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "no active scrubs running", 2024-05-10T04:04:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": {} 2024-05-10T04:04:43.044 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:04:43.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53c10b5700 1 -- 172.21.15.150:0/170485436 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f53a40a1260 msgr2=0x7f53a40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53c10b5700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f53a40a1260 0x7f53a40a3770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f53bc197fc0 tx=0x7f53b0008040 comp rx=0 tx=0).stop 2024-05-10T04:04:43.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53c10b5700 1 -- 172.21.15.150:0/170485436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53bc100ac0 msgr2=0x7f53bc196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53c10b5700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53bc100ac0 0x7f53bc196f20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00c970 tx=0x7f53ac00cd30 comp rx=0 tx=0).stop 2024-05-10T04:04:43.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53c10b5700 1 -- 172.21.15.150:0/170485436 shutdown_connections 2024-05-10T04:04:43.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53c10b5700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f53a40a1260 0x7f53a40a3770 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53c10b5700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc102580 0x7f53bc19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53c10b5700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53bc100ac0 0x7f53bc196f20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53c10b5700 1 --2- 172.21.15.150:0/170485436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53bc101be0 0x7f53bc197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53c10b5700 1 -- 172.21.15.150:0/170485436 >> 172.21.15.150:0/170485436 conn(0x7f53bc0fc0d0 msgr2=0x7f53bc104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53c10b5700 1 -- 172.21.15.150:0/170485436 shutdown_connections 2024-05-10T04:04:43.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.034+0000 7f53c10b5700 1 -- 172.21.15.150:0/170485436 wait complete. 2024-05-10T04:04:43.049 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T04:04:43.049 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:04:43.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.180+0000 7f5701f86700 1 -- 172.21.15.150:0/328959067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56fc100ac0 msgr2=0x7f56fc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.180+0000 7f5701f86700 1 --2- 172.21.15.150:0/328959067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56fc100ac0 0x7f56fc100ed0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009990 tx=0x7f56f0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.180+0000 7f5701f86700 1 -- 172.21.15.150:0/328959067 shutdown_connections 2024-05-10T04:04:43.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.180+0000 7f5701f86700 1 --2- 172.21.15.150:0/328959067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56fc102580 0x7f56fc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.180+0000 7f5701f86700 1 --2- 172.21.15.150:0/328959067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56fc100ac0 0x7f56fc100ed0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.180+0000 7f5701f86700 1 --2- 172.21.15.150:0/328959067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56fc101be0 0x7f56fc101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.180+0000 7f5701f86700 1 -- 172.21.15.150:0/328959067 >> 172.21.15.150:0/328959067 conn(0x7f56fc0fc0d0 msgr2=0x7f56fc0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.180+0000 7f5701f86700 1 -- 172.21.15.150:0/328959067 shutdown_connections 2024-05-10T04:04:43.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.180+0000 7f5701f86700 1 -- 172.21.15.150:0/328959067 wait complete. 2024-05-10T04:04:43.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.181+0000 7f5701f86700 1 Processor -- start 2024-05-10T04:04:43.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.181+0000 7f5701f86700 1 -- start start 2024-05-10T04:04:43.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.181+0000 7f5701f86700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56fc100ac0 0x7f56fc196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.181+0000 7f5701f86700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56fc101be0 0x7f56fc197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.181+0000 7f5701f86700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56fc102580 0x7f56fc19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.181+0000 7f5701f86700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc19e9d0 con 0x7f56fc101be0 2024-05-10T04:04:43.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.181+0000 7f5701f86700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc19eb40 con 0x7f56fc102580 2024-05-10T04:04:43.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.181+0000 7f5701f86700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc19ed00 con 0x7f56fc100ac0 2024-05-10T04:04:43.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.181+0000 7f56faffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56fc101be0 0x7f56fc197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f56faffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56fc101be0 0x7f56fc197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38326/0 (socket says 172.21.15.150:38326) 2024-05-10T04:04:43.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f56faffd700 1 -- 172.21.15.150:0/2788082056 learned_addr learned my addr 172.21.15.150:0/2788082056 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f56fb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56fc100ac0 0x7f56fc196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f56fbfff700 1 --2- 172.21.15.150:0/2788082056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56fc102580 0x7f56fc19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f56faffd700 1 -- 172.21.15.150:0/2788082056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56fc100ac0 msgr2=0x7f56fc196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f56faffd700 1 --2- 172.21.15.150:0/2788082056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56fc100ac0 0x7f56fc196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f56faffd700 1 -- 172.21.15.150:0/2788082056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56fc102580 msgr2=0x7f56fc19e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f56faffd700 1 --2- 172.21.15.150:0/2788082056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56fc102580 0x7f56fc19e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f56faffd700 1 -- 172.21.15.150:0/2788082056 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f00095f0 con 0x7f56fc101be0 2024-05-10T04:04:43.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f56faffd700 1 --2- 172.21.15.150:0/2788082056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56fc101be0 0x7f56fc197430 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00eab0 tx=0x7f56ec00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f56f8ff9700 1 -- 172.21.15.150:0/2788082056 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec013070 con 0x7f56fc101be0 2024-05-10T04:04:43.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f56f8ff9700 1 -- 172.21.15.150:0/2788082056 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56ec00be80 con 0x7f56fc101be0 2024-05-10T04:04:43.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f5701f86700 1 -- 172.21.15.150:0/2788082056 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56fc068420 con 0x7f56fc101be0 2024-05-10T04:04:43.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f56f8ff9700 1 -- 172.21.15.150:0/2788082056 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0163b0 con 0x7f56fc101be0 2024-05-10T04:04:43.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.182+0000 7f5701f86700 1 -- 172.21.15.150:0/2788082056 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56fc068930 con 0x7f56fc101be0 2024-05-10T04:04:43.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.183+0000 7f5701f86700 1 -- 172.21.15.150:0/2788082056 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56fc069380 con 0x7f56fc101be0 2024-05-10T04:04:43.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.185+0000 7f56f8ff9700 1 -- 172.21.15.150:0/2788082056 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f56ec0175a0 con 0x7f56fc101be0 2024-05-10T04:04:43.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.185+0000 7f56f8ff9700 1 --2- 172.21.15.150:0/2788082056 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56e40a11b0 0x7f56e40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.185+0000 7f56fb7fe700 1 --2- 172.21.15.150:0/2788082056 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56e40a11b0 0x7f56e40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.185+0000 7f56f8ff9700 1 -- 172.21.15.150:0/2788082056 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f56ec0ce610 con 0x7f56fc101be0 2024-05-10T04:04:43.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.186+0000 7f56fb7fe700 1 --2- 172.21.15.150:0/2788082056 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56e40a11b0 0x7f56e40a36c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f56f000b7b0 tx=0x7f56f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.189+0000 7f56f8ff9700 1 -- 172.21.15.150:0/2788082056 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f56ec097930 con 0x7f56fc101be0 2024-05-10T04:04:43.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.432+0000 7f5701f86700 1 -- 172.21.15.150:0/2788082056 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56fc10c5a0 con 0x7f56fc101be0 2024-05-10T04:04:43.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.433+0000 7f56f8ff9700 1 -- 172.21.15.150:0/2788082056 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10886 (secure 0 0 0) 0x7f56ec09af50 con 0x7f56fc101be0 2024-05-10T04:04:43.435 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:04:43.435 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:04:04.157819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:04:43.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.435+0000 7f5701f86700 1 -- 172.21.15.150:0/2788082056 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56e40a11b0 msgr2=0x7f56e40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.435+0000 7f5701f86700 1 --2- 172.21.15.150:0/2788082056 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56e40a11b0 0x7f56e40a36c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f56f000b7b0 tx=0x7f56f001a040 comp rx=0 tx=0).stop 2024-05-10T04:04:43.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.435+0000 7f5701f86700 1 -- 172.21.15.150:0/2788082056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56fc101be0 msgr2=0x7f56fc197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.435+0000 7f5701f86700 1 --2- 172.21.15.150:0/2788082056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56fc101be0 0x7f56fc197430 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00eab0 tx=0x7f56ec00edc0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.435+0000 7f5701f86700 1 -- 172.21.15.150:0/2788082056 shutdown_connections 2024-05-10T04:04:43.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.435+0000 7f5701f86700 1 --2- 172.21.15.150:0/2788082056 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f56e40a11b0 0x7f56e40a36c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.435+0000 7f5701f86700 1 --2- 172.21.15.150:0/2788082056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56fc102580 0x7f56fc19e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.435+0000 7f5701f86700 1 --2- 172.21.15.150:0/2788082056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56fc100ac0 0x7f56fc196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.435+0000 7f5701f86700 1 --2- 172.21.15.150:0/2788082056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56fc101be0 0x7f56fc197430 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.435+0000 7f5701f86700 1 -- 172.21.15.150:0/2788082056 >> 172.21.15.150:0/2788082056 conn(0x7f56fc0fc0d0 msgr2=0x7f56fc104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.435+0000 7f5701f86700 1 -- 172.21.15.150:0/2788082056 shutdown_connections 2024-05-10T04:04:43.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.435+0000 7f5701f86700 1 -- 172.21.15.150:0/2788082056 wait complete. 2024-05-10T04:04:43.439 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 25 2024-05-10T04:04:43.451 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T04:04:43.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.584+0000 7f5479262700 1 -- 172.21.15.150:0/1296843437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474102630 msgr2=0x7f5474108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.585+0000 7f5479262700 1 --2- 172.21.15.150:0/1296843437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474102630 0x7f5474108ee0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f546c00b0a0 tx=0x7f546c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.585+0000 7f5479262700 1 -- 172.21.15.150:0/1296843437 shutdown_connections 2024-05-10T04:04:43.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.585+0000 7f5479262700 1 --2- 172.21.15.150:0/1296843437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474102630 0x7f5474108ee0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.585+0000 7f5479262700 1 --2- 172.21.15.150:0/1296843437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5474100b70 0x7f5474100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.585+0000 7f5479262700 1 --2- 172.21.15.150:0/1296843437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5474101c90 0x7f54741020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.585+0000 7f5479262700 1 -- 172.21.15.150:0/1296843437 >> 172.21.15.150:0/1296843437 conn(0x7f54740fc180 msgr2=0x7f54740fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.585+0000 7f5479262700 1 -- 172.21.15.150:0/1296843437 shutdown_connections 2024-05-10T04:04:43.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.585+0000 7f5479262700 1 -- 172.21.15.150:0/1296843437 wait complete. 2024-05-10T04:04:43.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.585+0000 7f5479262700 1 Processor -- start 2024-05-10T04:04:43.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.585+0000 7f5479262700 1 -- start start 2024-05-10T04:04:43.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f5479262700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5474100b70 0x7f547406a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f5479262700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5474101c90 0x7f547419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f5479262700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474102630 0x7f547419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f5479262700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547419fad0 con 0x7f5474102630 2024-05-10T04:04:43.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f5479262700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547419fc40 con 0x7f5474100b70 2024-05-10T04:04:43.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f5479262700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547419fe00 con 0x7f5474101c90 2024-05-10T04:04:43.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f54737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474102630 0x7f547419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f54737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474102630 0x7f547419f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38342/0 (socket says 172.21.15.150:38342) 2024-05-10T04:04:43.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f54737fe700 1 -- 172.21.15.150:0/3210157460 learned_addr learned my addr 172.21.15.150:0/3210157460 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f5472ffd700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5474100b70 0x7f547406a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f54727fc700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5474101c90 0x7f547419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f54737fe700 1 -- 172.21.15.150:0/3210157460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5474101c90 msgr2=0x7f547419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f54737fe700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5474101c90 0x7f547419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f54737fe700 1 -- 172.21.15.150:0/3210157460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5474100b70 msgr2=0x7f547406a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f54737fe700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5474100b70 0x7f547406a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f54737fe700 1 -- 172.21.15.150:0/3210157460 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c009d00 con 0x7f5474102630 2024-05-10T04:04:43.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f54737fe700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474102630 0x7f547419f3d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f546c006070 tx=0x7f546c009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f545bfff700 1 -- 172.21.15.150:0/3210157460 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c00e040 con 0x7f5474102630 2024-05-10T04:04:43.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f545bfff700 1 -- 172.21.15.150:0/3210157460 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f546c004ce0 con 0x7f5474102630 2024-05-10T04:04:43.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.586+0000 7f5479262700 1 -- 172.21.15.150:0/3210157460 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54741a0080 con 0x7f5474102630 2024-05-10T04:04:43.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.587+0000 7f545bfff700 1 -- 172.21.15.150:0/3210157460 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c01d480 con 0x7f5474102630 2024-05-10T04:04:43.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.587+0000 7f5479262700 1 -- 172.21.15.150:0/3210157460 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54741a0500 con 0x7f5474102630 2024-05-10T04:04:43.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.589+0000 7f545bfff700 1 -- 172.21.15.150:0/3210157460 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f546c003a00 con 0x7f5474102630 2024-05-10T04:04:43.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.589+0000 7f545bfff700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f545c0a1260 0x7f545c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.590+0000 7f545bfff700 1 -- 172.21.15.150:0/3210157460 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f546c0ce140 con 0x7f5474102630 2024-05-10T04:04:43.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.590+0000 7f5472ffd700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f545c0a1260 0x7f545c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.590+0000 7f5472ffd700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f545c0a1260 0x7f545c0a3770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5474101b90 tx=0x7f5464007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.590+0000 7f54895a6b80 1 Processor -- start 2024-05-10T04:04:43.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.590+0000 7f54895a6b80 1 -- start start 2024-05-10T04:04:43.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.590+0000 7f54895a6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 0x562a6b0e1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.590+0000 7f54895a6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e2570 0x562a6b0e2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.590+0000 7f54895a6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2ec0 0x562a6b0e97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.590+0000 7f54895a6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b008150 con 0x562a6b0e16a0 2024-05-10T04:04:43.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.590+0000 7f54895a6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b005b10 con 0x562a6b0e2570 2024-05-10T04:04:43.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.590+0000 7f54895a6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b001400 con 0x562a6b0e2ec0 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f5472ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 0x562a6b0e1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f5472ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 0x562a6b0e1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38354/0 (socket says 172.21.15.150:38354) 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f5472ffd700 1 -- 172.21.15.150:0/246662931 learned_addr learned my addr 172.21.15.150:0/246662931 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f54727fc700 1 --2- 172.21.15.150:0/246662931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e2570 0x562a6b0e2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f54737fe700 1 --2- 172.21.15.150:0/246662931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2ec0 0x562a6b0e97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f5472ffd700 1 -- 172.21.15.150:0/246662931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2ec0 msgr2=0x562a6b0e97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f5472ffd700 1 --2- 172.21.15.150:0/246662931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2ec0 0x562a6b0e97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f5472ffd700 1 -- 172.21.15.150:0/246662931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e2570 msgr2=0x562a6b0e2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f5472ffd700 1 --2- 172.21.15.150:0/246662931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e2570 0x562a6b0e2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f5472ffd700 1 -- 172.21.15.150:0/246662931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a6ae823b0 con 0x562a6b0e16a0 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f54737fe700 1 --2- 172.21.15.150:0/246662931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2ec0 0x562a6b0e97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f54727fc700 1 --2- 172.21.15.150:0/246662931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e2570 0x562a6b0e2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f5472ffd700 1 --2- 172.21.15.150:0/246662931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 0x562a6b0e1af0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5464010e20 tx=0x7f5464013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ed6598939f1be5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f54597fa700 1 -- 172.21.15.150:0/246662931 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464004220 con 0x562a6b0e16a0 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f54597fa700 1 -- 172.21.15.150:0/246662931 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5464015070 con 0x562a6b0e16a0 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.591+0000 7f54597fa700 1 -- 172.21.15.150:0/246662931 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546401e680 con 0x562a6b0e16a0 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.592+0000 7f54895a6b80 1 -- 172.21.15.150:0/246662931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 msgr2=0x562a6b0e1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.592+0000 7f54895a6b80 1 --2- 172.21.15.150:0/246662931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 0x562a6b0e1af0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5464010e20 tx=0x7f5464013000 comp rx=0 tx=0).stop 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.592+0000 7f54895a6b80 1 -- 172.21.15.150:0/246662931 shutdown_connections 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.592+0000 7f54895a6b80 1 --2- 172.21.15.150:0/246662931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2ec0 0x562a6b0e97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.592+0000 7f54895a6b80 1 --2- 172.21.15.150:0/246662931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 0x562a6b0e1af0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.592+0000 7f54895a6b80 1 --2- 172.21.15.150:0/246662931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e2570 0x562a6b0e2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.592+0000 7f54895a6b80 1 -- 172.21.15.150:0/246662931 >> 172.21.15.150:0/246662931 conn(0x562a6b043a60 msgr2=0x562a6b0dbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.592+0000 7f54895a6b80 1 -- 172.21.15.150:0/246662931 shutdown_connections 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.592+0000 7f54895a6b80 1 -- 172.21.15.150:0/246662931 wait complete. 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.592+0000 7f54895a6b80 1 -- start start 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.592+0000 7f54895a6b80 1 Processor -- start 2024-05-10T04:04:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f54895a6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 0x562a6b0eb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f54895a6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2570 0x562a6b0ebc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f54895a6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e2ec0 0x562a6b0f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f54895a6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6af9d560 con 0x562a6b0e16a0 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f54895a6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6af9d200 con 0x562a6b0e2ec0 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f54895a6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6ae846a0 con 0x562a6b0e2570 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f54895a6b80 10 client.0 _ll_register_callbacks cb 0x562a6afa6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f5472ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 0x562a6b0eb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f5472ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 0x562a6b0eb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38364/0 (socket says 172.21.15.150:38364) 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f5472ffd700 1 -- 172.21.15.150:0/1446973708 learned_addr learned my addr 172.21.15.150:0/1446973708 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f54727fc700 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2570 0x562a6b0ebc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f54737fe700 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e2ec0 0x562a6b0f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f5472ffd700 1 -- 172.21.15.150:0/1446973708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2570 msgr2=0x562a6b0ebc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f5472ffd700 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2570 0x562a6b0ebc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f5472ffd700 1 -- 172.21.15.150:0/1446973708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e2ec0 msgr2=0x562a6b0f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f5472ffd700 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e2ec0 0x562a6b0f2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f5472ffd700 1 -- 172.21.15.150:0/1446973708 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5464010b00 con 0x562a6b0e16a0 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f54737fe700 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e2ec0 0x562a6b0f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f5472ffd700 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 0x562a6b0eb6e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5464007750 tx=0x7f54640039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b6db79c32c4f39 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.593+0000 7f5456ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f5456ff5700 1 -- 172.21.15.150:0/1446973708 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464005250 con 0x562a6b0e16a0 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f5456ff5700 1 -- 172.21.15.150:0/1446973708 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5464017070 con 0x562a6b0e16a0 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f54895a6b80 1 -- 172.21.15.150:0/1446973708 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a6af5a270 con 0x562a6b0e16a0 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f5456ff5700 1 -- 172.21.15.150:0/1446973708 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54640053b0 con 0x562a6b0e16a0 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f54895a6b80 10 client.15726 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f54895a6b80 1 -- 172.21.15.150:0/1446973708 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562a6ae2eb60 con 0x562a6b0e16a0 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f5456ff5700 1 -- 172.21.15.150:0/1446973708 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a6af5a270 con 0x562a6b0e16a0 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f5456ff5700 1 -- 172.21.15.150:0/1446973708 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5464024070 con 0x562a6b0e16a0 2024-05-10T04:04:43.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f54895a6b80 10 client.15726 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f54895a6b80 10 client.15726 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f54895a6b80 20 client.15726 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f54895a6b80 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x562a6b0f7290 0x562a6b0f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f54895a6b80 4 client.15726 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f54895a6b80 1 -- 172.21.15.150:0/1446973708 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a6b0b0780 con 0x562a6b0f7290 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.594+0000 7f54727fc700 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x562a6b0f7290 0x562a6b0f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.595+0000 7f5456ff5700 10 client.15726 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.595+0000 7f54727fc700 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x562a6b0f7290 0x562a6b0f96e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b15a023dd32d2308 server_cookie=142925cce23d2a97 in_seq=0 out_seq=0 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.595+0000 7f5456ff5700 1 -- 172.21.15.150:0/1446973708 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a6b0b0780 con 0x562a6b0f7290 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.595+0000 7f5456ff5700 10 client.15726 handle_command_reply: tid=0 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.595+0000 7f54895a6b80 1 client.15726 shutdown 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.596+0000 7f54895a6b80 1 -- 172.21.15.150:0/1446973708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 msgr2=0x562a6b0eb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.596+0000 7f54895a6b80 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 0x562a6b0eb6e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5464007750 tx=0x7f54640039f0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.596+0000 7f54895a6b80 1 -- 172.21.15.150:0/1446973708 shutdown_connections 2024-05-10T04:04:43.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.596+0000 7f54895a6b80 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e2ec0 0x562a6b0f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.596+0000 7f54895a6b80 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e16a0 0x562a6b0eb6e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.596+0000 7f54895a6b80 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x562a6b0f7290 0x562a6b0f96e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.596+0000 7f54895a6b80 1 --2- 172.21.15.150:0/1446973708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2570 0x562a6b0ebc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.596+0000 7f5456ff5700 0 client.15726 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:04:43.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.596+0000 7f54895a6b80 1 -- 172.21.15.150:0/1446973708 >> 172.21.15.150:0/1446973708 conn(0x562a6b043a60 msgr2=0x562a6b0dbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.596+0000 7f54895a6b80 1 -- 172.21.15.150:0/1446973708 shutdown_connections 2024-05-10T04:04:43.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.596+0000 7f54895a6b80 1 -- 172.21.15.150:0/1446973708 wait complete. 2024-05-10T04:04:43.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.596+0000 7f54895a6b80 20 client.15726 trim_cache size 0 max 16384 2024-05-10T04:04:43.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54895a6b80 1 Processor -- start 2024-05-10T04:04:43.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54895a6b80 1 -- start start 2024-05-10T04:04:43.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54895a6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e16a0 0x562a6b0f3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54895a6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2570 0x562a6b0f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54895a6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2ec0 0x562a6b0d8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54895a6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b0d6850 con 0x562a6b0e2ec0 2024-05-10T04:04:43.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54895a6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b0e96f0 con 0x562a6b0e16a0 2024-05-10T04:04:43.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54895a6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b0d86c0 con 0x562a6b0e2570 2024-05-10T04:04:43.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2ec0 0x562a6b0d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2ec0 0x562a6b0d8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38366/0 (socket says 172.21.15.150:38366) 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54737fe700 1 -- 172.21.15.150:0/3051361900 learned_addr learned my addr 172.21.15.150:0/3051361900 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f5472ffd700 1 --2- 172.21.15.150:0/3051361900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e16a0 0x562a6b0f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54737fe700 1 -- 172.21.15.150:0/3051361900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2570 msgr2=0x562a6b0f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54737fe700 1 --2- 172.21.15.150:0/3051361900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2570 0x562a6b0f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54737fe700 1 -- 172.21.15.150:0/3051361900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e16a0 msgr2=0x562a6b0f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54737fe700 1 --2- 172.21.15.150:0/3051361900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e16a0 0x562a6b0f3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.609+0000 7f54737fe700 1 -- 172.21.15.150:0/3051361900 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a6afc7560 con 0x562a6b0e2ec0 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.610+0000 7f54737fe700 1 --2- 172.21.15.150:0/3051361900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2ec0 0x562a6b0d8180 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f546c09aeb0 tx=0x7f546c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796f66101b6fb714 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.610+0000 7f5459ffb700 1 -- 172.21.15.150:0/3051361900 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c09b8b0 con 0x562a6b0e2ec0 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.610+0000 7f5459ffb700 1 -- 172.21.15.150:0/3051361900 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f546c09b1b0 con 0x562a6b0e2ec0 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.610+0000 7f5459ffb700 1 -- 172.21.15.150:0/3051361900 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c0a10c0 con 0x562a6b0e2ec0 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.610+0000 7f54895a6b80 1 -- 172.21.15.150:0/3051361900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2ec0 msgr2=0x562a6b0d8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.610+0000 7f54895a6b80 1 --2- 172.21.15.150:0/3051361900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2ec0 0x562a6b0d8180 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f546c09aeb0 tx=0x7f546c0099e0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.610+0000 7f54895a6b80 1 -- 172.21.15.150:0/3051361900 shutdown_connections 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.610+0000 7f54895a6b80 1 --2- 172.21.15.150:0/3051361900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2ec0 0x562a6b0d8180 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.610+0000 7f54895a6b80 1 --2- 172.21.15.150:0/3051361900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e16a0 0x562a6b0f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.610+0000 7f54895a6b80 1 --2- 172.21.15.150:0/3051361900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2570 0x562a6b0f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.610+0000 7f54895a6b80 1 -- 172.21.15.150:0/3051361900 >> 172.21.15.150:0/3051361900 conn(0x562a6b043a60 msgr2=0x562a6b0dbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.610+0000 7f54895a6b80 1 -- 172.21.15.150:0/3051361900 shutdown_connections 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.610+0000 7f54895a6b80 1 -- 172.21.15.150:0/3051361900 wait complete. 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f54895a6b80 1 -- start start 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f54895a6b80 1 Processor -- start 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f54895a6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e16a0 0x562a6b0f0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f54895a6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2570 0x562a6b0f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f54895a6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2ec0 0x562a6b0f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f54895a6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b0f1d10 con 0x562a6b0e2570 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f54895a6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b0f1e80 con 0x562a6b0e16a0 2024-05-10T04:04:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f54895a6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b0f2040 con 0x562a6b0e2ec0 2024-05-10T04:04:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f54895a6b80 10 client.0 _ll_register_callbacks cb 0x562a6afa6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f54727fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2570 0x562a6b0f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f54727fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2570 0x562a6b0f08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38382/0 (socket says 172.21.15.150:38382) 2024-05-10T04:04:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f54727fc700 1 -- 172.21.15.150:0/1453156680 learned_addr learned my addr 172.21.15.150:0/1453156680 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f5472ffd700 1 --2- 172.21.15.150:0/1453156680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e16a0 0x562a6b0f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.611+0000 7f54737fe700 1 --2- 172.21.15.150:0/1453156680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2ec0 0x562a6b0f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f54727fc700 1 -- 172.21.15.150:0/1453156680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2ec0 msgr2=0x562a6b0f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f54727fc700 1 --2- 172.21.15.150:0/1453156680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2ec0 0x562a6b0f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f54727fc700 1 -- 172.21.15.150:0/1453156680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e16a0 msgr2=0x562a6b0f0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f54727fc700 1 --2- 172.21.15.150:0/1453156680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e16a0 0x562a6b0f0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f54727fc700 1 -- 172.21.15.150:0/1453156680 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54741a0500 con 0x562a6b0e2570 2024-05-10T04:04:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f54727fc700 1 --2- 172.21.15.150:0/1453156680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2570 0x562a6b0f08a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5474069bf0 tx=0x7f546800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad261db48e6bad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f5456ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f5456ff5700 1 -- 172.21.15.150:0/1453156680 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468017040 con 0x562a6b0e2570 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f5456ff5700 1 -- 172.21.15.150:0/1453156680 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f546801c3b0 con 0x562a6b0e2570 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f54895a6b80 1 -- 172.21.15.150:0/1453156680 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5464015070 con 0x562a6b0e2570 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f5456ff5700 1 -- 172.21.15.150:0/1453156680 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5468003ed0 con 0x562a6b0e2570 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f54895a6b80 10 client.15732 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f54895a6b80 1 -- 172.21.15.150:0/1453156680 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562a6ae86cd0 con 0x562a6b0e2570 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f5456ff5700 1 -- 172.21.15.150:0/1453156680 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5464015070 con 0x562a6b0e2570 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f5456ff5700 1 -- 172.21.15.150:0/1453156680 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5468005120 con 0x562a6b0e2570 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f54895a6b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f54895a6b80 10 client.15732 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f54895a6b80 20 client.15732 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.612+0000 7f54895a6b80 1 --2- 172.21.15.150:0/1453156680 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x562a6b14e0a0 0x562a6b1504c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f54895a6b80 4 client.15732 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f54895a6b80 1 -- 172.21.15.150:0/1453156680 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a6aff9ec0 con 0x562a6b14e0a0 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f5472ffd700 1 --2- 172.21.15.150:0/1453156680 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x562a6b14e0a0 0x562a6b1504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f5456ff5700 10 client.15732 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f5472ffd700 1 --2- 172.21.15.150:0/1453156680 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x562a6b14e0a0 0x562a6b1504c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=128910517efbd49 server_cookie=21696798187b5f82 in_seq=0 out_seq=0 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f5456ff5700 1 -- 172.21.15.150:0/1453156680 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a6aff9ec0 con 0x562a6b14e0a0 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f5456ff5700 10 client.15732 handle_command_reply: tid=0 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f54895a6b80 1 client.15732 shutdown 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f54895a6b80 1 -- 172.21.15.150:0/1453156680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2570 msgr2=0x562a6b0f08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f54895a6b80 1 --2- 172.21.15.150:0/1453156680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2570 0x562a6b0f08a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5474069bf0 tx=0x7f546800bd60 comp rx=0 tx=0).stop 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f54895a6b80 1 -- 172.21.15.150:0/1453156680 shutdown_connections 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f54895a6b80 1 --2- 172.21.15.150:0/1453156680 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x562a6b14e0a0 0x562a6b1504c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f54895a6b80 1 --2- 172.21.15.150:0/1453156680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562a6b0e2ec0 0x562a6b0f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f54895a6b80 1 --2- 172.21.15.150:0/1453156680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b0e16a0 0x562a6b0f0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f5456ff5700 0 client.15732 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f54895a6b80 1 --2- 172.21.15.150:0/1453156680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562a6b0e2570 0x562a6b0f08a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.613+0000 7f54895a6b80 1 -- 172.21.15.150:0/1453156680 >> 172.21.15.150:0/1453156680 conn(0x562a6b043a60 msgr2=0x562a6b0e9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f54895a6b80 1 -- 172.21.15.150:0/1453156680 shutdown_connections 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f54895a6b80 1 -- 172.21.15.150:0/1453156680 wait complete. 2024-05-10T04:04:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f54895a6b80 20 client.15732 trim_cache size 0 max 16384 2024-05-10T04:04:43.622 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:04:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f5479262700 1 -- 172.21.15.150:0/3210157460 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f545c0a1260 msgr2=0x7f545c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f5479262700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f545c0a1260 0x7f545c0a3770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5474101b90 tx=0x7f5464007400 comp rx=0 tx=0).stop 2024-05-10T04:04:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f5479262700 1 -- 172.21.15.150:0/3210157460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474102630 msgr2=0x7f547419f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f5479262700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474102630 0x7f547419f3d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f546c006070 tx=0x7f546c009440 comp rx=0 tx=0).stop 2024-05-10T04:04:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f5479262700 1 -- 172.21.15.150:0/3210157460 shutdown_connections 2024-05-10T04:04:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f5479262700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f545c0a1260 0x7f545c0a3770 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f5479262700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474102630 0x7f547419f3d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f5479262700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5474100b70 0x7f547406a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f5479262700 1 --2- 172.21.15.150:0/3210157460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5474101c90 0x7f547419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f5479262700 1 -- 172.21.15.150:0/3210157460 >> 172.21.15.150:0/3210157460 conn(0x7f54740fc180 msgr2=0x7f5474104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f5479262700 1 -- 172.21.15.150:0/3210157460 shutdown_connections 2024-05-10T04:04:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.614+0000 7f5479262700 1 -- 172.21.15.150:0/3210157460 wait complete. 2024-05-10T04:04:43.629 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T04:04:43.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.764+0000 7f45e0cca700 1 -- 172.21.15.150:0/2355284757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45dc1031a0 msgr2=0x7f45dc1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.764+0000 7f45e0cca700 1 --2- 172.21.15.150:0/2355284757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45dc1031a0 0x7f45dc1035b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009990 tx=0x7f45d0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.764+0000 7f45e0cca700 1 -- 172.21.15.150:0/2355284757 shutdown_connections 2024-05-10T04:04:43.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.764+0000 7f45e0cca700 1 --2- 172.21.15.150:0/2355284757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45dc100530 0x7f45dc10a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.764+0000 7f45e0cca700 1 --2- 172.21.15.150:0/2355284757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45dc1031a0 0x7f45dc1035b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.764+0000 7f45e0cca700 1 --2- 172.21.15.150:0/2355284757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45dc0ffbe0 0x7f45dc0ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.764+0000 7f45e0cca700 1 -- 172.21.15.150:0/2355284757 >> 172.21.15.150:0/2355284757 conn(0x7f45dc0fb8b0 msgr2=0x7f45dc0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.764+0000 7f45e0cca700 1 -- 172.21.15.150:0/2355284757 shutdown_connections 2024-05-10T04:04:43.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.764+0000 7f45e0cca700 1 -- 172.21.15.150:0/2355284757 wait complete. 2024-05-10T04:04:43.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.764+0000 7f45e0cca700 1 Processor -- start 2024-05-10T04:04:43.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45e0cca700 1 -- start start 2024-05-10T04:04:43.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45e0cca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45dc0ffbe0 0x7f45dc197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45e0cca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45dc100530 0x7f45dc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45e0cca700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45dc1031a0 0x7f45dc19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45e0cca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc19f990 con 0x7f45dc0ffbe0 2024-05-10T04:04:43.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45e0cca700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc19fb00 con 0x7f45dc1031a0 2024-05-10T04:04:43.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45e0cca700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc19fcc0 con 0x7f45dc100530 2024-05-10T04:04:43.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45da59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45dc0ffbe0 0x7f45dc197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45da59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45dc0ffbe0 0x7f45dc197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38406/0 (socket says 172.21.15.150:38406) 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45da59c700 1 -- 172.21.15.150:0/3021981031 learned_addr learned my addr 172.21.15.150:0/3021981031 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45d9d9b700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45dc100530 0x7f45dc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45dad9d700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45dc1031a0 0x7f45dc19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45da59c700 1 -- 172.21.15.150:0/3021981031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45dc100530 msgr2=0x7f45dc1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45da59c700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45dc100530 0x7f45dc1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45da59c700 1 -- 172.21.15.150:0/3021981031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45dc1031a0 msgr2=0x7f45dc19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45da59c700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45dc1031a0 0x7f45dc19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45da59c700 1 -- 172.21.15.150:0/3021981031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d00095f0 con 0x7f45dc0ffbe0 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45da59c700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45dc0ffbe0 0x7f45dc197eb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f45d00098e0 tx=0x7f45d0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.765+0000 7f45cb7fe700 1 -- 172.21.15.150:0/3021981031 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d001d070 con 0x7f45dc0ffbe0 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.766+0000 7f45cb7fe700 1 -- 172.21.15.150:0/3021981031 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45d000fbd0 con 0x7f45dc0ffbe0 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.766+0000 7f45e0cca700 1 -- 172.21.15.150:0/3021981031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45dc19ff40 con 0x7f45dc0ffbe0 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.766+0000 7f45cb7fe700 1 -- 172.21.15.150:0/3021981031 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0017650 con 0x7f45dc0ffbe0 2024-05-10T04:04:43.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.766+0000 7f45e0cca700 1 -- 172.21.15.150:0/3021981031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc1a0300 con 0x7f45dc0ffbe0 2024-05-10T04:04:43.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.768+0000 7f45cb7fe700 1 -- 172.21.15.150:0/3021981031 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f45d00177b0 con 0x7f45dc0ffbe0 2024-05-10T04:04:43.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.768+0000 7f45cb7fe700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f45c40a12e0 0x7f45c40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.769+0000 7f45d9d9b700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f45c40a12e0 0x7f45c40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.769+0000 7f45cb7fe700 1 -- 172.21.15.150:0/3021981031 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f45d00ce0b0 con 0x7f45dc0ffbe0 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.769+0000 7f45f100eb80 1 Processor -- start 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.769+0000 7f45d9d9b700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f45c40a12e0 0x7f45c40a37f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f45cc005950 tx=0x7f45cc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.769+0000 7f45f100eb80 1 -- start start 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.769+0000 7f45f100eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089b6a0 0x55708089baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.769+0000 7f45f100eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089c570 0x55708089c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.769+0000 7f45f100eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089cec0 0x5570808a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45f100eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570807c2150 con 0x55708089b6a0 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45f100eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570807bfb10 con 0x55708089cec0 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45f100eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570807bb400 con 0x55708089c570 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45da59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089b6a0 0x55708089baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45da59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089b6a0 0x55708089baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38420/0 (socket says 172.21.15.150:38420) 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45da59c700 1 -- 172.21.15.150:0/2359273241 learned_addr learned my addr 172.21.15.150:0/2359273241 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45dad9d700 1 --2- 172.21.15.150:0/2359273241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089cec0 0x5570808a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45d9d9b700 1 --2- 172.21.15.150:0/2359273241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089c570 0x55708089c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45da59c700 1 -- 172.21.15.150:0/2359273241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089c570 msgr2=0x55708089c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45da59c700 1 --2- 172.21.15.150:0/2359273241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089c570 0x55708089c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45da59c700 1 -- 172.21.15.150:0/2359273241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089cec0 msgr2=0x5570808a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45da59c700 1 --2- 172.21.15.150:0/2359273241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089cec0 0x5570808a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45da59c700 1 -- 172.21.15.150:0/2359273241 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55708063c3b0 con 0x55708089b6a0 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45dad9d700 1 --2- 172.21.15.150:0/2359273241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089cec0 0x5570808a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45da59c700 1 --2- 172.21.15.150:0/2359273241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089b6a0 0x55708089baf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f45d0097730 tx=0x7f45d0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458eb29120a11edc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45c8ff9700 1 -- 172.21.15.150:0/2359273241 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d000fd40 con 0x55708089b6a0 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45c8ff9700 1 -- 172.21.15.150:0/2359273241 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45d009b930 con 0x55708089b6a0 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.770+0000 7f45c8ff9700 1 -- 172.21.15.150:0/2359273241 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d009a980 con 0x55708089b6a0 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.771+0000 7f45f100eb80 1 -- 172.21.15.150:0/2359273241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089b6a0 msgr2=0x55708089baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.771+0000 7f45f100eb80 1 --2- 172.21.15.150:0/2359273241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089b6a0 0x55708089baf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f45d0097730 tx=0x7f45d0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.771+0000 7f45f100eb80 1 -- 172.21.15.150:0/2359273241 shutdown_connections 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.771+0000 7f45f100eb80 1 --2- 172.21.15.150:0/2359273241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089cec0 0x5570808a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.771+0000 7f45f100eb80 1 --2- 172.21.15.150:0/2359273241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089b6a0 0x55708089baf0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.771+0000 7f45f100eb80 1 --2- 172.21.15.150:0/2359273241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089c570 0x55708089c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.771+0000 7f45f100eb80 1 -- 172.21.15.150:0/2359273241 >> 172.21.15.150:0/2359273241 conn(0x5570807fda60 msgr2=0x557080895fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.771+0000 7f45f100eb80 1 -- 172.21.15.150:0/2359273241 shutdown_connections 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.771+0000 7f45f100eb80 1 -- 172.21.15.150:0/2359273241 wait complete. 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45f100eb80 1 -- start start 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45f100eb80 1 Processor -- start 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45f100eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089b6a0 0x5570808a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45f100eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089c570 0x5570808a5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45f100eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089cec0 0x5570808ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45f100eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557080757560 con 0x55708089c570 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45f100eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557080757200 con 0x55708089b6a0 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45f100eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708063e6a0 con 0x55708089cec0 2024-05-10T04:04:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45f100eb80 10 client.0 _ll_register_callbacks cb 0x557080760b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45d9d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089c570 0x5570808a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45da59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089b6a0 0x5570808a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45d9d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089c570 0x5570808a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38422/0 (socket says 172.21.15.150:38422) 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45d9d9b700 1 -- 172.21.15.150:0/263470523 learned_addr learned my addr 172.21.15.150:0/263470523 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.772+0000 7f45dad9d700 1 --2- 172.21.15.150:0/263470523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089cec0 0x5570808ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45d9d9b700 1 -- 172.21.15.150:0/263470523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089cec0 msgr2=0x5570808ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45d9d9b700 1 --2- 172.21.15.150:0/263470523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089cec0 0x5570808ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45d9d9b700 1 -- 172.21.15.150:0/263470523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089b6a0 msgr2=0x5570808a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45d9d9b700 1 --2- 172.21.15.150:0/263470523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089b6a0 0x5570808a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45d9d9b700 1 -- 172.21.15.150:0/263470523 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc1a0300 con 0x55708089c570 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45da59c700 1 --2- 172.21.15.150:0/263470523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089b6a0 0x5570808a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45d9d9b700 1 --2- 172.21.15.150:0/263470523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089c570 0x5570808a5910 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00ffa0 tx=0x7f45cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c36372c25e8796d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45c1ffb700 1 -- 172.21.15.150:0/263470523 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc01f430 con 0x55708089c570 2024-05-10T04:04:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45c1ffb700 1 -- 172.21.15.150:0/263470523 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f45cc01f590 con 0x55708089c570 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45f100eb80 1 -- 172.21.15.150:0/263470523 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557080714270 con 0x55708089c570 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45c1ffb700 1 -- 172.21.15.150:0/263470523 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45cc01f750 con 0x55708089c570 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45f100eb80 10 client.15744 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45f100eb80 1 -- 172.21.15.150:0/263470523 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5570805e8b60 con 0x55708089c570 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45c1ffb700 1 -- 172.21.15.150:0/263470523 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557080714270 con 0x55708089c570 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45c1ffb700 1 -- 172.21.15.150:0/263470523 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f45cc025020 con 0x55708089c570 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45f100eb80 10 client.15744 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.773+0000 7f45f100eb80 10 client.15744 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.774+0000 7f45f100eb80 20 client.15744 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.774+0000 7f45f100eb80 1 --2- 172.21.15.150:0/263470523 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x5570808b1100 0x5570808b3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.774+0000 7f45f100eb80 4 client.15744 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.774+0000 7f45f100eb80 1 -- 172.21.15.150:0/263470523 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55708086a780 con 0x5570808b1100 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.774+0000 7f45da59c700 1 --2- 172.21.15.150:0/263470523 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x5570808b1100 0x5570808b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.774+0000 7f45da59c700 1 --2- 172.21.15.150:0/263470523 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x5570808b1100 0x5570808b3550 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1693af8e1986c1db server_cookie=99081529df2d04a8 in_seq=0 out_seq=0 2024-05-10T04:04:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.774+0000 7f45c1ffb700 10 client.15744 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:04:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45c1ffb700 1 -- 172.21.15.150:0/263470523 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55708086a780 con 0x5570808b1100 2024-05-10T04:04:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45c1ffb700 10 client.15744 handle_command_reply: tid=0 2024-05-10T04:04:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45f100eb80 1 client.15744 shutdown 2024-05-10T04:04:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45f100eb80 1 -- 172.21.15.150:0/263470523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089c570 msgr2=0x5570808a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45f100eb80 1 --2- 172.21.15.150:0/263470523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089c570 0x5570808a5910 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00ffa0 tx=0x7f45cc004750 comp rx=0 tx=0).stop 2024-05-10T04:04:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45f100eb80 1 -- 172.21.15.150:0/263470523 shutdown_connections 2024-05-10T04:04:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45f100eb80 1 --2- 172.21.15.150:0/263470523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089cec0 0x5570808ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45f100eb80 1 --2- 172.21.15.150:0/263470523 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x5570808b1100 0x5570808b3550 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45f100eb80 1 --2- 172.21.15.150:0/263470523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089b6a0 0x5570808a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45f100eb80 1 --2- 172.21.15.150:0/263470523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55708089c570 0x5570808a5910 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45c1ffb700 0 client.15744 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:04:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45f100eb80 1 -- 172.21.15.150:0/263470523 >> 172.21.15.150:0/263470523 conn(0x5570807fda60 msgr2=0x557080895fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45f100eb80 1 -- 172.21.15.150:0/263470523 shutdown_connections 2024-05-10T04:04:43.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.775+0000 7f45f100eb80 1 -- 172.21.15.150:0/263470523 wait complete. 2024-05-10T04:04:43.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.776+0000 7f45f100eb80 20 client.15744 trim_cache size 0 max 16384 2024-05-10T04:04:43.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45f100eb80 1 Processor -- start 2024-05-10T04:04:43.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45f100eb80 1 -- start start 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45f100eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 0x5570808aa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45f100eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089b6a0 0x5570808aad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45f100eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089c570 0x55708089bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45f100eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557080714270 con 0x5570807fda60 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45f100eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557080892d30 con 0x55708089b6a0 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45f100eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570808ad970 con 0x55708089c570 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45da59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 0x5570808aa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45da59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 0x5570808aa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38438/0 (socket says 172.21.15.150:38438) 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45da59c700 1 -- 172.21.15.150:0/1379290753 learned_addr learned my addr 172.21.15.150:0/1379290753 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45dad9d700 1 --2- 172.21.15.150:0/1379290753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089c570 0x55708089bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45d9d9b700 1 --2- 172.21.15.150:0/1379290753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089b6a0 0x5570808aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45da59c700 1 -- 172.21.15.150:0/1379290753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089c570 msgr2=0x55708089bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45da59c700 1 --2- 172.21.15.150:0/1379290753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089c570 0x55708089bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45da59c700 1 -- 172.21.15.150:0/1379290753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089b6a0 msgr2=0x5570808aad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45da59c700 1 --2- 172.21.15.150:0/1379290753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089b6a0 0x5570808aad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45da59c700 1 -- 172.21.15.150:0/1379290753 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557080781560 con 0x5570807fda60 2024-05-10T04:04:43.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.789+0000 7f45da59c700 1 --2- 172.21.15.150:0/1379290753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 0x5570808aa820 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f45d00cb8e0 tx=0x7f45d009aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f29043d7f0713e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.790+0000 7f45c97fa700 1 -- 172.21.15.150:0/1379290753 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d009f060 con 0x5570807fda60 2024-05-10T04:04:43.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.790+0000 7f45c97fa700 1 -- 172.21.15.150:0/1379290753 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45d00cba80 con 0x5570807fda60 2024-05-10T04:04:43.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.790+0000 7f45c97fa700 1 -- 172.21.15.150:0/1379290753 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d00a1310 con 0x5570807fda60 2024-05-10T04:04:43.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.790+0000 7f45f100eb80 1 -- 172.21.15.150:0/1379290753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 msgr2=0x5570808aa820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.790+0000 7f45f100eb80 1 --2- 172.21.15.150:0/1379290753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 0x5570808aa820 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f45d00cb8e0 tx=0x7f45d009aea0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.790+0000 7f45f100eb80 1 -- 172.21.15.150:0/1379290753 shutdown_connections 2024-05-10T04:04:43.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.790+0000 7f45f100eb80 1 --2- 172.21.15.150:0/1379290753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089c570 0x55708089bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.790+0000 7f45f100eb80 1 --2- 172.21.15.150:0/1379290753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 0x5570808aa820 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.790+0000 7f45f100eb80 1 --2- 172.21.15.150:0/1379290753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089b6a0 0x5570808aad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.790+0000 7f45f100eb80 1 -- 172.21.15.150:0/1379290753 >> 172.21.15.150:0/1379290753 conn(0x557080892450 msgr2=0x55708089f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.790+0000 7f45f100eb80 1 -- 172.21.15.150:0/1379290753 shutdown_connections 2024-05-10T04:04:43.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.790+0000 7f45f100eb80 1 -- 172.21.15.150:0/1379290753 wait complete. 2024-05-10T04:04:43.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45f100eb80 1 -- start start 2024-05-10T04:04:43.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45f100eb80 1 Processor -- start 2024-05-10T04:04:43.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45f100eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 0x5570808a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45f100eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089b6a0 0x5570808aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45f100eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089c570 0x557080905660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45f100eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557080905ba0 con 0x5570807fda60 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45f100eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557080905ce0 con 0x55708089c570 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45f100eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557080905ea0 con 0x55708089b6a0 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45f100eb80 10 client.0 _ll_register_callbacks cb 0x557080760b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45da59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 0x5570808a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45da59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 0x5570808a9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38448/0 (socket says 172.21.15.150:38448) 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45da59c700 1 -- 172.21.15.150:0/2668721915 learned_addr learned my addr 172.21.15.150:0/2668721915 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45d9d9b700 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089b6a0 0x5570808aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45dad9d700 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089c570 0x557080905660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45da59c700 1 -- 172.21.15.150:0/2668721915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089b6a0 msgr2=0x5570808aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45da59c700 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089b6a0 0x5570808aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45da59c700 1 -- 172.21.15.150:0/2668721915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089c570 msgr2=0x557080905660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45da59c700 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089c570 0x557080905660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45da59c700 1 -- 172.21.15.150:0/2668721915 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55708063c3b0 con 0x5570807fda60 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.791+0000 7f45dad9d700 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089c570 0x557080905660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45da59c700 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 0x5570808a9fe0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f45d0000c00 tx=0x7f45d0021bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5075b42b7d39b034 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45c1ffb700 1 -- 172.21.15.150:0/2668721915 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d00a3620 con 0x5570807fda60 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45c1ffb700 1 -- 172.21.15.150:0/2668721915 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f45d0095e40 con 0x5570807fda60 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45f100eb80 1 -- 172.21.15.150:0/2668721915 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45d009b930 con 0x5570807fda60 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45c1ffb700 1 -- 172.21.15.150:0/2668721915 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45d009f060 con 0x5570807fda60 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45f100eb80 10 client.15750 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45f100eb80 1 -- 172.21.15.150:0/2668721915 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557080640cd0 con 0x5570807fda60 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45c1ffb700 1 -- 172.21.15.150:0/2668721915 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45d009b930 con 0x5570807fda60 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45c1ffb700 1 -- 172.21.15.150:0/2668721915 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f45d00b5d90 con 0x5570807fda60 2024-05-10T04:04:43.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45f100eb80 10 client.15750 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45f100eb80 10 client.15750 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45f100eb80 20 client.15750 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45f100eb80 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x55708089cec0 0x557080909310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45f100eb80 4 client.15750 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.792+0000 7f45f100eb80 1 -- 172.21.15.150:0/2668721915 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5570807b3ec0 con 0x55708089cec0 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45d9d9b700 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x55708089cec0 0x557080909310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45c1ffb700 10 client.15750 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45d9d9b700 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x55708089cec0 0x557080909310 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=334f036843962da3 server_cookie=d3cada86067bb70 in_seq=0 out_seq=0 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45c1ffb700 1 -- 172.21.15.150:0/2668721915 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5570807b3ec0 con 0x55708089cec0 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45c1ffb700 10 client.15750 handle_command_reply: tid=0 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45f100eb80 1 client.15750 shutdown 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45f100eb80 1 -- 172.21.15.150:0/2668721915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 msgr2=0x5570808a9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45f100eb80 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 0x5570808a9fe0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f45d0000c00 tx=0x7f45d0021bf0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45f100eb80 1 -- 172.21.15.150:0/2668721915 shutdown_connections 2024-05-10T04:04:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45f100eb80 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x55708089cec0 0x557080909310 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45f100eb80 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55708089c570 0x557080905660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45f100eb80 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5570807fda60 0x5570808a9fe0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45f100eb80 1 --2- 172.21.15.150:0/2668721915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55708089b6a0 0x5570808aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45c1ffb700 0 client.15750 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:04:43.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45f100eb80 1 -- 172.21.15.150:0/2668721915 >> 172.21.15.150:0/2668721915 conn(0x557080892450 msgr2=0x55708089f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45f100eb80 1 -- 172.21.15.150:0/2668721915 shutdown_connections 2024-05-10T04:04:43.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45f100eb80 1 -- 172.21.15.150:0/2668721915 wait complete. 2024-05-10T04:04:43.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.793+0000 7f45f100eb80 20 client.15750 trim_cache size 0 max 16384 2024-05-10T04:04:43.806 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:04:43.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.794+0000 7f45e0cca700 1 -- 172.21.15.150:0/3021981031 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f45c40a12e0 msgr2=0x7f45c40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.794+0000 7f45e0cca700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f45c40a12e0 0x7f45c40a37f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f45cc005950 tx=0x7f45cc0058e0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.794+0000 7f45e0cca700 1 -- 172.21.15.150:0/3021981031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45dc0ffbe0 msgr2=0x7f45dc197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.794+0000 7f45e0cca700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45dc0ffbe0 0x7f45dc197eb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f45d00098e0 tx=0x7f45d0009580 comp rx=0 tx=0).stop 2024-05-10T04:04:43.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.794+0000 7f45e0cca700 1 -- 172.21.15.150:0/3021981031 shutdown_connections 2024-05-10T04:04:43.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.794+0000 7f45e0cca700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f45c40a12e0 0x7f45c40a37f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.794+0000 7f45e0cca700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45dc1031a0 0x7f45dc19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.794+0000 7f45e0cca700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45dc0ffbe0 0x7f45dc197eb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.794+0000 7f45e0cca700 1 --2- 172.21.15.150:0/3021981031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45dc100530 0x7f45dc1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.794+0000 7f45e0cca700 1 -- 172.21.15.150:0/3021981031 >> 172.21.15.150:0/3021981031 conn(0x7f45dc0fb8b0 msgr2=0x7f45dc106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.794+0000 7f45e0cca700 1 -- 172.21.15.150:0/3021981031 shutdown_connections 2024-05-10T04:04:43.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.794+0000 7f45e0cca700 1 -- 172.21.15.150:0/3021981031 wait complete. 2024-05-10T04:04:43.809 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T04:04:43.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.941+0000 7fa1d6d59700 1 -- 172.21.15.150:0/1915705881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d0102580 msgr2=0x7fa1d0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.941+0000 7fa1d6d59700 1 --2- 172.21.15.150:0/1915705881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d0102580 0x7fa1d0108e30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800b0a0 tx=0x7fa1c800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.941+0000 7fa1d6d59700 1 -- 172.21.15.150:0/1915705881 shutdown_connections 2024-05-10T04:04:43.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.941+0000 7fa1d6d59700 1 --2- 172.21.15.150:0/1915705881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d0102580 0x7fa1d0108e30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.941+0000 7fa1d6d59700 1 --2- 172.21.15.150:0/1915705881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0100ac0 0x7fa1d0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.941+0000 7fa1d6d59700 1 --2- 172.21.15.150:0/1915705881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1d0101be0 0x7fa1d0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.941+0000 7fa1d6d59700 1 -- 172.21.15.150:0/1915705881 >> 172.21.15.150:0/1915705881 conn(0x7fa1d00fc0d0 msgr2=0x7fa1d00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.941+0000 7fa1d6d59700 1 -- 172.21.15.150:0/1915705881 shutdown_connections 2024-05-10T04:04:43.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.942+0000 7fa1d6d59700 1 -- 172.21.15.150:0/1915705881 wait complete. 2024-05-10T04:04:43.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.942+0000 7fa1d6d59700 1 Processor -- start 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.942+0000 7fa1d6d59700 1 -- start start 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.942+0000 7fa1d6d59700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1d0100ac0 0x7fa1d0196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.942+0000 7fa1d6d59700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0101be0 0x7fa1d0197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.942+0000 7fa1d6d59700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d0102580 0x7fa1d019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.942+0000 7fa1d6d59700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d019ea00 con 0x7fa1d0102580 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.942+0000 7fa1d6d59700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d019eb70 con 0x7fa1d0100ac0 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.942+0000 7fa1d6d59700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d019ed30 con 0x7fa1d0101be0 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.942+0000 7fa1d52f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d0102580 0x7fa1d019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.942+0000 7fa1d52f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d0102580 0x7fa1d019e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38462/0 (socket says 172.21.15.150:38462) 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.942+0000 7fa1d52f6700 1 -- 172.21.15.150:0/332189371 learned_addr learned my addr 172.21.15.150:0/332189371 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.943+0000 7fa1cffff700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0101be0 0x7fa1d0197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.943+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1d0100ac0 0x7fa1d0196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.943+0000 7fa1d52f6700 1 -- 172.21.15.150:0/332189371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0101be0 msgr2=0x7fa1d0197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.943+0000 7fa1d52f6700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0101be0 0x7fa1d0197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.943+0000 7fa1d52f6700 1 -- 172.21.15.150:0/332189371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1d0100ac0 msgr2=0x7fa1d0196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.943+0000 7fa1d52f6700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1d0100ac0 0x7fa1d0196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.943+0000 7fa1d52f6700 1 -- 172.21.15.150:0/332189371 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c8009d00 con 0x7fa1d0102580 2024-05-10T04:04:43.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.943+0000 7fa1d52f6700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d0102580 0x7fa1d019e3f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800b070 tx=0x7fa1c80078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.943+0000 7fa1cdffb700 1 -- 172.21.15.150:0/332189371 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c800e040 con 0x7fa1d0102580 2024-05-10T04:04:43.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.943+0000 7fa1cdffb700 1 -- 172.21.15.150:0/332189371 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1c801d470 con 0x7fa1d0102580 2024-05-10T04:04:43.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.943+0000 7fa1d6d59700 1 -- 172.21.15.150:0/332189371 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d0068480 con 0x7fa1d0102580 2024-05-10T04:04:43.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.943+0000 7fa1cdffb700 1 -- 172.21.15.150:0/332189371 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80052a0 con 0x7fa1d0102580 2024-05-10T04:04:43.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.943+0000 7fa1d6d59700 1 -- 172.21.15.150:0/332189371 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d0068990 con 0x7fa1d0102580 2024-05-10T04:04:43.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.945+0000 7fa1cdffb700 1 -- 172.21.15.150:0/332189371 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa1c801d5e0 con 0x7fa1d0102580 2024-05-10T04:04:43.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.946+0000 7fa1cdffb700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa1c00a1260 0x7fa1c00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.946+0000 7fa1cdffb700 1 -- 172.21.15.150:0/332189371 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa1c80ce0a0 con 0x7fa1d0102580 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.946+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa1c00a1260 0x7fa1c00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa1c00a1260 0x7fa1c00a3770 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0101ae0 tx=0x7fa1c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1e709db80 1 Processor -- start 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1e709db80 1 -- start start 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1e709db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 0x55b27ca5daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1e709db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5e570 0x55b27ca5e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1e709db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5eec0 0x55b27ca657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1e709db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27c984150 con 0x55b27ca5d6a0 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1e709db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27c981b10 con 0x55b27ca5eec0 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1e709db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27c97d400 con 0x55b27ca5e570 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1d4af5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 0x55b27ca5daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1d4af5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 0x55b27ca5daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38480/0 (socket says 172.21.15.150:38480) 2024-05-10T04:04:43.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1d4af5700 1 -- 172.21.15.150:0/422975418 learned_addr learned my addr 172.21.15.150:0/422975418 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1cffff700 1 --2- 172.21.15.150:0/422975418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5e570 0x55b27ca5e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1d52f6700 1 --2- 172.21.15.150:0/422975418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5eec0 0x55b27ca657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1d4af5700 1 -- 172.21.15.150:0/422975418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5e570 msgr2=0x55b27ca5e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/422975418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5e570 0x55b27ca5e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1d4af5700 1 -- 172.21.15.150:0/422975418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5eec0 msgr2=0x55b27ca657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/422975418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5eec0 0x55b27ca657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.947+0000 7fa1d4af5700 1 -- 172.21.15.150:0/422975418 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b27c7fe3b0 con 0x55b27ca5d6a0 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.948+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/422975418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 0x55b27ca5daf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4011d20 tx=0x7fa1c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65706f05db51987 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.948+0000 7fa1baffd700 1 -- 172.21.15.150:0/422975418 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c400ed20 con 0x55b27ca5d6a0 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.948+0000 7fa1baffd700 1 -- 172.21.15.150:0/422975418 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1c4021070 con 0x55b27ca5d6a0 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.948+0000 7fa1baffd700 1 -- 172.21.15.150:0/422975418 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c401d670 con 0x55b27ca5d6a0 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.949+0000 7fa1e709db80 1 -- 172.21.15.150:0/422975418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 msgr2=0x55b27ca5daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.949+0000 7fa1e709db80 1 --2- 172.21.15.150:0/422975418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 0x55b27ca5daf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4011d20 tx=0x7fa1c4014000 comp rx=0 tx=0).stop 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.949+0000 7fa1e709db80 1 -- 172.21.15.150:0/422975418 shutdown_connections 2024-05-10T04:04:43.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.949+0000 7fa1e709db80 1 --2- 172.21.15.150:0/422975418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5eec0 0x55b27ca657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.949+0000 7fa1e709db80 1 --2- 172.21.15.150:0/422975418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 0x55b27ca5daf0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.949+0000 7fa1e709db80 1 --2- 172.21.15.150:0/422975418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5e570 0x55b27ca5e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.949+0000 7fa1e709db80 1 -- 172.21.15.150:0/422975418 >> 172.21.15.150:0/422975418 conn(0x55b27c9bfa60 msgr2=0x55b27ca57fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.949+0000 7fa1e709db80 1 -- 172.21.15.150:0/422975418 shutdown_connections 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.949+0000 7fa1e709db80 1 -- 172.21.15.150:0/422975418 wait complete. 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.949+0000 7fa1e709db80 1 -- start start 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.949+0000 7fa1e709db80 1 Processor -- start 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1e709db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5d6a0 0x55b27ca676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1e709db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5e570 0x55b27ca67be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1e709db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 0x55b27ca6eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1e709db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27c919560 con 0x55b27ca5eec0 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1e709db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27c919200 con 0x55b27ca5d6a0 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1e709db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27c8006a0 con 0x55b27ca5e570 2024-05-10T04:04:43.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1e709db80 10 client.0 _ll_register_callbacks cb 0x55b27c922b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1d52f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 0x55b27ca6eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1d52f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 0x55b27ca6eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38492/0 (socket says 172.21.15.150:38492) 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1d52f6700 1 -- 172.21.15.150:0/2669029119 learned_addr learned my addr 172.21.15.150:0/2669029119 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/2669029119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5d6a0 0x55b27ca676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1cffff700 1 --2- 172.21.15.150:0/2669029119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5e570 0x55b27ca67be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1d52f6700 1 -- 172.21.15.150:0/2669029119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5e570 msgr2=0x55b27ca67be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1d52f6700 1 --2- 172.21.15.150:0/2669029119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5e570 0x55b27ca67be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1d52f6700 1 -- 172.21.15.150:0/2669029119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5d6a0 msgr2=0x55b27ca676a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1d52f6700 1 --2- 172.21.15.150:0/2669029119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5d6a0 0x55b27ca676a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1d52f6700 1 -- 172.21.15.150:0/2669029119 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c4017070 con 0x55b27ca5eec0 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1d52f6700 1 --2- 172.21.15.150:0/2669029119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 0x55b27ca6eb40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa1c809ae10 tx=0x7fa1c809acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21e60e1085f71e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1affff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1affff700 1 -- 172.21.15.150:0/2669029119 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c809b110 con 0x55b27ca5eec0 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1affff700 1 -- 172.21.15.150:0/2669029119 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa1c809ef40 con 0x55b27ca5eec0 2024-05-10T04:04:43.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1e709db80 1 -- 172.21.15.150:0/2669029119 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b27c8d6270 con 0x55b27ca5eec0 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1affff700 1 -- 172.21.15.150:0/2669029119 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1c80a3500 con 0x55b27ca5eec0 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1e709db80 10 client.15762 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.950+0000 7fa1e709db80 1 -- 172.21.15.150:0/2669029119 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b27c7aab60 con 0x55b27ca5eec0 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.951+0000 7fa1affff700 1 -- 172.21.15.150:0/2669029119 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b27c8d6270 con 0x55b27ca5eec0 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.951+0000 7fa1affff700 1 -- 172.21.15.150:0/2669029119 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa1c809b370 con 0x55b27ca5eec0 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.951+0000 7fa1e709db80 10 client.15762 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.951+0000 7fa1e709db80 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.951+0000 7fa1e709db80 20 client.15762 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.951+0000 7fa1e709db80 1 --2- 172.21.15.150:0/2669029119 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55b27ca73080 0x55b27ca754d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.951+0000 7fa1e709db80 4 client.15762 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.951+0000 7fa1e709db80 1 -- 172.21.15.150:0/2669029119 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b27ca2c780 con 0x55b27ca73080 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.951+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/2669029119 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55b27ca73080 0x55b27ca754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.952+0000 7fa1affff700 10 client.15762 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:43.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.952+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/2669029119 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55b27ca73080 0x55b27ca754d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ec994ca6d9d433 server_cookie=9d86ce55048f90b2 in_seq=0 out_seq=0 2024-05-10T04:04:43.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.952+0000 7fa1affff700 1 -- 172.21.15.150:0/2669029119 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b27ca2c780 con 0x55b27ca73080 2024-05-10T04:04:43.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.952+0000 7fa1affff700 10 client.15762 handle_command_reply: tid=0 2024-05-10T04:04:43.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.952+0000 7fa1e709db80 1 client.15762 shutdown 2024-05-10T04:04:43.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.953+0000 7fa1e709db80 1 -- 172.21.15.150:0/2669029119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 msgr2=0x55b27ca6eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.953+0000 7fa1e709db80 1 --2- 172.21.15.150:0/2669029119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 0x55b27ca6eb40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa1c809ae10 tx=0x7fa1c809acd0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.953+0000 7fa1e709db80 1 -- 172.21.15.150:0/2669029119 shutdown_connections 2024-05-10T04:04:43.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.953+0000 7fa1e709db80 1 --2- 172.21.15.150:0/2669029119 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55b27ca73080 0x55b27ca754d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.953+0000 7fa1e709db80 1 --2- 172.21.15.150:0/2669029119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 0x55b27ca6eb40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.953+0000 7fa1e709db80 1 --2- 172.21.15.150:0/2669029119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5d6a0 0x55b27ca676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.953+0000 7fa1affff700 0 client.15762 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:43.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.953+0000 7fa1e709db80 1 --2- 172.21.15.150:0/2669029119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5e570 0x55b27ca67be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.953+0000 7fa1e709db80 1 -- 172.21.15.150:0/2669029119 >> 172.21.15.150:0/2669029119 conn(0x55b27c9bfa60 msgr2=0x55b27ca57fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.953+0000 7fa1e709db80 1 -- 172.21.15.150:0/2669029119 shutdown_connections 2024-05-10T04:04:43.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.953+0000 7fa1e709db80 1 -- 172.21.15.150:0/2669029119 wait complete. 2024-05-10T04:04:43.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.953+0000 7fa1e709db80 20 client.15762 trim_cache size 0 max 16384 2024-05-10T04:04:43.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.966+0000 7fa1e709db80 1 Processor -- start 2024-05-10T04:04:43.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.966+0000 7fa1e709db80 1 -- start start 2024-05-10T04:04:43.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.966+0000 7fa1e709db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 0x55b27ca5dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.966+0000 7fa1e709db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5e570 0x55b27ca6eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.966+0000 7fa1e709db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5eec0 0x55b27ca67660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.966+0000 7fa1e709db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27ca68770 con 0x55b27ca5d6a0 2024-05-10T04:04:43.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.966+0000 7fa1e709db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27ca6fc40 con 0x55b27ca5e570 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.966+0000 7fa1e709db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27ca597a0 con 0x55b27ca5eec0 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.966+0000 7fa1d4af5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 0x55b27ca5dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.966+0000 7fa1d4af5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 0x55b27ca5dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38506/0 (socket says 172.21.15.150:38506) 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.966+0000 7fa1d4af5700 1 -- 172.21.15.150:0/2454798695 learned_addr learned my addr 172.21.15.150:0/2454798695 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.966+0000 7fa1cffff700 1 --2- 172.21.15.150:0/2454798695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5e570 0x55b27ca6eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.967+0000 7fa1d52f6700 1 --2- 172.21.15.150:0/2454798695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5eec0 0x55b27ca67660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.967+0000 7fa1d4af5700 1 -- 172.21.15.150:0/2454798695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5eec0 msgr2=0x55b27ca67660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.967+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/2454798695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5eec0 0x55b27ca67660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.967+0000 7fa1d4af5700 1 -- 172.21.15.150:0/2454798695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5e570 msgr2=0x55b27ca6eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.967+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/2454798695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5e570 0x55b27ca6eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.967+0000 7fa1d4af5700 1 -- 172.21.15.150:0/2454798695 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b27c943560 con 0x55b27ca5d6a0 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.967+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/2454798695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 0x55b27ca5dd00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55b27ca5d380 tx=0x7fa1c400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614eb2b76634f9f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.967+0000 7fa1bb7fe700 1 -- 172.21.15.150:0/2454798695 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c401cdb0 con 0x55b27ca5d6a0 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.967+0000 7fa1bb7fe700 1 -- 172.21.15.150:0/2454798695 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1c400ed80 con 0x55b27ca5d6a0 2024-05-10T04:04:43.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.967+0000 7fa1bb7fe700 1 -- 172.21.15.150:0/2454798695 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4005580 con 0x55b27ca5d6a0 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.968+0000 7fa1e709db80 1 -- 172.21.15.150:0/2454798695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 msgr2=0x55b27ca5dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.968+0000 7fa1e709db80 1 --2- 172.21.15.150:0/2454798695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 0x55b27ca5dd00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55b27ca5d380 tx=0x7fa1c400ea20 comp rx=0 tx=0).stop 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.968+0000 7fa1e709db80 1 -- 172.21.15.150:0/2454798695 shutdown_connections 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.968+0000 7fa1e709db80 1 --2- 172.21.15.150:0/2454798695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5eec0 0x55b27ca67660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.968+0000 7fa1e709db80 1 --2- 172.21.15.150:0/2454798695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5d6a0 0x55b27ca5dd00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.968+0000 7fa1e709db80 1 --2- 172.21.15.150:0/2454798695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5e570 0x55b27ca6eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.968+0000 7fa1e709db80 1 -- 172.21.15.150:0/2454798695 >> 172.21.15.150:0/2454798695 conn(0x55b27c9bfa60 msgr2=0x55b27ca54180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.968+0000 7fa1e709db80 1 -- 172.21.15.150:0/2454798695 shutdown_connections 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.968+0000 7fa1e709db80 1 -- 172.21.15.150:0/2454798695 wait complete. 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.968+0000 7fa1e709db80 1 -- start start 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.968+0000 7fa1e709db80 1 Processor -- start 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1e709db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5d6a0 0x55b27ca6c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1e709db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5e570 0x55b27ca6cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1e709db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 0x55b27ca6dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1e709db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27ca6e1f0 con 0x55b27ca5eec0 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1e709db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27ca6e360 con 0x55b27ca5e570 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1e709db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27ca6e520 con 0x55b27ca5d6a0 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1e709db80 10 client.0 _ll_register_callbacks cb 0x55b27c922b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1d52f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 0x55b27ca6dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1d4af5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5d6a0 0x55b27ca6c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1d52f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 0x55b27ca6dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38512/0 (socket says 172.21.15.150:38512) 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1d52f6700 1 -- 172.21.15.150:0/3275647592 learned_addr learned my addr 172.21.15.150:0/3275647592 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1cffff700 1 --2- 172.21.15.150:0/3275647592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5e570 0x55b27ca6cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1d52f6700 1 -- 172.21.15.150:0/3275647592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5d6a0 msgr2=0x55b27ca6c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1d52f6700 1 --2- 172.21.15.150:0/3275647592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5d6a0 0x55b27ca6c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1d52f6700 1 -- 172.21.15.150:0/3275647592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5e570 msgr2=0x55b27ca6cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1d52f6700 1 --2- 172.21.15.150:0/3275647592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5e570 0x55b27ca6cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1d52f6700 1 -- 172.21.15.150:0/3275647592 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c402a040 con 0x55b27ca5eec0 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1cffff700 1 --2- 172.21.15.150:0/3275647592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5e570 0x55b27ca6cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1d52f6700 1 --2- 172.21.15.150:0/3275647592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 0x55b27ca6dcb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa1c809b6a0 tx=0x7fa1c80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b17445b5d1676 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1affff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1affff700 1 -- 172.21.15.150:0/3275647592 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c809aa70 con 0x55b27ca5eec0 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1affff700 1 -- 172.21.15.150:0/3275647592 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa1c809ef40 con 0x55b27ca5eec0 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.969+0000 7fa1e709db80 1 -- 172.21.15.150:0/3275647592 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1c4021070 con 0x55b27ca5eec0 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.970+0000 7fa1affff700 1 -- 172.21.15.150:0/3275647592 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1c80b9450 con 0x55b27ca5eec0 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.970+0000 7fa1e709db80 10 client.15768 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.970+0000 7fa1e709db80 1 -- 172.21.15.150:0/3275647592 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b27c802cd0 con 0x55b27ca5eec0 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.970+0000 7fa1affff700 1 -- 172.21.15.150:0/3275647592 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1c4021070 con 0x55b27ca5eec0 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.970+0000 7fa1affff700 1 -- 172.21.15.150:0/3275647592 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa1c80a3070 con 0x55b27ca5eec0 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.970+0000 7fa1e709db80 10 client.15768 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.970+0000 7fa1e709db80 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.970+0000 7fa1e709db80 20 client.15768 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:43.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.970+0000 7fa1e709db80 1 --2- 172.21.15.150:0/3275647592 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55b27cac9e30 0x55b27cacc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.970+0000 7fa1e709db80 4 client.15768 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.970+0000 7fa1e709db80 1 -- 172.21.15.150:0/3275647592 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b27c975ec0 con 0x55b27cac9e30 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.970+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/3275647592 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55b27cac9e30 0x55b27cacc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1affff700 10 client.15768 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1d4af5700 1 --2- 172.21.15.150:0/3275647592 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55b27cac9e30 0x55b27cacc250 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c08209b83317fe server_cookie=c2330534be6cb65d in_seq=0 out_seq=0 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1affff700 1 -- 172.21.15.150:0/3275647592 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b27c975ec0 con 0x55b27cac9e30 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1affff700 10 client.15768 handle_command_reply: tid=0 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1e709db80 1 client.15768 shutdown 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1e709db80 1 -- 172.21.15.150:0/3275647592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 msgr2=0x55b27ca6dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1e709db80 1 --2- 172.21.15.150:0/3275647592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 0x55b27ca6dcb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa1c809b6a0 tx=0x7fa1c80099e0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1e709db80 1 -- 172.21.15.150:0/3275647592 shutdown_connections 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1e709db80 1 --2- 172.21.15.150:0/3275647592 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55b27cac9e30 0x55b27cacc250 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1e709db80 1 --2- 172.21.15.150:0/3275647592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b27ca5eec0 0x55b27ca6dcb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1e709db80 1 --2- 172.21.15.150:0/3275647592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b27ca5d6a0 0x55b27ca6c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1affff700 0 client.15768 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:43.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1e709db80 1 --2- 172.21.15.150:0/3275647592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b27ca5e570 0x55b27ca6cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1e709db80 1 -- 172.21.15.150:0/3275647592 >> 172.21.15.150:0/3275647592 conn(0x55b27c9bfa60 msgr2=0x55b27ca54180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1e709db80 1 -- 172.21.15.150:0/3275647592 shutdown_connections 2024-05-10T04:04:43.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1e709db80 1 -- 172.21.15.150:0/3275647592 wait complete. 2024-05-10T04:04:43.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.971+0000 7fa1e709db80 20 client.15768 trim_cache size 0 max 16384 2024-05-10T04:04:43.980 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:04:43.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.972+0000 7fa1d6d59700 1 -- 172.21.15.150:0/332189371 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa1c00a1260 msgr2=0x7fa1c00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.972+0000 7fa1d6d59700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa1c00a1260 0x7fa1c00a3770 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0101ae0 tx=0x7fa1c4009260 comp rx=0 tx=0).stop 2024-05-10T04:04:43.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.972+0000 7fa1d6d59700 1 -- 172.21.15.150:0/332189371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d0102580 msgr2=0x7fa1d019e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:43.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.972+0000 7fa1d6d59700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d0102580 0x7fa1d019e3f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800b070 tx=0x7fa1c80078a0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.972+0000 7fa1d6d59700 1 -- 172.21.15.150:0/332189371 shutdown_connections 2024-05-10T04:04:43.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.972+0000 7fa1d6d59700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fa1c00a1260 0x7fa1c00a3770 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.972+0000 7fa1d6d59700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d0102580 0x7fa1d019e3f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.972+0000 7fa1d6d59700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1d0100ac0 0x7fa1d0196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.972+0000 7fa1d6d59700 1 --2- 172.21.15.150:0/332189371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0101be0 0x7fa1d0197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:43.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.972+0000 7fa1d6d59700 1 -- 172.21.15.150:0/332189371 >> 172.21.15.150:0/332189371 conn(0x7fa1d00fc0d0 msgr2=0x7fa1d0104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:43.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.972+0000 7fa1d6d59700 1 -- 172.21.15.150:0/332189371 shutdown_connections 2024-05-10T04:04:43.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:43.972+0000 7fa1d6d59700 1 -- 172.21.15.150:0/332189371 wait complete. 2024-05-10T04:04:44.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:44 smithi150 ceph-mon[42477]: pgmap v294: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T04:04:44.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:44 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2788082056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:04:44.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:44 smithi181 ceph-mon[46602]: pgmap v294: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T04:04:44.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:44 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2788082056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:04:44.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:44 smithi161 ceph-mon[46918]: pgmap v294: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T04:04:44.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:44 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2788082056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:04:44.987 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T04:04:44.987 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T04:04:45.141 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.137+0000 7f1043779700 1 -- 172.21.15.150:0/609603920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f103c100ac0 msgr2=0x7f103c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.141 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.137+0000 7f1043779700 1 --2- 172.21.15.150:0/609603920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f103c100ac0 0x7f103c100ed0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1038009990 tx=0x7f1038009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.137+0000 7f1043779700 1 -- 172.21.15.150:0/609603920 shutdown_connections 2024-05-10T04:04:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.137+0000 7f1043779700 1 --2- 172.21.15.150:0/609603920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f103c102580 0x7f103c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.137+0000 7f1043779700 1 --2- 172.21.15.150:0/609603920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f103c100ac0 0x7f103c100ed0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.137+0000 7f1043779700 1 --2- 172.21.15.150:0/609603920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f103c101be0 0x7f103c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.137+0000 7f1043779700 1 -- 172.21.15.150:0/609603920 >> 172.21.15.150:0/609603920 conn(0x7f103c0fc0d0 msgr2=0x7f103c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.137+0000 7f1043779700 1 -- 172.21.15.150:0/609603920 shutdown_connections 2024-05-10T04:04:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.137+0000 7f1043779700 1 -- 172.21.15.150:0/609603920 wait complete. 2024-05-10T04:04:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.137+0000 7f1043779700 1 Processor -- start 2024-05-10T04:04:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1043779700 1 -- start start 2024-05-10T04:04:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1043779700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f103c100ac0 0x7f103c068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1043779700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f103c101be0 0x7f103c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1043779700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f103c102580 0x7f103c19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1043779700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c19fa00 con 0x7f103c100ac0 2024-05-10T04:04:45.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1043779700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c19fb70 con 0x7f103c101be0 2024-05-10T04:04:45.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1043779700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c19fd30 con 0x7f103c102580 2024-05-10T04:04:45.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1041515700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f103c100ac0 0x7f103c068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1041515700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f103c100ac0 0x7f103c068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38522/0 (socket says 172.21.15.150:38522) 2024-05-10T04:04:45.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1041515700 1 -- 172.21.15.150:0/3875092916 learned_addr learned my addr 172.21.15.150:0/3875092916 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:45.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1040d14700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f103c101be0 0x7f103c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1041d16700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f103c102580 0x7f103c19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1041515700 1 -- 172.21.15.150:0/3875092916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f103c102580 msgr2=0x7f103c19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1041515700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f103c102580 0x7f103c19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1041515700 1 -- 172.21.15.150:0/3875092916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f103c101be0 msgr2=0x7f103c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1041515700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f103c101be0 0x7f103c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1041515700 1 -- 172.21.15.150:0/3875092916 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10380095f0 con 0x7f103c100ac0 2024-05-10T04:04:45.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1041d16700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f103c102580 0x7f103c19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:45.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f1041515700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f103c100ac0 0x7f103c068e50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1038006010 tx=0x7f103800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:45.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.138+0000 7f10327fc700 1 -- 172.21.15.150:0/3875092916 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103801d070 con 0x7f103c100ac0 2024-05-10T04:04:45.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.139+0000 7f10327fc700 1 -- 172.21.15.150:0/3875092916 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1038003960 con 0x7f103c100ac0 2024-05-10T04:04:45.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.139+0000 7f1043779700 1 -- 172.21.15.150:0/3875092916 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103c19ff50 con 0x7f103c100ac0 2024-05-10T04:04:45.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.139+0000 7f10327fc700 1 -- 172.21.15.150:0/3875092916 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10380054f0 con 0x7f103c100ac0 2024-05-10T04:04:45.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.139+0000 7f1043779700 1 -- 172.21.15.150:0/3875092916 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103c1a0310 con 0x7f103c100ac0 2024-05-10T04:04:45.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.141+0000 7f10327fc700 1 -- 172.21.15.150:0/3875092916 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f103802b440 con 0x7f103c100ac0 2024-05-10T04:04:45.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.141+0000 7f10327fc700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f10280a12e0 0x7f10280a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.142+0000 7f10327fc700 1 -- 172.21.15.150:0/3875092916 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f10380cdf60 con 0x7f103c100ac0 2024-05-10T04:04:45.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.142+0000 7f1040d14700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f10280a12e0 0x7f10280a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.142+0000 7f1040d14700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f10280a12e0 0x7f10280a37f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f103c0687f0 tx=0x7f102c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:45.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.142+0000 7f1053abdb80 1 Processor -- start 2024-05-10T04:04:45.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.142+0000 7f1053abdb80 1 -- start start 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1053abdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 0x561bd15f9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1053abdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 0x561bd15faa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1053abdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15faf40 0x561bd1601840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1053abdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd15201d0 con 0x561bd15fa5f0 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1053abdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd151db90 con 0x561bd15faf40 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1053abdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd1519480 con 0x561bd15f9720 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1040d14700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 0x561bd15faa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1040d14700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 0x561bd15faa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38536/0 (socket says 172.21.15.150:38536) 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1040d14700 1 -- 172.21.15.150:0/1142889044 learned_addr learned my addr 172.21.15.150:0/1142889044 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1041515700 1 --2- 172.21.15.150:0/1142889044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 0x561bd15f9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1041d16700 1 --2- 172.21.15.150:0/1142889044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15faf40 0x561bd1601840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1040d14700 1 -- 172.21.15.150:0/1142889044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 msgr2=0x561bd15f9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1040d14700 1 --2- 172.21.15.150:0/1142889044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 0x561bd15f9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1040d14700 1 -- 172.21.15.150:0/1142889044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15faf40 msgr2=0x561bd1601840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1040d14700 1 --2- 172.21.15.150:0/1142889044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15faf40 0x561bd1601840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1040d14700 1 -- 172.21.15.150:0/1142889044 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bd139a430 con 0x561bd15fa5f0 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.143+0000 7f1040d14700 1 --2- 172.21.15.150:0/1142889044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 0x561bd15faa00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f102c010f70 tx=0x7f102c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e82ae0c311045b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.144+0000 7f10277fe700 1 -- 172.21.15.150:0/1142889044 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c011db0 con 0x561bd15fa5f0 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.144+0000 7f10277fe700 1 -- 172.21.15.150:0/1142889044 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f102c015070 con 0x561bd15fa5f0 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.144+0000 7f10277fe700 1 -- 172.21.15.150:0/1142889044 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c01c5d0 con 0x561bd15fa5f0 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.144+0000 7f1053abdb80 1 -- 172.21.15.150:0/1142889044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 msgr2=0x561bd15faa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.144+0000 7f1053abdb80 1 --2- 172.21.15.150:0/1142889044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 0x561bd15faa00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f102c010f70 tx=0x7f102c013000 comp rx=0 tx=0).stop 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 -- 172.21.15.150:0/1142889044 shutdown_connections 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 --2- 172.21.15.150:0/1142889044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15faf40 0x561bd1601840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 --2- 172.21.15.150:0/1142889044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 0x561bd15f9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 --2- 172.21.15.150:0/1142889044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 0x561bd15faa00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 -- 172.21.15.150:0/1142889044 >> 172.21.15.150:0/1142889044 conn(0x561bd155bae0 msgr2=0x561bd15f4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 -- 172.21.15.150:0/1142889044 shutdown_connections 2024-05-10T04:04:45.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 -- 172.21.15.150:0/1142889044 wait complete. 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 -- start start 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 Processor -- start 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 0x561bd16036e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 0x561bd1603c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15faf40 0x561bd160ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd14b55e0 con 0x561bd15fa5f0 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd14b5280 con 0x561bd15faf40 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.145+0000 7f1053abdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd139c720 con 0x561bd15f9720 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1053abdb80 10 client.0 _ll_register_callbacks cb 0x561bd14bebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1040d14700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 0x561bd1603c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1040d14700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 0x561bd1603c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38538/0 (socket says 172.21.15.150:38538) 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1040d14700 1 -- 172.21.15.150:0/3798340849 learned_addr learned my addr 172.21.15.150:0/3798340849 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1041515700 1 --2- 172.21.15.150:0/3798340849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 0x561bd16036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1041d16700 1 --2- 172.21.15.150:0/3798340849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15faf40 0x561bd160ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1040d14700 1 -- 172.21.15.150:0/3798340849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 msgr2=0x561bd16036e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1040d14700 1 --2- 172.21.15.150:0/3798340849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 0x561bd16036e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1040d14700 1 -- 172.21.15.150:0/3798340849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15faf40 msgr2=0x561bd160ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1040d14700 1 --2- 172.21.15.150:0/3798340849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15faf40 0x561bd160ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1040d14700 1 -- 172.21.15.150:0/3798340849 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102c010c20 con 0x561bd15fa5f0 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1040d14700 1 --2- 172.21.15.150:0/3798340849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 0x561bd1603c20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f102c011e80 tx=0x7f102c01e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae0656b18c5ffb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1024ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1024ff9700 1 -- 172.21.15.150:0/3798340849 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c02c070 con 0x561bd15fa5f0 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1024ff9700 1 -- 172.21.15.150:0/3798340849 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f102c017070 con 0x561bd15fa5f0 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1053abdb80 1 -- 172.21.15.150:0/3798340849 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bd14722f0 con 0x561bd15fa5f0 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1024ff9700 1 -- 172.21.15.150:0/3798340849 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f102c01e940 con 0x561bd15fa5f0 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1053abdb80 10 client.15780 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.146+0000 7f1053abdb80 1 -- 172.21.15.150:0/3798340849 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561bd1346be0 con 0x561bd15fa5f0 2024-05-10T04:04:45.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.147+0000 7f1024ff9700 1 -- 172.21.15.150:0/3798340849 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561bd14722f0 con 0x561bd15fa5f0 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.147+0000 7f1024ff9700 1 -- 172.21.15.150:0/3798340849 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f102c038070 con 0x561bd15fa5f0 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.147+0000 7f1053abdb80 10 client.15780 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.147+0000 7f1053abdb80 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.147+0000 7f1053abdb80 20 client.15780 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.147+0000 7f1053abdb80 1 --2- 172.21.15.150:0/3798340849 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561bd160f300 0x561bd1611750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.147+0000 7f1053abdb80 4 client.15780 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.147+0000 7f1053abdb80 1 -- 172.21.15.150:0/3798340849 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bd15c8800 con 0x561bd160f300 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.147+0000 7f1041515700 1 --2- 172.21.15.150:0/3798340849 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561bd160f300 0x561bd1611750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1024ff9700 10 client.15780 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1041515700 1 --2- 172.21.15.150:0/3798340849 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561bd160f300 0x561bd1611750 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7297ae58be11ae84 server_cookie=c3997ed29647b979 in_seq=0 out_seq=0 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1024ff9700 1 -- 172.21.15.150:0/3798340849 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561bd15c8800 con 0x561bd160f300 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1024ff9700 10 client.15780 handle_command_reply: tid=0 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1053abdb80 1 client.15780 shutdown 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1053abdb80 1 -- 172.21.15.150:0/3798340849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 msgr2=0x561bd1603c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1053abdb80 1 --2- 172.21.15.150:0/3798340849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 0x561bd1603c20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f102c011e80 tx=0x7f102c01e420 comp rx=0 tx=0).stop 2024-05-10T04:04:45.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1053abdb80 1 -- 172.21.15.150:0/3798340849 shutdown_connections 2024-05-10T04:04:45.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1053abdb80 1 --2- 172.21.15.150:0/3798340849 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561bd160f300 0x561bd1611750 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1053abdb80 1 --2- 172.21.15.150:0/3798340849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15faf40 0x561bd160ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1024ff9700 0 client.15780 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:45.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1053abdb80 1 --2- 172.21.15.150:0/3798340849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 0x561bd16036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1053abdb80 1 --2- 172.21.15.150:0/3798340849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15fa5f0 0x561bd1603c20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1053abdb80 1 -- 172.21.15.150:0/3798340849 >> 172.21.15.150:0/3798340849 conn(0x561bd155bae0 msgr2=0x561bd15f4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:45.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.148+0000 7f1053abdb80 1 -- 172.21.15.150:0/3798340849 shutdown_connections 2024-05-10T04:04:45.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.149+0000 7f1053abdb80 1 -- 172.21.15.150:0/3798340849 wait complete. 2024-05-10T04:04:45.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.149+0000 7f1053abdb80 20 client.15780 trim_cache size 0 max 16384 2024-05-10T04:04:45.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1053abdb80 1 Processor -- start 2024-05-10T04:04:45.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1053abdb80 1 -- start start 2024-05-10T04:04:45.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1053abdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 0x561bd15fd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1053abdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15fa5f0 0x561bd160ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1053abdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 0x561bd16036e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1053abdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd15ee8b0 con 0x561bd15faf40 2024-05-10T04:04:45.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1053abdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd1603530 con 0x561bd15fa5f0 2024-05-10T04:04:45.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1053abdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd15f4bd0 con 0x561bd15f9720 2024-05-10T04:04:45.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1041d16700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 0x561bd16036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1041d16700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 0x561bd16036e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38542/0 (socket says 172.21.15.150:38542) 2024-05-10T04:04:45.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1041d16700 1 -- 172.21.15.150:0/3686218342 learned_addr learned my addr 172.21.15.150:0/3686218342 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:45.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1041515700 1 --2- 172.21.15.150:0/3686218342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 0x561bd15fd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1040d14700 1 --2- 172.21.15.150:0/3686218342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15fa5f0 0x561bd160ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1041d16700 1 -- 172.21.15.150:0/3686218342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 msgr2=0x561bd15fd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1041d16700 1 --2- 172.21.15.150:0/3686218342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 0x561bd15fd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1041d16700 1 -- 172.21.15.150:0/3686218342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15fa5f0 msgr2=0x561bd160ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1041d16700 1 --2- 172.21.15.150:0/3686218342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15fa5f0 0x561bd160ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.166+0000 7f1041d16700 1 -- 172.21.15.150:0/3686218342 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bd14df5e0 con 0x561bd15faf40 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.167+0000 7f1041d16700 1 --2- 172.21.15.150:0/3686218342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 0x561bd16036e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1034009b10 tx=0x7f1034009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8051b02564c228dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.167+0000 7f1027fff700 1 -- 172.21.15.150:0/3686218342 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034007ab0 con 0x561bd15faf40 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.167+0000 7f1027fff700 1 -- 172.21.15.150:0/3686218342 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1034007c10 con 0x561bd15faf40 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.167+0000 7f1027fff700 1 -- 172.21.15.150:0/3686218342 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034004890 con 0x561bd15faf40 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.167+0000 7f1053abdb80 1 -- 172.21.15.150:0/3686218342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 msgr2=0x561bd16036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.167+0000 7f1053abdb80 1 --2- 172.21.15.150:0/3686218342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 0x561bd16036e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1034009b10 tx=0x7f1034009040 comp rx=0 tx=0).stop 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.167+0000 7f1053abdb80 1 -- 172.21.15.150:0/3686218342 shutdown_connections 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.167+0000 7f1053abdb80 1 --2- 172.21.15.150:0/3686218342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 0x561bd16036e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.167+0000 7f1053abdb80 1 --2- 172.21.15.150:0/3686218342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15f9720 0x561bd15fd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.167+0000 7f1053abdb80 1 --2- 172.21.15.150:0/3686218342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15fa5f0 0x561bd160ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.167+0000 7f1053abdb80 1 -- 172.21.15.150:0/3686218342 >> 172.21.15.150:0/3686218342 conn(0x561bd155bae0 msgr2=0x561bd15f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.167+0000 7f1053abdb80 1 -- 172.21.15.150:0/3686218342 shutdown_connections 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.168+0000 7f1053abdb80 1 -- 172.21.15.150:0/3686218342 wait complete. 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.168+0000 7f1053abdb80 1 -- start start 2024-05-10T04:04:45.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.168+0000 7f1053abdb80 1 Processor -- start 2024-05-10T04:04:45.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.168+0000 7f1053abdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15f9720 0x561bd160a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.168+0000 7f1053abdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15fa5f0 0x561bd160ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.168+0000 7f1053abdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 0x561bd1608e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.168+0000 7f1053abdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd1609440 con 0x561bd15faf40 2024-05-10T04:04:45.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.168+0000 7f1053abdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd16095b0 con 0x561bd15f9720 2024-05-10T04:04:45.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.168+0000 7f1053abdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd1609770 con 0x561bd15fa5f0 2024-05-10T04:04:45.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.168+0000 7f1053abdb80 10 client.0 _ll_register_callbacks cb 0x561bd14bebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:45.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.168+0000 7f1041d16700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 0x561bd1608e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.168+0000 7f1041d16700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 0x561bd1608e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38554/0 (socket says 172.21.15.150:38554) 2024-05-10T04:04:45.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.168+0000 7f1041d16700 1 -- 172.21.15.150:0/927923308 learned_addr learned my addr 172.21.15.150:0/927923308 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:45.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1041515700 1 --2- 172.21.15.150:0/927923308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15f9720 0x561bd160a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1040d14700 1 --2- 172.21.15.150:0/927923308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15fa5f0 0x561bd160ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1041d16700 1 -- 172.21.15.150:0/927923308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15fa5f0 msgr2=0x561bd160ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1041d16700 1 --2- 172.21.15.150:0/927923308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15fa5f0 0x561bd160ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1041d16700 1 -- 172.21.15.150:0/927923308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15f9720 msgr2=0x561bd160a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1041d16700 1 --2- 172.21.15.150:0/927923308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15f9720 0x561bd160a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1041d16700 1 -- 172.21.15.150:0/927923308 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1034011040 con 0x561bd15faf40 2024-05-10T04:04:45.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1041d16700 1 --2- 172.21.15.150:0/927923308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 0x561bd1608e00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1034000f80 tx=0x7f1034003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adfda53916e9653 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:45.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1024ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:45.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1024ff9700 1 -- 172.21.15.150:0/927923308 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034007ab0 con 0x561bd15faf40 2024-05-10T04:04:45.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1024ff9700 1 -- 172.21.15.150:0/927923308 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1034021450 con 0x561bd15faf40 2024-05-10T04:04:45.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1053abdb80 1 -- 172.21.15.150:0/927923308 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f102c015070 con 0x561bd15faf40 2024-05-10T04:04:45.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1024ff9700 1 -- 172.21.15.150:0/927923308 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10340041b0 con 0x561bd15faf40 2024-05-10T04:04:45.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1053abdb80 10 client.15786 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:45.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1053abdb80 1 -- 172.21.15.150:0/927923308 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561bd139ed50 con 0x561bd15faf40 2024-05-10T04:04:45.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.169+0000 7f1024ff9700 1 -- 172.21.15.150:0/927923308 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1034033020 con 0x561bd15faf40 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.170+0000 7f1024ff9700 1 -- 172.21.15.150:0/927923308 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f103401d070 con 0x561bd15faf40 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.170+0000 7f1053abdb80 10 client.15786 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.170+0000 7f1053abdb80 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.170+0000 7f1053abdb80 20 client.15786 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.170+0000 7f1053abdb80 1 --2- 172.21.15.150:0/927923308 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561bd1666a80 0x561bd1668ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.170+0000 7f1053abdb80 4 client.15786 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.170+0000 7f1053abdb80 1 -- 172.21.15.150:0/927923308 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561bd1511f40 con 0x561bd1666a80 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.170+0000 7f1041515700 1 --2- 172.21.15.150:0/927923308 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561bd1666a80 0x561bd1668ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.170+0000 7f1024ff9700 10 client.15786 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.170+0000 7f1041515700 1 --2- 172.21.15.150:0/927923308 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561bd1666a80 0x561bd1668ea0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b6f7ed67c28b14 server_cookie=3e2d9b61cfb2d368 in_seq=0 out_seq=0 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.171+0000 7f1024ff9700 1 -- 172.21.15.150:0/927923308 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561bd1511f40 con 0x561bd1666a80 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.171+0000 7f1024ff9700 10 client.15786 handle_command_reply: tid=0 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.171+0000 7f1053abdb80 1 client.15786 shutdown 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.171+0000 7f1053abdb80 1 -- 172.21.15.150:0/927923308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 msgr2=0x561bd1608e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.171+0000 7f1053abdb80 1 --2- 172.21.15.150:0/927923308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 0x561bd1608e00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1034000f80 tx=0x7f1034003ce0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.171+0000 7f1053abdb80 1 -- 172.21.15.150:0/927923308 shutdown_connections 2024-05-10T04:04:45.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.171+0000 7f1053abdb80 1 --2- 172.21.15.150:0/927923308 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561bd1666a80 0x561bd1668ea0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1053abdb80 1 --2- 172.21.15.150:0/927923308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561bd15faf40 0x561bd1608e00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1053abdb80 1 --2- 172.21.15.150:0/927923308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561bd15f9720 0x561bd160a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1053abdb80 1 --2- 172.21.15.150:0/927923308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd15fa5f0 0x561bd160ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1024ff9700 0 client.15786 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:45.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1053abdb80 1 -- 172.21.15.150:0/927923308 >> 172.21.15.150:0/927923308 conn(0x561bd155bae0 msgr2=0x561bd15f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:45.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1053abdb80 1 -- 172.21.15.150:0/927923308 shutdown_connections 2024-05-10T04:04:45.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1053abdb80 1 -- 172.21.15.150:0/927923308 wait complete. 2024-05-10T04:04:45.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1053abdb80 20 client.15786 trim_cache size 0 max 16384 2024-05-10T04:04:45.180 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:04:45.181 INFO:teuthology.orchestra.run.smithi150.stdout: "return_code": 0, 2024-05-10T04:04:45.181 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_tag": "81b407c1-9171-4946-8f14-6683f708b2b7", 2024-05-10T04:04:45.181 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": "asynchronous" 2024-05-10T04:04:45.181 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:04:45.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1043779700 1 -- 172.21.15.150:0/3875092916 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f10280a12e0 msgr2=0x7f10280a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1043779700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f10280a12e0 0x7f10280a37f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f103c0687f0 tx=0x7f102c00b410 comp rx=0 tx=0).stop 2024-05-10T04:04:45.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1043779700 1 -- 172.21.15.150:0/3875092916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f103c100ac0 msgr2=0x7f103c068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1043779700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f103c100ac0 0x7f103c068e50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1038006010 tx=0x7f103800f690 comp rx=0 tx=0).stop 2024-05-10T04:04:45.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1043779700 1 -- 172.21.15.150:0/3875092916 shutdown_connections 2024-05-10T04:04:45.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1043779700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f10280a12e0 0x7f10280a37f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1043779700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f103c102580 0x7f103c19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1043779700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f103c100ac0 0x7f103c068e50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1043779700 1 --2- 172.21.15.150:0/3875092916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f103c101be0 0x7f103c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1043779700 1 -- 172.21.15.150:0/3875092916 >> 172.21.15.150:0/3875092916 conn(0x7f103c0fc0d0 msgr2=0x7f103c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:45.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1043779700 1 -- 172.21.15.150:0/3875092916 shutdown_connections 2024-05-10T04:04:45.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.172+0000 7f1043779700 1 -- 172.21.15.150:0/3875092916 wait complete. 2024-05-10T04:04:45.187 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:04:45.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.319+0000 7f810bf13700 1 -- 172.21.15.150:0/1092224333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8104100270 msgr2=0x7f8104100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.319+0000 7f810bf13700 1 --2- 172.21.15.150:0/1092224333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8104100270 0x7f8104100680 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8100009990 tx=0x7f8100009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.319+0000 7f810bf13700 1 -- 172.21.15.150:0/1092224333 shutdown_connections 2024-05-10T04:04:45.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.319+0000 7f810bf13700 1 --2- 172.21.15.150:0/1092224333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8104101d30 0x7f81041085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.319+0000 7f810bf13700 1 --2- 172.21.15.150:0/1092224333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8104100270 0x7f8104100680 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.319+0000 7f810bf13700 1 --2- 172.21.15.150:0/1092224333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8104101390 0x7f81041017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.319+0000 7f810bf13700 1 -- 172.21.15.150:0/1092224333 >> 172.21.15.150:0/1092224333 conn(0x7f81040fb880 msgr2=0x7f81040fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:45.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.319+0000 7f810bf13700 1 -- 172.21.15.150:0/1092224333 shutdown_connections 2024-05-10T04:04:45.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.319+0000 7f810bf13700 1 -- 172.21.15.150:0/1092224333 wait complete. 2024-05-10T04:04:45.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.320+0000 7f810bf13700 1 Processor -- start 2024-05-10T04:04:45.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.320+0000 7f810bf13700 1 -- start start 2024-05-10T04:04:45.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.320+0000 7f810bf13700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8104100270 0x7f8104197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.320+0000 7f810bf13700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8104101390 0x7f8104198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.320+0000 7f810bf13700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8104101d30 0x7f810419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.320+0000 7f810bf13700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810419fa20 con 0x7f8104100270 2024-05-10T04:04:45.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.320+0000 7f810bf13700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810419fb90 con 0x7f8104101d30 2024-05-10T04:04:45.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.320+0000 7f810bf13700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810419fd50 con 0x7f8104101390 2024-05-10T04:04:45.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.320+0000 7f8109caf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8104100270 0x7f8104197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f8109caf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8104100270 0x7f8104197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38576/0 (socket says 172.21.15.150:38576) 2024-05-10T04:04:45.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f8109caf700 1 -- 172.21.15.150:0/1147850674 learned_addr learned my addr 172.21.15.150:0/1147850674 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:45.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f81094ae700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8104101390 0x7f8104198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f810a4b0700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8104101d30 0x7f810419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f8109caf700 1 -- 172.21.15.150:0/1147850674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8104101390 msgr2=0x7f8104198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f8109caf700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8104101390 0x7f8104198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f8109caf700 1 -- 172.21.15.150:0/1147850674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8104101d30 msgr2=0x7f810419f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f8109caf700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8104101d30 0x7f810419f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f8109caf700 1 -- 172.21.15.150:0/1147850674 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81000095f0 con 0x7f8104100270 2024-05-10T04:04:45.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f810a4b0700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8104101d30 0x7f810419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:45.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f8109caf700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8104100270 0x7f8104197f40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8100000c00 tx=0x7f8100005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:45.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f80faffd700 1 -- 172.21.15.150:0/1147850674 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810001d070 con 0x7f8104100270 2024-05-10T04:04:45.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f80faffd700 1 -- 172.21.15.150:0/1147850674 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8100021410 con 0x7f8104100270 2024-05-10T04:04:45.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f810bf13700 1 -- 172.21.15.150:0/1147850674 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f810419ffd0 con 0x7f8104100270 2024-05-10T04:04:45.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f80faffd700 1 -- 172.21.15.150:0/1147850674 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8100017650 con 0x7f8104100270 2024-05-10T04:04:45.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.321+0000 7f810bf13700 1 -- 172.21.15.150:0/1147850674 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81041a0340 con 0x7f8104100270 2024-05-10T04:04:45.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.324+0000 7f80faffd700 1 -- 172.21.15.150:0/1147850674 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f81000177b0 con 0x7f8104100270 2024-05-10T04:04:45.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.325+0000 7f80faffd700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f80f00a1260 0x7f80f00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.325+0000 7f81094ae700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f80f00a1260 0x7f80f00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.325+0000 7f80faffd700 1 -- 172.21.15.150:0/1147850674 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f81000ce0e0 con 0x7f8104100270 2024-05-10T04:04:45.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.325+0000 7f81094ae700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f80f00a1260 0x7f80f00a3770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8104198fe0 tx=0x7f80f4010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:45.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.325+0000 7f811c257b80 1 Processor -- start 2024-05-10T04:04:45.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f811c257b80 1 -- start start 2024-05-10T04:04:45.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f811c257b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb606a0 0x56295eb60af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f811c257b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb61570 0x56295eb61980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f811c257b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb687c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f811c257b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295ea87150 con 0x56295eb61ec0 2024-05-10T04:04:45.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f811c257b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295ea84b10 con 0x56295eb61570 2024-05-10T04:04:45.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f811c257b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295ea80400 con 0x56295eb606a0 2024-05-10T04:04:45.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f810a4b0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f810a4b0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb687c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38584/0 (socket says 172.21.15.150:38584) 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f810a4b0700 1 -- 172.21.15.150:0/3540367111 learned_addr learned my addr 172.21.15.150:0/3540367111 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f8109caf700 1 --2- 172.21.15.150:0/3540367111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb606a0 0x56295eb60af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f81094ae700 1 --2- 172.21.15.150:0/3540367111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb61570 0x56295eb61980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f810a4b0700 1 -- 172.21.15.150:0/3540367111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb606a0 msgr2=0x56295eb60af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f810a4b0700 1 --2- 172.21.15.150:0/3540367111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb606a0 0x56295eb60af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f810a4b0700 1 -- 172.21.15.150:0/3540367111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb61570 msgr2=0x56295eb61980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f810a4b0700 1 --2- 172.21.15.150:0/3540367111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb61570 0x56295eb61980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f810a4b0700 1 -- 172.21.15.150:0/3540367111 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56295e9013b0 con 0x56295eb61ec0 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.326+0000 7f810a4b0700 1 --2- 172.21.15.150:0/3540367111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb687c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0093a0 tx=0x7f80fc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7753418018c204f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.327+0000 7f80effff700 1 -- 172.21.15.150:0/3540367111 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00e030 con 0x56295eb61ec0 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.327+0000 7f80effff700 1 -- 172.21.15.150:0/3540367111 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80fc00f040 con 0x56295eb61ec0 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.327+0000 7f80effff700 1 -- 172.21.15.150:0/3540367111 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc015bd0 con 0x56295eb61ec0 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.327+0000 7f811c257b80 1 -- 172.21.15.150:0/3540367111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 msgr2=0x56295eb687c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.327+0000 7f811c257b80 1 --2- 172.21.15.150:0/3540367111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb687c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0093a0 tx=0x7f80fc009920 comp rx=0 tx=0).stop 2024-05-10T04:04:45.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.327+0000 7f811c257b80 1 -- 172.21.15.150:0/3540367111 shutdown_connections 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.327+0000 7f811c257b80 1 --2- 172.21.15.150:0/3540367111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb687c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.327+0000 7f811c257b80 1 --2- 172.21.15.150:0/3540367111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb606a0 0x56295eb60af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.327+0000 7f811c257b80 1 --2- 172.21.15.150:0/3540367111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb61570 0x56295eb61980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.328+0000 7f811c257b80 1 -- 172.21.15.150:0/3540367111 >> 172.21.15.150:0/3540367111 conn(0x56295eac2a60 msgr2=0x56295eb5afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.328+0000 7f811c257b80 1 -- 172.21.15.150:0/3540367111 shutdown_connections 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.328+0000 7f811c257b80 1 -- 172.21.15.150:0/3540367111 wait complete. 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.328+0000 7f811c257b80 1 -- start start 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.328+0000 7f811c257b80 1 Processor -- start 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.328+0000 7f811c257b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb606a0 0x56295eb6a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.328+0000 7f811c257b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb61570 0x56295eb6ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.328+0000 7f811c257b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61ec0 0x56295eb71af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f811c257b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295ea1c560 con 0x56295eb606a0 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f811c257b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295ea1c200 con 0x56295eb61570 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f811c257b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295e9036a0 con 0x56295eb61ec0 2024-05-10T04:04:45.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f811c257b80 10 client.0 _ll_register_callbacks cb 0x56295ea25b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f8109caf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb606a0 0x56295eb6a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f8109caf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb606a0 0x56295eb6a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38586/0 (socket says 172.21.15.150:38586) 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f8109caf700 1 -- 172.21.15.150:0/2709639322 learned_addr learned my addr 172.21.15.150:0/2709639322 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f81094ae700 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb61570 0x56295eb6ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f810a4b0700 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61ec0 0x56295eb71af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f8109caf700 1 -- 172.21.15.150:0/2709639322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61ec0 msgr2=0x56295eb71af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f8109caf700 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61ec0 0x56295eb71af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f8109caf700 1 -- 172.21.15.150:0/2709639322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb61570 msgr2=0x56295eb6ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f8109caf700 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb61570 0x56295eb6ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f8109caf700 1 -- 172.21.15.150:0/2709639322 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80fc009050 con 0x56295eb606a0 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f810a4b0700 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61ec0 0x56295eb71af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f81094ae700 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb61570 0x56295eb6ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f8109caf700 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb606a0 0x56295eb6a650 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8100004800 tx=0x7f810009b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accded5153fa0c81 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:45.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f80ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f80ed7fa700 1 -- 172.21.15.150:0/2709639322 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810009ef20 con 0x56295eb606a0 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f80ed7fa700 1 -- 172.21.15.150:0/2709639322 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f810009fe40 con 0x56295eb606a0 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f811c257b80 1 -- 172.21.15.150:0/2709639322 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56295e9d9270 con 0x56295eb606a0 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f80ed7fa700 1 -- 172.21.15.150:0/2709639322 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81000a34e0 con 0x56295eb606a0 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f811c257b80 10 client.15798 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.329+0000 7f811c257b80 1 -- 172.21.15.150:0/2709639322 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56295e8adb60 con 0x56295eb606a0 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.330+0000 7f80ed7fa700 1 -- 172.21.15.150:0/2709639322 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8100065080 con 0x56295eb606a0 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.330+0000 7f80ed7fa700 1 -- 172.21.15.150:0/2709639322 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f81000a3050 con 0x56295eb606a0 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.330+0000 7f811c257b80 10 client.15798 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.330+0000 7f811c257b80 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.330+0000 7f811c257b80 20 client.15798 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.330+0000 7f811c257b80 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56295eb75ec0 0x56295eb78310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.330+0000 7f811c257b80 4 client.15798 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.330+0000 7f811c257b80 1 -- 172.21.15.150:0/2709639322 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56295eb2f780 con 0x56295eb75ec0 2024-05-10T04:04:45.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.330+0000 7f81094ae700 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56295eb75ec0 0x56295eb78310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.331+0000 7f80ed7fa700 10 client.15798 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.331+0000 7f81094ae700 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56295eb75ec0 0x56295eb78310 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167bd784df15b429 server_cookie=e087dbf56cd7ac44 in_seq=0 out_seq=0 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.331+0000 7f80ed7fa700 1 -- 172.21.15.150:0/2709639322 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56295eb2f780 con 0x56295eb75ec0 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.331+0000 7f80ed7fa700 10 client.15798 handle_command_reply: tid=0 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.331+0000 7f811c257b80 1 client.15798 shutdown 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.332+0000 7f811c257b80 1 -- 172.21.15.150:0/2709639322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb606a0 msgr2=0x56295eb6a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.332+0000 7f811c257b80 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb606a0 0x56295eb6a650 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8100004800 tx=0x7f810009b7f0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.332+0000 7f811c257b80 1 -- 172.21.15.150:0/2709639322 shutdown_connections 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.332+0000 7f811c257b80 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56295eb75ec0 0x56295eb78310 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.332+0000 7f811c257b80 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61ec0 0x56295eb71af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.332+0000 7f811c257b80 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb606a0 0x56295eb6a650 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.332+0000 7f80ed7fa700 0 client.15798 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.332+0000 7f811c257b80 1 --2- 172.21.15.150:0/2709639322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb61570 0x56295eb6ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.332+0000 7f811c257b80 1 -- 172.21.15.150:0/2709639322 >> 172.21.15.150:0/2709639322 conn(0x56295eac2a60 msgr2=0x56295eb5afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:45.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.332+0000 7f811c257b80 1 -- 172.21.15.150:0/2709639322 shutdown_connections 2024-05-10T04:04:45.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.332+0000 7f811c257b80 1 -- 172.21.15.150:0/2709639322 wait complete. 2024-05-10T04:04:45.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.332+0000 7f811c257b80 20 client.15798 trim_cache size 0 max 16384 2024-05-10T04:04:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f811c257b80 1 Processor -- start 2024-05-10T04:04:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f811c257b80 1 -- start start 2024-05-10T04:04:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f811c257b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb606a0 0x56295eb65310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f811c257b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61570 0x56295eb71ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f811c257b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb65850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f811c257b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295eb78a30 con 0x56295eb61ec0 2024-05-10T04:04:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f811c257b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295eb75bf0 con 0x56295eb606a0 2024-05-10T04:04:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f811c257b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295eb5afb0 con 0x56295eb61570 2024-05-10T04:04:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f810a4b0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb65850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f810a4b0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb65850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38594/0 (socket says 172.21.15.150:38594) 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f810a4b0700 1 -- 172.21.15.150:0/3528109756 learned_addr learned my addr 172.21.15.150:0/3528109756 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f8109caf700 1 --2- 172.21.15.150:0/3528109756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb606a0 0x56295eb65310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f81094ae700 1 --2- 172.21.15.150:0/3528109756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61570 0x56295eb71ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f810a4b0700 1 -- 172.21.15.150:0/3528109756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61570 msgr2=0x56295eb71ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f810a4b0700 1 --2- 172.21.15.150:0/3528109756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61570 0x56295eb71ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f810a4b0700 1 -- 172.21.15.150:0/3528109756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb606a0 msgr2=0x56295eb65310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f810a4b0700 1 --2- 172.21.15.150:0/3528109756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb606a0 0x56295eb65310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f810a4b0700 1 -- 172.21.15.150:0/3528109756 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56295ea46560 con 0x56295eb61ec0 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f8109caf700 1 --2- 172.21.15.150:0/3528109756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb606a0 0x56295eb65310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.349+0000 7f810a4b0700 1 --2- 172.21.15.150:0/3528109756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb65850 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0093a0 tx=0x7f80fc0060f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b5b4cc2c0b51f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.350+0000 7f80f8ff9700 1 -- 172.21.15.150:0/3528109756 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00e070 con 0x56295eb61ec0 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.350+0000 7f80f8ff9700 1 -- 172.21.15.150:0/3528109756 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80fc015580 con 0x56295eb61ec0 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.350+0000 7f80f8ff9700 1 -- 172.21.15.150:0/3528109756 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc015e20 con 0x56295eb61ec0 2024-05-10T04:04:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.350+0000 7f811c257b80 1 -- 172.21.15.150:0/3528109756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 msgr2=0x56295eb65850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.350+0000 7f811c257b80 1 --2- 172.21.15.150:0/3528109756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb65850 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0093a0 tx=0x7f80fc0060f0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.350+0000 7f811c257b80 1 -- 172.21.15.150:0/3528109756 shutdown_connections 2024-05-10T04:04:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.350+0000 7f811c257b80 1 --2- 172.21.15.150:0/3528109756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb65850 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.350+0000 7f811c257b80 1 --2- 172.21.15.150:0/3528109756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb606a0 0x56295eb65310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.350+0000 7f811c257b80 1 --2- 172.21.15.150:0/3528109756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61570 0x56295eb71ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.350+0000 7f811c257b80 1 -- 172.21.15.150:0/3528109756 >> 172.21.15.150:0/3528109756 conn(0x56295eac2a60 msgr2=0x56295eb647a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.350+0000 7f811c257b80 1 -- 172.21.15.150:0/3528109756 shutdown_connections 2024-05-10T04:04:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.350+0000 7f811c257b80 1 -- 172.21.15.150:0/3528109756 wait complete. 2024-05-10T04:04:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.351+0000 7f811c257b80 1 -- start start 2024-05-10T04:04:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.351+0000 7f811c257b80 1 Processor -- start 2024-05-10T04:04:45.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.351+0000 7f811c257b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb606a0 0x56295eb6f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.351+0000 7f811c257b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61570 0x56295eb6feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.351+0000 7f811c257b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb70d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.351+0000 7f811c257b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295eb71280 con 0x56295eb61ec0 2024-05-10T04:04:45.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.351+0000 7f811c257b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295eb713f0 con 0x56295eb606a0 2024-05-10T04:04:45.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.351+0000 7f811c257b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295eb715b0 con 0x56295eb61570 2024-05-10T04:04:45.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f811c257b80 10 client.0 _ll_register_callbacks cb 0x56295ea25b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:45.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f810a4b0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb70d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f810a4b0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb70d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38600/0 (socket says 172.21.15.150:38600) 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f810a4b0700 1 -- 172.21.15.150:0/3554094249 learned_addr learned my addr 172.21.15.150:0/3554094249 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f8109caf700 1 --2- 172.21.15.150:0/3554094249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb606a0 0x56295eb6f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f81094ae700 1 --2- 172.21.15.150:0/3554094249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61570 0x56295eb6feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f810a4b0700 1 -- 172.21.15.150:0/3554094249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61570 msgr2=0x56295eb6feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f810a4b0700 1 --2- 172.21.15.150:0/3554094249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61570 0x56295eb6feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f810a4b0700 1 -- 172.21.15.150:0/3554094249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb606a0 msgr2=0x56295eb6f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f810a4b0700 1 --2- 172.21.15.150:0/3554094249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb606a0 0x56295eb6f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f810a4b0700 1 -- 172.21.15.150:0/3554094249 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56295e9013b0 con 0x56295eb61ec0 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f810a4b0700 1 --2- 172.21.15.150:0/3554094249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb70d40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f80fc004c80 tx=0x7f80fc004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb33a45bea0c59fd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f80ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f80ed7fa700 1 -- 172.21.15.150:0/3554094249 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00e050 con 0x56295eb61ec0 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f80ed7fa700 1 -- 172.21.15.150:0/3554094249 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f80fc003e10 con 0x56295eb61ec0 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f811c257b80 1 -- 172.21.15.150:0/3554094249 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80fc00f040 con 0x56295eb61ec0 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f80ed7fa700 1 -- 172.21.15.150:0/3554094249 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80fc004020 con 0x56295eb61ec0 2024-05-10T04:04:45.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f811c257b80 10 client.15804 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.352+0000 7f811c257b80 1 -- 172.21.15.150:0/3554094249 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56295e905cd0 con 0x56295eb61ec0 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.353+0000 7f80ed7fa700 1 -- 172.21.15.150:0/3554094249 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80fc00f040 con 0x56295eb61ec0 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.353+0000 7f80ed7fa700 1 -- 172.21.15.150:0/3554094249 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f80fc0148f0 con 0x56295eb61ec0 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.353+0000 7f811c257b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.353+0000 7f811c257b80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.353+0000 7f811c257b80 20 client.15804 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.353+0000 7f811c257b80 1 --2- 172.21.15.150:0/3554094249 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56295eb75ec0 0x56295ebceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.353+0000 7f811c257b80 4 client.15804 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.353+0000 7f811c257b80 1 -- 172.21.15.150:0/3554094249 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56295ea78ec0 con 0x56295eb75ec0 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.353+0000 7f8109caf700 1 --2- 172.21.15.150:0/3554094249 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56295eb75ec0 0x56295ebceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f80ed7fa700 10 client.15804 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f8109caf700 1 --2- 172.21.15.150:0/3554094249 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56295eb75ec0 0x56295ebceef0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed748dac4833bd6c server_cookie=74af5bbff6a5856f in_seq=0 out_seq=0 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f80ed7fa700 1 -- 172.21.15.150:0/3554094249 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56295ea78ec0 con 0x56295eb75ec0 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f80ed7fa700 10 client.15804 handle_command_reply: tid=0 2024-05-10T04:04:45.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f811c257b80 1 client.15804 shutdown 2024-05-10T04:04:45.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f811c257b80 1 -- 172.21.15.150:0/3554094249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 msgr2=0x56295eb70d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f811c257b80 1 --2- 172.21.15.150:0/3554094249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb70d40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f80fc004c80 tx=0x7f80fc004cb0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f811c257b80 1 -- 172.21.15.150:0/3554094249 shutdown_connections 2024-05-10T04:04:45.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f811c257b80 1 --2- 172.21.15.150:0/3554094249 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56295eb75ec0 0x56295ebceef0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f811c257b80 1 --2- 172.21.15.150:0/3554094249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56295eb61ec0 0x56295eb70d40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f811c257b80 1 --2- 172.21.15.150:0/3554094249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295eb606a0 0x56295eb6f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f80ed7fa700 0 client.15804 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:45.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f811c257b80 1 --2- 172.21.15.150:0/3554094249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56295eb61570 0x56295eb6feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f811c257b80 1 -- 172.21.15.150:0/3554094249 >> 172.21.15.150:0/3554094249 conn(0x56295eac2a60 msgr2=0x56295eb647a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:45.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f811c257b80 1 -- 172.21.15.150:0/3554094249 shutdown_connections 2024-05-10T04:04:45.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f811c257b80 1 -- 172.21.15.150:0/3554094249 wait complete. 2024-05-10T04:04:45.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.354+0000 7f811c257b80 20 client.15804 trim_cache size 0 max 16384 2024-05-10T04:04:45.363 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:04:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T04:04:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": { 2024-05-10T04:04:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "81b407c1-9171-4946-8f14-6683f708b2b7": { 2024-05-10T04:04:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "path": "/", 2024-05-10T04:04:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "tag": "81b407c1-9171-4946-8f14-6683f708b2b7", 2024-05-10T04:04:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "options": "recursive,force" 2024-05-10T04:04:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:04:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:04:45.363 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:04:45.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.355+0000 7f810bf13700 1 -- 172.21.15.150:0/1147850674 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f80f00a1260 msgr2=0x7f80f00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.355+0000 7f810bf13700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f80f00a1260 0x7f80f00a3770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8104198fe0 tx=0x7f80f4010040 comp rx=0 tx=0).stop 2024-05-10T04:04:45.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.355+0000 7f810bf13700 1 -- 172.21.15.150:0/1147850674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8104100270 msgr2=0x7f8104197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:45.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.355+0000 7f810bf13700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8104100270 0x7f8104197f40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8100000c00 tx=0x7f8100005b00 comp rx=0 tx=0).stop 2024-05-10T04:04:45.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.355+0000 7f810bf13700 1 -- 172.21.15.150:0/1147850674 shutdown_connections 2024-05-10T04:04:45.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.355+0000 7f810bf13700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f80f00a1260 0x7f80f00a3770 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.355+0000 7f810bf13700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8104101d30 0x7f810419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.355+0000 7f810bf13700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8104100270 0x7f8104197f40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.355+0000 7f810bf13700 1 --2- 172.21.15.150:0/1147850674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8104101390 0x7f8104198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:45.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.355+0000 7f810bf13700 1 -- 172.21.15.150:0/1147850674 >> 172.21.15.150:0/1147850674 conn(0x7f81040fb880 msgr2=0x7f81041067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:45.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.355+0000 7f810bf13700 1 -- 172.21.15.150:0/1147850674 shutdown_connections 2024-05-10T04:04:45.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:04:45.355+0000 7f810bf13700 1 -- 172.21.15.150:0/1147850674 wait complete. 2024-05-10T04:04:45.369 INFO:tasks.cephfs.filesystem:scrub status for tag:81b407c1-9171-4946-8f14-6683f708b2b7 - {'path': '/', 'tag': '81b407c1-9171-4946-8f14-6683f708b2b7', 'options': 'recursive,force'} 2024-05-10T04:04:46.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:46 smithi150 ceph-mon[42477]: pgmap v295: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T04:04:46.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:46 smithi150 ceph-mon[42477]: scrub queued for path: / 2024-05-10T04:04:46.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:46 smithi150 ceph-mon[42477]: scrub summary: idle+waiting paths [/] 2024-05-10T04:04:46.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:46 smithi150 ceph-mon[42477]: scrub summary: active paths [/] 2024-05-10T04:04:46.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:46 smithi150 ceph-mon[42477]: scrub summary: idle+waiting paths [/] 2024-05-10T04:04:46.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:46 smithi150 ceph-mon[42477]: scrub summary: idle 2024-05-10T04:04:46.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:46 smithi181 ceph-mon[46602]: pgmap v295: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T04:04:46.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:46 smithi181 ceph-mon[46602]: scrub queued for path: / 2024-05-10T04:04:46.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:46 smithi181 ceph-mon[46602]: scrub summary: idle+waiting paths [/] 2024-05-10T04:04:46.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:46 smithi181 ceph-mon[46602]: scrub summary: active paths [/] 2024-05-10T04:04:46.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:46 smithi181 ceph-mon[46602]: scrub summary: idle+waiting paths [/] 2024-05-10T04:04:46.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:46 smithi181 ceph-mon[46602]: scrub summary: idle 2024-05-10T04:04:46.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:46 smithi161 ceph-mon[46918]: pgmap v295: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T04:04:46.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:46 smithi161 ceph-mon[46918]: scrub queued for path: / 2024-05-10T04:04:46.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:46 smithi161 ceph-mon[46918]: scrub summary: idle+waiting paths [/] 2024-05-10T04:04:46.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:46 smithi161 ceph-mon[46918]: scrub summary: active paths [/] 2024-05-10T04:04:46.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:46 smithi161 ceph-mon[46918]: scrub summary: idle+waiting paths [/] 2024-05-10T04:04:46.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:46 smithi161 ceph-mon[46918]: scrub summary: idle 2024-05-10T04:04:48.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:48 smithi150 ceph-mon[42477]: pgmap v296: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T04:04:48.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:48 smithi181 ceph-mon[46602]: pgmap v296: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T04:04:48.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:48 smithi161 ceph-mon[46918]: pgmap v296: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T04:04:50.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:50 smithi150 ceph-mon[42477]: pgmap v297: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T04:04:50.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:50 smithi181 ceph-mon[46602]: pgmap v297: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T04:04:50.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:50 smithi161 ceph-mon[46918]: pgmap v297: 129 pgs: 129 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T04:04:51.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:51 smithi150 ceph-mon[42477]: pgmap v298: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-10T04:04:51.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:51 smithi181 ceph-mon[46602]: pgmap v298: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-10T04:04:51.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:51 smithi161 ceph-mon[46918]: pgmap v298: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-10T04:04:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.949+0000 7fc15737c700 1 -- 172.21.15.150:0/4086042984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1501004d0 msgr2=0x7fc1501028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.949+0000 7fc15737c700 1 --2- 172.21.15.150:0/4086042984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1501004d0 0x7fc1501028f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009990 tx=0x7fc14c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.949+0000 7fc15737c700 1 -- 172.21.15.150:0/4086042984 shutdown_connections 2024-05-10T04:04:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.949+0000 7fc15737c700 1 --2- 172.21.15.150:0/4086042984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc150105790 0x7fc150105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.949+0000 7fc15737c700 1 --2- 172.21.15.150:0/4086042984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1501004d0 0x7fc1501028f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.949+0000 7fc15737c700 1 --2- 172.21.15.150:0/4086042984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc150102e30 0x7fc150105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.949+0000 7fc15737c700 1 -- 172.21.15.150:0/4086042984 >> 172.21.15.150:0/4086042984 conn(0x7fc1500fc100 msgr2=0x7fc1500fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.949+0000 7fc15737c700 1 -- 172.21.15.150:0/4086042984 shutdown_connections 2024-05-10T04:04:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.949+0000 7fc15737c700 1 -- 172.21.15.150:0/4086042984 wait complete. 2024-05-10T04:04:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc15737c700 1 Processor -- start 2024-05-10T04:04:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc15737c700 1 -- start start 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc15737c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1501004d0 0x7fc15019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc15737c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc150102e30 0x7fc15019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc15737c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc150105790 0x7fc1501a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc15737c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1501a6000 con 0x7fc1501004d0 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc15737c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1501a6140 con 0x7fc150102e30 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc15737c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1501a62d0 con 0x7fc150105790 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc155118700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1501004d0 0x7fc15019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc155118700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1501004d0 0x7fc15019e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39976/0 (socket says 172.21.15.150:39976) 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc155118700 1 -- 172.21.15.150:0/726686659 learned_addr learned my addr 172.21.15.150:0/726686659 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc154917700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc150102e30 0x7fc15019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc155919700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc150105790 0x7fc1501a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc155118700 1 -- 172.21.15.150:0/726686659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc150105790 msgr2=0x7fc1501a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc155118700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc150105790 0x7fc1501a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc155118700 1 -- 172.21.15.150:0/726686659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc150102e30 msgr2=0x7fc15019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc155118700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc150102e30 0x7fc15019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc155118700 1 -- 172.21.15.150:0/726686659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c0095f0 con 0x7fc1501004d0 2024-05-10T04:04:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.950+0000 7fc155919700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc150105790 0x7fc1501a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.951+0000 7fc155118700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1501004d0 0x7fc15019e5f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc14c005740 tx=0x7fc14c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.951+0000 7fc1467fc700 1 -- 172.21.15.150:0/726686659 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c01d070 con 0x7fc1501004d0 2024-05-10T04:04:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.951+0000 7fc1467fc700 1 -- 172.21.15.150:0/726686659 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc14c021410 con 0x7fc1501004d0 2024-05-10T04:04:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.951+0000 7fc15737c700 1 -- 172.21.15.150:0/726686659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1501a6550 con 0x7fc1501004d0 2024-05-10T04:04:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.951+0000 7fc1467fc700 1 -- 172.21.15.150:0/726686659 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c017650 con 0x7fc1501004d0 2024-05-10T04:04:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.951+0000 7fc15737c700 1 -- 172.21.15.150:0/726686659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1501a6970 con 0x7fc1501004d0 2024-05-10T04:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.953+0000 7fc1467fc700 1 -- 172.21.15.150:0/726686659 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc14c021580 con 0x7fc1501004d0 2024-05-10T04:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.954+0000 7fc1467fc700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc13c0a11b0 0x7fc13c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.954+0000 7fc1467fc700 1 -- 172.21.15.150:0/726686659 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc14c0ce240 con 0x7fc1501004d0 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.954+0000 7fc154917700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc13c0a11b0 0x7fc13c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.955+0000 7fc154917700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc13c0a11b0 0x7fc13c0a36c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc15019f690 tx=0x7fc14000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.955+0000 7fc1676c0b80 1 Processor -- start 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.955+0000 7fc1676c0b80 1 -- start start 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.955+0000 7fc1676c0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 0x5639d170e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.955+0000 7fc1676c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170f050 0x5639d170f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.955+0000 7fc1676c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f9a0 0x5639d17162a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.955+0000 7fc1676c0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d16350f0 con 0x5639d170e180 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.955+0000 7fc1676c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d1632ab0 con 0x5639d170f9a0 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.955+0000 7fc1676c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d162e3a0 con 0x5639d170f050 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.955+0000 7fc155118700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 0x5639d170e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.955+0000 7fc155118700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 0x5639d170e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39988/0 (socket says 172.21.15.150:39988) 2024-05-10T04:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.955+0000 7fc155118700 1 -- 172.21.15.150:0/963608822 learned_addr learned my addr 172.21.15.150:0/963608822 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.956+0000 7fc155919700 1 --2- 172.21.15.150:0/963608822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f9a0 0x5639d17162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.956+0000 7fc154917700 1 --2- 172.21.15.150:0/963608822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170f050 0x5639d170f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.956+0000 7fc155118700 1 -- 172.21.15.150:0/963608822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170f050 msgr2=0x5639d170f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.956+0000 7fc155118700 1 --2- 172.21.15.150:0/963608822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170f050 0x5639d170f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.956+0000 7fc155118700 1 -- 172.21.15.150:0/963608822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f9a0 msgr2=0x5639d17162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.956+0000 7fc155118700 1 --2- 172.21.15.150:0/963608822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f9a0 0x5639d17162a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.956+0000 7fc155118700 1 -- 172.21.15.150:0/963608822 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639d14af350 con 0x5639d170e180 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.956+0000 7fc154917700 1 --2- 172.21.15.150:0/963608822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170f050 0x5639d170f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.956+0000 7fc155118700 1 --2- 172.21.15.150:0/963608822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 0x5639d170e5b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc14c006010 tx=0x7fc14c09af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2526503b584c19d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.956+0000 7fc13b7fe700 1 -- 172.21.15.150:0/963608822 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c09b2b0 con 0x5639d170e180 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.956+0000 7fc13b7fe700 1 -- 172.21.15.150:0/963608822 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc14c09f0e0 con 0x5639d170e180 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.956+0000 7fc13b7fe700 1 -- 172.21.15.150:0/963608822 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c09a7a0 con 0x5639d170e180 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.957+0000 7fc1676c0b80 1 -- 172.21.15.150:0/963608822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 msgr2=0x5639d170e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.957+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/963608822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 0x5639d170e5b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc14c006010 tx=0x7fc14c09af20 comp rx=0 tx=0).stop 2024-05-10T04:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.957+0000 7fc1676c0b80 1 -- 172.21.15.150:0/963608822 shutdown_connections 2024-05-10T04:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.957+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/963608822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f9a0 0x5639d17162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.957+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/963608822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 0x5639d170e5b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.957+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/963608822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170f050 0x5639d170f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.957+0000 7fc1676c0b80 1 -- 172.21.15.150:0/963608822 >> 172.21.15.150:0/963608822 conn(0x5639d1670a00 msgr2=0x5639d1708a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.957+0000 7fc1676c0b80 1 -- 172.21.15.150:0/963608822 shutdown_connections 2024-05-10T04:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.957+0000 7fc1676c0b80 1 -- 172.21.15.150:0/963608822 wait complete. 2024-05-10T04:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.957+0000 7fc1676c0b80 1 -- start start 2024-05-10T04:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.957+0000 7fc1676c0b80 1 Processor -- start 2024-05-10T04:04:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc1676c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170e180 0x5639d1717e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc1676c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 0x5639d1718390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc1676c0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 0x5639d171f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc1676c0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d15ca500 con 0x5639d170f9a0 2024-05-10T04:04:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc1676c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d15ca1a0 con 0x5639d170f050 2024-05-10T04:04:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc1676c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d14b1640 con 0x5639d170e180 2024-05-10T04:04:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc1676c0b80 10 client.0 _ll_register_callbacks cb 0x5639d15d3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc155919700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 0x5639d171f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc155919700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 0x5639d171f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40000/0 (socket says 172.21.15.150:40000) 2024-05-10T04:04:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc155919700 1 -- 172.21.15.150:0/226834143 learned_addr learned my addr 172.21.15.150:0/226834143 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc155118700 1 --2- 172.21.15.150:0/226834143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170e180 0x5639d1717e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc154917700 1 --2- 172.21.15.150:0/226834143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 0x5639d1718390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc155919700 1 -- 172.21.15.150:0/226834143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170e180 msgr2=0x5639d1717e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc155919700 1 --2- 172.21.15.150:0/226834143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170e180 0x5639d1717e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc155919700 1 -- 172.21.15.150:0/226834143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 msgr2=0x5639d1718390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc155919700 1 --2- 172.21.15.150:0/226834143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 0x5639d1718390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.958+0000 7fc155919700 1 -- 172.21.15.150:0/226834143 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1501a6970 con 0x5639d170f9a0 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc155919700 1 --2- 172.21.15.150:0/226834143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 0x5639d171f2f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc148007b90 tx=0x7fc148009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4bc03b254474d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc138ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc138ff9700 1 -- 172.21.15.150:0/226834143 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148011890 con 0x5639d170f9a0 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc1676c0b80 1 -- 172.21.15.150:0/226834143 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639d1587210 con 0x5639d170f9a0 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc138ff9700 1 -- 172.21.15.150:0/226834143 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc1480119f0 con 0x5639d170f9a0 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc138ff9700 1 -- 172.21.15.150:0/226834143 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc148011bb0 con 0x5639d170f9a0 2024-05-10T04:04:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc1676c0b80 10 client.15816 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc1676c0b80 1 -- 172.21.15.150:0/226834143 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5639d145bb00 con 0x5639d170f9a0 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc138ff9700 1 -- 172.21.15.150:0/226834143 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639d1587210 con 0x5639d170f9a0 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc138ff9700 1 -- 172.21.15.150:0/226834143 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc14801d060 con 0x5639d170f9a0 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc1676c0b80 10 client.15816 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc1676c0b80 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc1676c0b80 20 client.15816 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/226834143 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639d1723aa0 0x5639d1725ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc1676c0b80 4 client.15816 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.959+0000 7fc1676c0b80 1 -- 172.21.15.150:0/226834143 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639d16dd720 con 0x5639d1723aa0 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.960+0000 7fc155118700 1 --2- 172.21.15.150:0/226834143 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639d1723aa0 0x5639d1725ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.960+0000 7fc138ff9700 10 client.15816 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.960+0000 7fc155118700 1 --2- 172.21.15.150:0/226834143 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639d1723aa0 0x5639d1725ef0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d19fe396e9b17629 server_cookie=14099d6e365763a6 in_seq=0 out_seq=0 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc138ff9700 1 -- 172.21.15.150:0/226834143 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639d16dd720 con 0x5639d1723aa0 2024-05-10T04:04:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc138ff9700 10 client.15816 handle_command_reply: tid=0 2024-05-10T04:04:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc1676c0b80 1 client.15816 shutdown 2024-05-10T04:04:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc1676c0b80 1 -- 172.21.15.150:0/226834143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 msgr2=0x5639d171f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/226834143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 0x5639d171f2f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc148007b90 tx=0x7fc148009510 comp rx=0 tx=0).stop 2024-05-10T04:04:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc1676c0b80 1 -- 172.21.15.150:0/226834143 shutdown_connections 2024-05-10T04:04:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/226834143 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639d1723aa0 0x5639d1725ef0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/226834143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 0x5639d171f2f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/226834143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170e180 0x5639d1717e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc138ff9700 0 client.15816 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/226834143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 0x5639d1718390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc1676c0b80 1 -- 172.21.15.150:0/226834143 >> 172.21.15.150:0/226834143 conn(0x5639d1670a00 msgr2=0x5639d1708a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc1676c0b80 1 -- 172.21.15.150:0/226834143 shutdown_connections 2024-05-10T04:04:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc1676c0b80 1 -- 172.21.15.150:0/226834143 wait complete. 2024-05-10T04:04:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.961+0000 7fc1676c0b80 20 client.15816 trim_cache size 0 max 16384 2024-05-10T04:04:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.976+0000 7fc1676c0b80 1 Processor -- start 2024-05-10T04:04:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.976+0000 7fc1676c0b80 1 -- start start 2024-05-10T04:04:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.976+0000 7fc1676c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170e180 0x5639d171d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.976+0000 7fc1676c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 0x5639d171d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.976+0000 7fc1676c0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 0x5639d1716400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.976+0000 7fc1676c0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d1713400 con 0x5639d170f9a0 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.976+0000 7fc1676c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d1703360 con 0x5639d170f050 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.976+0000 7fc1676c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d170ee60 con 0x5639d170e180 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.976+0000 7fc155919700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 0x5639d1716400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.976+0000 7fc155919700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 0x5639d1716400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40016/0 (socket says 172.21.15.150:40016) 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.976+0000 7fc155919700 1 -- 172.21.15.150:0/2566540607 learned_addr learned my addr 172.21.15.150:0/2566540607 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.976+0000 7fc154917700 1 --2- 172.21.15.150:0/2566540607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 0x5639d171d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.977+0000 7fc155919700 1 -- 172.21.15.150:0/2566540607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170e180 msgr2=0x5639d171d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.977+0000 7fc155118700 1 --2- 172.21.15.150:0/2566540607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170e180 0x5639d171d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.977+0000 7fc155919700 1 --2- 172.21.15.150:0/2566540607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170e180 0x5639d171d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.977+0000 7fc155919700 1 -- 172.21.15.150:0/2566540607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 msgr2=0x5639d171d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.977+0000 7fc155919700 1 --2- 172.21.15.150:0/2566540607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 0x5639d171d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.977+0000 7fc155919700 1 -- 172.21.15.150:0/2566540607 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639d15f4500 con 0x5639d170f9a0 2024-05-10T04:04:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.977+0000 7fc154917700 1 --2- 172.21.15.150:0/2566540607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 0x5639d171d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.977+0000 7fc155919700 1 --2- 172.21.15.150:0/2566540607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 0x5639d1716400 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc14801f4a0 tx=0x7fc148006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117f815fce0fd95a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.977+0000 7fc13bfff700 1 -- 172.21.15.150:0/2566540607 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148014550 con 0x5639d170f9a0 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.977+0000 7fc13bfff700 1 -- 172.21.15.150:0/2566540607 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc14800f050 con 0x5639d170f9a0 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.977+0000 7fc13bfff700 1 -- 172.21.15.150:0/2566540607 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148011d30 con 0x5639d170f9a0 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.977+0000 7fc1676c0b80 1 -- 172.21.15.150:0/2566540607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 msgr2=0x5639d1716400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.977+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/2566540607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 0x5639d1716400 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc14801f4a0 tx=0x7fc148006000 comp rx=0 tx=0).stop 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 -- 172.21.15.150:0/2566540607 shutdown_connections 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/2566540607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170f9a0 0x5639d1716400 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/2566540607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170e180 0x5639d171d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/2566540607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 0x5639d171d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 -- 172.21.15.150:0/2566540607 >> 172.21.15.150:0/2566540607 conn(0x5639d1670a00 msgr2=0x5639d1712cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 -- 172.21.15.150:0/2566540607 shutdown_connections 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 -- 172.21.15.150:0/2566540607 wait complete. 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 -- start start 2024-05-10T04:04:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 Processor -- start 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 0x5639d171d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 0x5639d1775f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170f9a0 0x5639d1778570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d1778be0 con 0x5639d170e180 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d1778d50 con 0x5639d170f050 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d1778f10 con 0x5639d170f9a0 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc1676c0b80 10 client.0 _ll_register_callbacks cb 0x5639d15d3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.978+0000 7fc155118700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 0x5639d171d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc155118700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 0x5639d171d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40026/0 (socket says 172.21.15.150:40026) 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc155118700 1 -- 172.21.15.150:0/2028923917 learned_addr learned my addr 172.21.15.150:0/2028923917 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc154917700 1 --2- 172.21.15.150:0/2028923917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 0x5639d1775f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc155919700 1 --2- 172.21.15.150:0/2028923917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170f9a0 0x5639d1778570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc155118700 1 -- 172.21.15.150:0/2028923917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170f9a0 msgr2=0x5639d1778570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc155118700 1 --2- 172.21.15.150:0/2028923917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170f9a0 0x5639d1778570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc155118700 1 -- 172.21.15.150:0/2028923917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 msgr2=0x5639d1775f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:04:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc155118700 1 --2- 172.21.15.150:0/2028923917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 0x5639d1775f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc155118700 1 -- 172.21.15.150:0/2028923917 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639d145bb00 con 0x5639d170e180 2024-05-10T04:04:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc155118700 1 --2- 172.21.15.150:0/2028923917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 0x5639d171d650 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc14c09b240 tx=0x7fc14c021c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04add6ee4f6e32c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:04:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc138ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:04:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc138ff9700 1 -- 172.21.15.150:0/2028923917 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c09b2b0 con 0x5639d170e180 2024-05-10T04:04:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc138ff9700 1 -- 172.21.15.150:0/2028923917 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc14c0a36a0 con 0x5639d170e180 2024-05-10T04:04:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc1676c0b80 1 -- 172.21.15.150:0/2028923917 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc14c09f0e0 con 0x5639d170e180 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc138ff9700 1 -- 172.21.15.150:0/2028923917 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc14c099650 con 0x5639d170e180 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc1676c0b80 10 client.15822 fetch_fsmap learned FSMap version 25 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc1676c0b80 1 -- 172.21.15.150:0/2028923917 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5639d14b3c70 con 0x5639d170e180 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.979+0000 7fc138ff9700 1 -- 172.21.15.150:0/2028923917 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc14c09f0e0 con 0x5639d170e180 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.980+0000 7fc138ff9700 1 -- 172.21.15.150:0/2028923917 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc14c064020 con 0x5639d170e180 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.980+0000 7fc1676c0b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.980+0000 7fc1676c0b80 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.980+0000 7fc1676c0b80 20 client.15822 populate_metadata read hostname 'smithi150' 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.980+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/2028923917 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639d177a400 0x5639d177c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.980+0000 7fc1676c0b80 4 client.15822 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.980+0000 7fc1676c0b80 1 -- 172.21.15.150:0/2028923917 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5639d1626e60 con 0x5639d177a400 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.980+0000 7fc154917700 1 --2- 172.21.15.150:0/2028923917 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639d177a400 0x5639d177c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.980+0000 7fc138ff9700 10 client.15822 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.980+0000 7fc154917700 1 --2- 172.21.15.150:0/2028923917 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639d177a400 0x5639d177c820 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0091965eaa2f92f server_cookie=6b69fdf549997e3e in_seq=0 out_seq=0 2024-05-10T04:04:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.987+0000 7fc138ff9700 1 -- 172.21.15.150:0/2028923917 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5639d1626e60 con 0x5639d177a400 2024-05-10T04:04:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.987+0000 7fc138ff9700 10 client.15822 handle_command_reply: tid=0 2024-05-10T04:04:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.987+0000 7fc1676c0b80 1 client.15822 shutdown 2024-05-10T04:04:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc1676c0b80 1 -- 172.21.15.150:0/2028923917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 msgr2=0x5639d171d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/2028923917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 0x5639d171d650 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc14c09b240 tx=0x7fc14c021c00 comp rx=0 tx=0).stop 2024-05-10T04:04:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc1676c0b80 1 -- 172.21.15.150:0/2028923917 shutdown_connections 2024-05-10T04:04:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/2028923917 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639d177a400 0x5639d177c820 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/2028923917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639d170f9a0 0x5639d1778570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/2028923917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639d170e180 0x5639d171d650 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc138ff9700 0 client.15822 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:04:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc1676c0b80 1 --2- 172.21.15.150:0/2028923917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639d170f050 0x5639d1775f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc1676c0b80 1 -- 172.21.15.150:0/2028923917 >> 172.21.15.150:0/2028923917 conn(0x5639d1670a00 msgr2=0x5639d1712cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc1676c0b80 1 -- 172.21.15.150:0/2028923917 shutdown_connections 2024-05-10T04:04:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc1676c0b80 1 -- 172.21.15.150:0/2028923917 wait complete. 2024-05-10T04:04:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc1676c0b80 20 client.15822 trim_cache size 0 max 16384 2024-05-10T04:04:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:04:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:04:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:04:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:04:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc15737c700 1 -- 172.21.15.150:0/726686659 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc13c0a11b0 msgr2=0x7fc13c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc15737c700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc13c0a11b0 0x7fc13c0a36c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc15019f690 tx=0x7fc14000a380 comp rx=0 tx=0).stop 2024-05-10T04:04:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc15737c700 1 -- 172.21.15.150:0/726686659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1501004d0 msgr2=0x7fc15019e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:04:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc15737c700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1501004d0 0x7fc15019e5f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc14c005740 tx=0x7fc14c009580 comp rx=0 tx=0).stop 2024-05-10T04:04:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc15737c700 1 -- 172.21.15.150:0/726686659 shutdown_connections 2024-05-10T04:04:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc15737c700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc13c0a11b0 0x7fc13c0a36c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc15737c700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc150105790 0x7fc1501a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc15737c700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1501004d0 0x7fc15019e5f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc15737c700 1 --2- 172.21.15.150:0/726686659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc150102e30 0x7fc15019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:04:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.988+0000 7fc15737c700 1 -- 172.21.15.150:0/726686659 >> 172.21.15.150:0/726686659 conn(0x7fc1500fc100 msgr2=0x7fc150100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:04:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.989+0000 7fc15737c700 1 -- 172.21.15.150:0/726686659 shutdown_connections 2024-05-10T04:04:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:04:51.989+0000 7fc15737c700 1 -- 172.21.15.150:0/726686659 wait complete. 2024-05-10T04:04:54.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:54 smithi150 ceph-mon[42477]: pgmap v299: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-10T04:04:54.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:54 smithi181 ceph-mon[46602]: pgmap v299: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-10T04:04:54.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:54 smithi161 ceph-mon[46918]: pgmap v299: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-10T04:04:56.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:56 smithi150 ceph-mon[42477]: pgmap v300: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-10T04:04:56.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:56 smithi181 ceph-mon[46602]: pgmap v300: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-10T04:04:56.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:56 smithi161 ceph-mon[46918]: pgmap v300: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-10T04:04:58.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:04:58 smithi150 ceph-mon[42477]: pgmap v301: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T04:04:58.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:04:58 smithi181 ceph-mon[46602]: pgmap v301: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T04:04:58.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:04:58 smithi161 ceph-mon[46918]: pgmap v301: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T04:05:00.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:00 smithi150 ceph-mon[42477]: pgmap v302: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-10T04:05:00.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:00 smithi181 ceph-mon[46602]: pgmap v302: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-10T04:05:00.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:00 smithi161 ceph-mon[46918]: pgmap v302: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-10T04:05:02.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:02 smithi150 ceph-mon[42477]: pgmap v303: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T04:05:02.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:02 smithi181 ceph-mon[46602]: pgmap v303: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T04:05:02.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:02 smithi161 ceph-mon[46918]: pgmap v303: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T04:05:04.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:04 smithi150 ceph-mon[42477]: pgmap v304: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-10T04:05:04.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:04 smithi181 ceph-mon[46602]: pgmap v304: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-10T04:05:04.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:04 smithi161 ceph-mon[46918]: pgmap v304: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-10T04:05:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.111+0000 7f3f0b99a700 1 -- 172.21.15.150:0/3179526472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041013c0 msgr2=0x7f3f041017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.111+0000 7f3f0b99a700 1 --2- 172.21.15.150:0/3179526472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041013c0 0x7f3f041017d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400aaa0 tx=0x7f3ef400adb0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.111+0000 7f3f0b99a700 1 -- 172.21.15.150:0/3179526472 shutdown_connections 2024-05-10T04:05:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.111+0000 7f3f0b99a700 1 --2- 172.21.15.150:0/3179526472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f04101d60 0x7f3f04108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.111+0000 7f3f0b99a700 1 --2- 172.21.15.150:0/3179526472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f041002a0 0x7f3f041006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.111+0000 7f3f0b99a700 1 --2- 172.21.15.150:0/3179526472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041013c0 0x7f3f041017d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.111+0000 7f3f0b99a700 1 -- 172.21.15.150:0/3179526472 >> 172.21.15.150:0/3179526472 conn(0x7f3f040fb8d0 msgr2=0x7f3f040fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.111+0000 7f3f0b99a700 1 -- 172.21.15.150:0/3179526472 shutdown_connections 2024-05-10T04:05:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.111+0000 7f3f0b99a700 1 -- 172.21.15.150:0/3179526472 wait complete. 2024-05-10T04:05:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.112+0000 7f3f0b99a700 1 Processor -- start 2024-05-10T04:05:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.112+0000 7f3f0b99a700 1 -- start start 2024-05-10T04:05:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f0b99a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041002a0 0x7f3f04195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f0b99a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f041013c0 0x7f3f04196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f0b99a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f04101d60 0x7f3f0419d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f0b99a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0419d860 con 0x7f3f041002a0 2024-05-10T04:05:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f0b99a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0419d9a0 con 0x7f3f041013c0 2024-05-10T04:05:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f0b99a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0419db60 con 0x7f3f04101d60 2024-05-10T04:05:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f09736700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041002a0 0x7f3f04195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f09736700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041002a0 0x7f3f04195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40338/0 (socket says 172.21.15.150:40338) 2024-05-10T04:05:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f09736700 1 -- 172.21.15.150:0/660478985 learned_addr learned my addr 172.21.15.150:0/660478985 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f09f37700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f04101d60 0x7f3f0419d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f08f35700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f041013c0 0x7f3f04196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f09736700 1 -- 172.21.15.150:0/660478985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f04101d60 msgr2=0x7f3f0419d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f09736700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f04101d60 0x7f3f0419d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f09736700 1 -- 172.21.15.150:0/660478985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f041013c0 msgr2=0x7f3f04196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f09736700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f041013c0 0x7f3f04196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f09736700 1 -- 172.21.15.150:0/660478985 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef400a700 con 0x7f3f041002a0 2024-05-10T04:05:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f09736700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041002a0 0x7f3f04195e00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3f000098c0 tx=0x7f3f00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3efa7fc700 1 -- 172.21.15.150:0/660478985 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00012070 con 0x7f3f041002a0 2024-05-10T04:05:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3efa7fc700 1 -- 172.21.15.150:0/660478985 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f000044e0 con 0x7f3f041002a0 2024-05-10T04:05:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3f0b99a700 1 -- 172.21.15.150:0/660478985 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0419de40 con 0x7f3f041002a0 2024-05-10T04:05:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.113+0000 7f3efa7fc700 1 -- 172.21.15.150:0/660478985 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00003d80 con 0x7f3f041002a0 2024-05-10T04:05:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.114+0000 7f3f0b99a700 1 -- 172.21.15.150:0/660478985 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0419e270 con 0x7f3f041002a0 2024-05-10T04:05:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.117+0000 7f3efa7fc700 1 -- 172.21.15.150:0/660478985 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3f00016070 con 0x7f3f041002a0 2024-05-10T04:05:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.117+0000 7f3f1bcdeb80 1 Processor -- start 2024-05-10T04:05:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.117+0000 7f3efa7fc700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3ef00a11b0 0x7f3ef00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.117+0000 7f3f1bcdeb80 1 -- start start 2024-05-10T04:05:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3efa7fc700 1 -- 172.21.15.150:0/660478985 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3f000cdd60 con 0x7f3f041002a0 2024-05-10T04:05:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f09f37700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3ef00a11b0 0x7f3ef00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f1bcdeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254c180 0x564b3254c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f1bcdeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d050 0x564b3254d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f1bcdeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254d9a0 0x564b325542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f1bcdeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b324730f0 con 0x564b3254c180 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f1bcdeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b32470ab0 con 0x564b3254d050 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f1bcdeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3246c3a0 con 0x564b3254d9a0 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f09736700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254c180 0x564b3254c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f09736700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254c180 0x564b3254c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40344/0 (socket says 172.21.15.150:40344) 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f09736700 1 -- 172.21.15.150:0/2232721794 learned_addr learned my addr 172.21.15.150:0/2232721794 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f08f35700 1 --2- 172.21.15.150:0/2232721794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d050 0x564b3254d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f09f37700 1 --2- 172.21.15.150:0/2232721794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254d9a0 0x564b325542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f09736700 1 -- 172.21.15.150:0/2232721794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254d9a0 msgr2=0x564b325542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f09736700 1 --2- 172.21.15.150:0/2232721794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254d9a0 0x564b325542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f09736700 1 -- 172.21.15.150:0/2232721794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d050 msgr2=0x564b3254d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f09736700 1 --2- 172.21.15.150:0/2232721794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d050 0x564b3254d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f09736700 1 -- 172.21.15.150:0/2232721794 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b322ed350 con 0x564b3254c180 2024-05-10T04:05:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f09f37700 1 --2- 172.21.15.150:0/2232721794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254d9a0 0x564b325542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f09f37700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3ef00a11b0 0x7f3ef00a36c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3efc003f20 tx=0x7f3efc009ba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.118+0000 7f3f09736700 1 --2- 172.21.15.150:0/2232721794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254c180 0x564b3254c5b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3f00003860 tx=0x7f3f0009ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fc3f433b4adeb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.119+0000 7f3eef7fe700 1 -- 172.21.15.150:0/2232721794 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0009b440 con 0x564b3254c180 2024-05-10T04:05:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.119+0000 7f3eef7fe700 1 -- 172.21.15.150:0/2232721794 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f0009ad40 con 0x564b3254c180 2024-05-10T04:05:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.119+0000 7f3eef7fe700 1 -- 172.21.15.150:0/2232721794 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0009a230 con 0x564b3254c180 2024-05-10T04:05:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.119+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/2232721794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254c180 msgr2=0x564b3254c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.119+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/2232721794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254c180 0x564b3254c5b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3f00003860 tx=0x7f3f0009ac40 comp rx=0 tx=0).stop 2024-05-10T04:05:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.119+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/2232721794 shutdown_connections 2024-05-10T04:05:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.119+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/2232721794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254d9a0 0x564b325542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.119+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/2232721794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254c180 0x564b3254c5b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.119+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/2232721794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d050 0x564b3254d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.119+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/2232721794 >> 172.21.15.150:0/2232721794 conn(0x564b324aea00 msgr2=0x564b32546a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.120+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/2232721794 shutdown_connections 2024-05-10T04:05:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.120+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/2232721794 wait complete. 2024-05-10T04:05:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.120+0000 7f3f1bcdeb80 1 -- start start 2024-05-10T04:05:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.120+0000 7f3f1bcdeb80 1 Processor -- start 2024-05-10T04:05:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.120+0000 7f3f1bcdeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254c180 0x564b3255a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.120+0000 7f3f1bcdeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 0x564b3255ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.120+0000 7f3f1bcdeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d9a0 0x564b32561a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f1bcdeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b32408500 con 0x564b3254d050 2024-05-10T04:05:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f1bcdeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b324081a0 con 0x564b3254d9a0 2024-05-10T04:05:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f1bcdeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b322ef640 con 0x564b3254c180 2024-05-10T04:05:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f1bcdeb80 10 client.0 _ll_register_callbacks cb 0x564b32411ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f08f35700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 0x564b3255ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f08f35700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 0x564b3255ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40360/0 (socket says 172.21.15.150:40360) 2024-05-10T04:05:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f08f35700 1 -- 172.21.15.150:0/1938574209 learned_addr learned my addr 172.21.15.150:0/1938574209 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f09736700 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254c180 0x564b3255a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f09f37700 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d9a0 0x564b32561a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f08f35700 1 -- 172.21.15.150:0/1938574209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254c180 msgr2=0x564b3255a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f08f35700 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254c180 0x564b3255a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f08f35700 1 -- 172.21.15.150:0/1938574209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d9a0 msgr2=0x564b32561a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f08f35700 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d9a0 0x564b32561a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f08f35700 1 -- 172.21.15.150:0/1938574209 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0419e270 con 0x564b3254d050 2024-05-10T04:05:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f09f37700 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d9a0 0x564b32561a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:05:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f08f35700 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 0x564b3255ab10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4003710 tx=0x7f3ef4003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf83e13bd87a321 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3eecff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3eecff9700 1 -- 172.21.15.150:0/1938574209 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4003a10 con 0x564b3254d050 2024-05-10T04:05:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3eecff9700 1 -- 172.21.15.150:0/1938574209 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3ef4012070 con 0x564b3254d050 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.121+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/1938574209 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b323c5210 con 0x564b3254d050 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.122+0000 7f3eecff9700 1 -- 172.21.15.150:0/1938574209 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ef40050b0 con 0x564b3254d050 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.122+0000 7f3f1bcdeb80 10 client.15834 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.122+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/1938574209 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564b32299b00 con 0x564b3254d050 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.122+0000 7f3eecff9700 1 -- 172.21.15.150:0/1938574209 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564b323c5210 con 0x564b3254d050 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.122+0000 7f3eecff9700 1 -- 172.21.15.150:0/1938574209 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ef4021b10 con 0x564b3254d050 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.122+0000 7f3f1bcdeb80 10 client.15834 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.122+0000 7f3f1bcdeb80 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.122+0000 7f3f1bcdeb80 20 client.15834 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.122+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564b32566250 0x564b325686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.122+0000 7f3f1bcdeb80 4 client.15834 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.122+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/1938574209 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b3251b720 con 0x564b32566250 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.122+0000 7f3f09736700 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564b32566250 0x564b325686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.123+0000 7f3eecff9700 10 client.15834 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.123+0000 7f3f09736700 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564b32566250 0x564b325686a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17301bfc526d6c6 server_cookie=e3f86ba584e88a51 in_seq=0 out_seq=0 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.123+0000 7f3eecff9700 1 -- 172.21.15.150:0/1938574209 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564b3251b720 con 0x564b32566250 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.123+0000 7f3eecff9700 10 client.15834 handle_command_reply: tid=0 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.123+0000 7f3f1bcdeb80 1 client.15834 shutdown 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.124+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/1938574209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 msgr2=0x564b3255ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.124+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 0x564b3255ab10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4003710 tx=0x7f3ef4003740 comp rx=0 tx=0).stop 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.124+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/1938574209 shutdown_connections 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.124+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564b32566250 0x564b325686a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.124+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d9a0 0x564b32561a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.124+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254c180 0x564b3255a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.124+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/1938574209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 0x564b3255ab10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.124+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/1938574209 >> 172.21.15.150:0/1938574209 conn(0x564b324aea00 msgr2=0x564b32546a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.124+0000 7f3eecff9700 0 client.15834 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.124+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/1938574209 shutdown_connections 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.124+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/1938574209 wait complete. 2024-05-10T04:05:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.124+0000 7f3f1bcdeb80 20 client.15834 trim_cache size 0 max 16384 2024-05-10T04:05:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.139+0000 7f3f1bcdeb80 1 Processor -- start 2024-05-10T04:05:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.139+0000 7f3f1bcdeb80 1 -- start start 2024-05-10T04:05:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.139+0000 7f3f1bcdeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254c180 0x564b3255f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f1bcdeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254d050 0x564b3255ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f1bcdeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d9a0 0x564b32550e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f1bcdeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3254ca10 con 0x564b3254d9a0 2024-05-10T04:05:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f1bcdeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3255b420 con 0x564b3254c180 2024-05-10T04:05:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f1bcdeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b325476c0 con 0x564b3254d050 2024-05-10T04:05:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f09f37700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d9a0 0x564b32550e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f09f37700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d9a0 0x564b32550e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40366/0 (socket says 172.21.15.150:40366) 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f09f37700 1 -- 172.21.15.150:0/3127481049 learned_addr learned my addr 172.21.15.150:0/3127481049 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f09736700 1 --2- 172.21.15.150:0/3127481049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254c180 0x564b3255f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f08f35700 1 --2- 172.21.15.150:0/3127481049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254d050 0x564b3255ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f09f37700 1 -- 172.21.15.150:0/3127481049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254d050 msgr2=0x564b3255ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f09f37700 1 --2- 172.21.15.150:0/3127481049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254d050 0x564b3255ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f09f37700 1 -- 172.21.15.150:0/3127481049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254c180 msgr2=0x564b3255f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f09f37700 1 --2- 172.21.15.150:0/3127481049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254c180 0x564b3255f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f09f37700 1 -- 172.21.15.150:0/3127481049 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b32432500 con 0x564b3254d9a0 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f09736700 1 --2- 172.21.15.150:0/3127481049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254c180 0x564b3255f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3f09f37700 1 --2- 172.21.15.150:0/3127481049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d9a0 0x564b32550e50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3efc004920 tx=0x7f3efc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796790cf970bcf41 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.140+0000 7f3eeffff700 1 -- 172.21.15.150:0/3127481049 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc023620 con 0x564b3254d9a0 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.141+0000 7f3eeffff700 1 -- 172.21.15.150:0/3127481049 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3efc00f040 con 0x564b3254d9a0 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.141+0000 7f3eeffff700 1 -- 172.21.15.150:0/3127481049 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc0189d0 con 0x564b3254d9a0 2024-05-10T04:05:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.141+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/3127481049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d9a0 msgr2=0x564b32550e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.141+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/3127481049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d9a0 0x564b32550e50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3efc004920 tx=0x7f3efc015000 comp rx=0 tx=0).stop 2024-05-10T04:05:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.141+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/3127481049 shutdown_connections 2024-05-10T04:05:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.141+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/3127481049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d9a0 0x564b32550e50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.141+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/3127481049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254c180 0x564b3255f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.141+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/3127481049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254d050 0x564b3255ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.141+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/3127481049 >> 172.21.15.150:0/3127481049 conn(0x564b324aea00 msgr2=0x564b3255a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.142+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/3127481049 shutdown_connections 2024-05-10T04:05:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.142+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/3127481049 wait complete. 2024-05-10T04:05:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.142+0000 7f3f1bcdeb80 1 -- start start 2024-05-10T04:05:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.142+0000 7f3f1bcdeb80 1 Processor -- start 2024-05-10T04:05:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.142+0000 7f3f1bcdeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254c180 0x564b3255efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.142+0000 7f3f1bcdeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 0x564b3255f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.142+0000 7f3f1bcdeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d9a0 0x564b325ba710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.142+0000 7f3f1bcdeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b325bac50 con 0x564b3254d050 2024-05-10T04:05:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.142+0000 7f3f1bcdeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b325bad90 con 0x564b3254d9a0 2024-05-10T04:05:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.142+0000 7f3f1bcdeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b325baf20 con 0x564b3254c180 2024-05-10T04:05:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.142+0000 7f3f1bcdeb80 10 client.0 _ll_register_callbacks cb 0x564b32411ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f08f35700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 0x564b3255f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f08f35700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 0x564b3255f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40374/0 (socket says 172.21.15.150:40374) 2024-05-10T04:05:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f08f35700 1 -- 172.21.15.150:0/332588455 learned_addr learned my addr 172.21.15.150:0/332588455 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f09f37700 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d9a0 0x564b325ba710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f09736700 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254c180 0x564b3255efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f08f35700 1 -- 172.21.15.150:0/332588455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254c180 msgr2=0x564b3255efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f08f35700 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254c180 0x564b3255efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f08f35700 1 -- 172.21.15.150:0/332588455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d9a0 msgr2=0x564b325ba710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f08f35700 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d9a0 0x564b325ba710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f08f35700 1 -- 172.21.15.150:0/332588455 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b322ed350 con 0x564b3254d050 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f09736700 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254c180 0x564b3255efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f08f35700 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 0x564b3255f520 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4005f50 tx=0x7f3ef400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11765c179d56f8db server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3eecff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3eecff9700 1 -- 172.21.15.150:0/332588455 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4039070 con 0x564b3254d050 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3eecff9700 1 -- 172.21.15.150:0/332588455 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3ef4012070 con 0x564b3254d050 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/332588455 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f0009ad40 con 0x564b3254d050 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3eecff9700 1 -- 172.21.15.150:0/332588455 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ef400ea60 con 0x564b3254d050 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f1bcdeb80 10 client.15840 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.143+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/332588455 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f3efc004520 con 0x564b3254d050 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.144+0000 7f3eecff9700 1 -- 172.21.15.150:0/332588455 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f0009ad40 con 0x564b3254d050 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.144+0000 7f3eecff9700 1 -- 172.21.15.150:0/332588455 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ef403c020 con 0x564b3254d050 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.144+0000 7f3f1bcdeb80 10 client.15840 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.144+0000 7f3f1bcdeb80 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.144+0000 7f3f1bcdeb80 20 client.15840 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.144+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564b325bc450 0x564b325be870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.144+0000 7f3f1bcdeb80 4 client.15840 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:05:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.144+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/332588455 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564b32464e60 con 0x564b325bc450 2024-05-10T04:05:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.144+0000 7f3f09736700 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564b325bc450 0x564b325be870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.145+0000 7f3eecff9700 10 client.15840 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.145+0000 7f3f09736700 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564b325bc450 0x564b325be870 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84513aed50a4a2c6 server_cookie=5a4d6ef4ba2ad50c in_seq=0 out_seq=0 2024-05-10T04:05:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3eecff9700 1 -- 172.21.15.150:0/332588455 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564b32464e60 con 0x564b325bc450 2024-05-10T04:05:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3eecff9700 10 client.15840 handle_command_reply: tid=0 2024-05-10T04:05:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3f1bcdeb80 1 client.15840 shutdown 2024-05-10T04:05:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/332588455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 msgr2=0x564b3255f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 0x564b3255f520 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4005f50 tx=0x7f3ef400e940 comp rx=0 tx=0).stop 2024-05-10T04:05:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/332588455 shutdown_connections 2024-05-10T04:05:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564b325bc450 0x564b325be870 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564b3254d9a0 0x564b325ba710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564b3254c180 0x564b3255efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3eecff9700 0 client.15840 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3f1bcdeb80 1 --2- 172.21.15.150:0/332588455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b3254d050 0x564b3255f520 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/332588455 >> 172.21.15.150:0/332588455 conn(0x564b324aea00 msgr2=0x564b3255a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/332588455 shutdown_connections 2024-05-10T04:05:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3f1bcdeb80 1 -- 172.21.15.150:0/332588455 wait complete. 2024-05-10T04:05:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.149+0000 7f3f1bcdeb80 20 client.15840 trim_cache size 0 max 16384 2024-05-10T04:05:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:05:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:05:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:05:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:05:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.150+0000 7f3f0b99a700 1 -- 172.21.15.150:0/660478985 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3ef00a11b0 msgr2=0x7f3ef00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.150+0000 7f3f0b99a700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3ef00a11b0 0x7f3ef00a36c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3efc003f20 tx=0x7f3efc009ba0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.150+0000 7f3f0b99a700 1 -- 172.21.15.150:0/660478985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041002a0 msgr2=0x7f3f04195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.150+0000 7f3f0b99a700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041002a0 0x7f3f04195e00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3f000098c0 tx=0x7f3f00009bd0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.150+0000 7f3f0b99a700 1 -- 172.21.15.150:0/660478985 shutdown_connections 2024-05-10T04:05:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.150+0000 7f3f0b99a700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3ef00a11b0 0x7f3ef00a36c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.150+0000 7f3f0b99a700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f04101d60 0x7f3f0419d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.150+0000 7f3f0b99a700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041002a0 0x7f3f04195e00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.150+0000 7f3f0b99a700 1 --2- 172.21.15.150:0/660478985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f041013c0 0x7f3f04196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.150+0000 7f3f0b99a700 1 -- 172.21.15.150:0/660478985 >> 172.21.15.150:0/660478985 conn(0x7f3f040fb8d0 msgr2=0x7f3f041045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.150+0000 7f3f0b99a700 1 -- 172.21.15.150:0/660478985 shutdown_connections 2024-05-10T04:05:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:05.150+0000 7f3f0b99a700 1 -- 172.21.15.150:0/660478985 wait complete. 2024-05-10T04:05:06.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:06 smithi150 ceph-mon[42477]: pgmap v305: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-10T04:05:06.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:06 smithi181 ceph-mon[46602]: pgmap v305: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-10T04:05:06.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:06 smithi161 ceph-mon[46918]: pgmap v305: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-10T04:05:08.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:08 smithi150 ceph-mon[42477]: pgmap v306: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-10T04:05:08.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:08 smithi181 ceph-mon[46602]: pgmap v306: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-10T04:05:08.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:08 smithi161 ceph-mon[46918]: pgmap v306: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-10T04:05:10.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:10 smithi150 ceph-mon[42477]: pgmap v307: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-10T04:05:10.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:10 smithi181 ceph-mon[46602]: pgmap v307: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-10T04:05:10.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:10 smithi161 ceph-mon[46918]: pgmap v307: 129 pgs: 129 active+clean; 668 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-10T04:05:12.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:12 smithi150 ceph-mon[42477]: pgmap v308: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T04:05:12.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:12 smithi181 ceph-mon[46602]: pgmap v308: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T04:05:12.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:12 smithi161 ceph-mon[46918]: pgmap v308: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T04:05:14.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:14 smithi150 ceph-mon[42477]: pgmap v309: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T04:05:14.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:14 smithi181 ceph-mon[46602]: pgmap v309: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T04:05:14.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:14 smithi161 ceph-mon[46918]: pgmap v309: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T04:05:15.371 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:05:15.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.522+0000 7f1c80d20700 1 -- 172.21.15.150:0/4232384414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c7c101c00 msgr2=0x7f1c7c102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:15.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.522+0000 7f1c80d20700 1 --2- 172.21.15.150:0/4232384414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c7c101c00 0x7f1c7c102010 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009990 tx=0x7f1c6c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.522+0000 7f1c80d20700 1 -- 172.21.15.150:0/4232384414 shutdown_connections 2024-05-10T04:05:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.522+0000 7f1c80d20700 1 --2- 172.21.15.150:0/4232384414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c7c1036c0 0x7f1c7c109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.522+0000 7f1c80d20700 1 --2- 172.21.15.150:0/4232384414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c7c101c00 0x7f1c7c102010 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.522+0000 7f1c80d20700 1 --2- 172.21.15.150:0/4232384414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c7c102d20 0x7f1c7c103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.522+0000 7f1c80d20700 1 -- 172.21.15.150:0/4232384414 >> 172.21.15.150:0/4232384414 conn(0x7f1c7c0fd210 msgr2=0x7f1c7c0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.522+0000 7f1c80d20700 1 -- 172.21.15.150:0/4232384414 shutdown_connections 2024-05-10T04:05:15.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.522+0000 7f1c80d20700 1 -- 172.21.15.150:0/4232384414 wait complete. 2024-05-10T04:05:15.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.523+0000 7f1c80d20700 1 Processor -- start 2024-05-10T04:05:15.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.523+0000 7f1c80d20700 1 -- start start 2024-05-10T04:05:15.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c80d20700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c7c101c00 0x7f1c7c19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c80d20700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c7c102d20 0x7f1c7c19ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c80d20700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c7c1036c0 0x7f1c7c1a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c80d20700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c1a5fe0 con 0x7f1c7c101c00 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c80d20700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c1a6120 con 0x7f1c7c102d20 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c80d20700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c1a62b0 con 0x7f1c7c1036c0 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c7a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c7c101c00 0x7f1c7c19e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c7a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c7c101c00 0x7f1c7c19e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50434/0 (socket says 172.21.15.150:50434) 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c7a59c700 1 -- 172.21.15.150:0/3755657547 learned_addr learned my addr 172.21.15.150:0/3755657547 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c79d9b700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c7c102d20 0x7f1c7c19ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c7ad9d700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c7c1036c0 0x7f1c7c1a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c7a59c700 1 -- 172.21.15.150:0/3755657547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c7c1036c0 msgr2=0x7f1c7c1a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c7a59c700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c7c1036c0 0x7f1c7c1a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c7a59c700 1 -- 172.21.15.150:0/3755657547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c7c102d20 msgr2=0x7f1c7c19ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c7a59c700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c7c102d20 0x7f1c7c19ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.524+0000 7f1c7a59c700 1 -- 172.21.15.150:0/3755657547 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c0095f0 con 0x7f1c7c101c00 2024-05-10T04:05:15.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.525+0000 7f1c7a59c700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c7c101c00 0x7f1c7c19e540 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c005740 tx=0x7f1c6c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.525+0000 7f1c6b7fe700 1 -- 172.21.15.150:0/3755657547 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c01d070 con 0x7f1c7c101c00 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.525+0000 7f1c6b7fe700 1 -- 172.21.15.150:0/3755657547 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c6c021410 con 0x7f1c7c101c00 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.525+0000 7f1c6b7fe700 1 -- 172.21.15.150:0/3755657547 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c017650 con 0x7f1c7c101c00 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.526+0000 7f1c80d20700 1 -- 172.21.15.150:0/3755657547 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7c1a6530 con 0x7f1c7c101c00 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.526+0000 7f1c80d20700 1 -- 172.21.15.150:0/3755657547 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c1a6900 con 0x7f1c7c101c00 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.529+0000 7f1c6b7fe700 1 -- 172.21.15.150:0/3755657547 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1c6c021580 con 0x7f1c7c101c00 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.529+0000 7f1c6b7fe700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1c640a12e0 0x7f1c640a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.530+0000 7f1c79d9b700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1c640a12e0 0x7f1c640a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.530+0000 7f1c6b7fe700 1 -- 172.21.15.150:0/3755657547 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1c6c0cde30 con 0x7f1c7c101c00 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.530+0000 7f1c79d9b700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1c640a12e0 0x7f1c640a37f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c19f5e0 tx=0x7f1c70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.531+0000 7f1c91064b80 1 Processor -- start 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.531+0000 7f1c91064b80 1 -- start start 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c91064b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a27946a0 0x55c3a2794af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c91064b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 0x55c3a2795980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c91064b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a2795ec0 0x55c3a279c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c91064b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a26bb150 con 0x55c3a2795570 2024-05-10T04:05:15.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c91064b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a26b8b10 con 0x55c3a27946a0 2024-05-10T04:05:15.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c91064b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a26b4400 con 0x55c3a2795ec0 2024-05-10T04:05:15.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c79d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 0x55c3a2795980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c79d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 0x55c3a2795980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50436/0 (socket says 172.21.15.150:50436) 2024-05-10T04:05:15.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c79d9b700 1 -- 172.21.15.150:0/3755202569 learned_addr learned my addr 172.21.15.150:0/3755202569 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:15.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c7a59c700 1 --2- 172.21.15.150:0/3755202569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a27946a0 0x55c3a2794af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c7ad9d700 1 --2- 172.21.15.150:0/3755202569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a2795ec0 0x55c3a279c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c79d9b700 1 -- 172.21.15.150:0/3755202569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a2795ec0 msgr2=0x55c3a279c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:15.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c79d9b700 1 --2- 172.21.15.150:0/3755202569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a2795ec0 0x55c3a279c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c79d9b700 1 -- 172.21.15.150:0/3755202569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a27946a0 msgr2=0x55c3a2794af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:15.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c79d9b700 1 --2- 172.21.15.150:0/3755202569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a27946a0 0x55c3a2794af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c79d9b700 1 -- 172.21.15.150:0/3755202569 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3a25353b0 con 0x55c3a2795570 2024-05-10T04:05:15.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.532+0000 7f1c7ad9d700 1 --2- 172.21.15.150:0/3755202569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a2795ec0 0x55c3a279c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:15.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.533+0000 7f1c79d9b700 1 --2- 172.21.15.150:0/3755202569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 0x55c3a2795980 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1c70011d50 tx=0x7f1c70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6506198d5315005 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:15.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.533+0000 7f1c68ff9700 1 -- 172.21.15.150:0/3755202569 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7000eb20 con 0x55c3a2795570 2024-05-10T04:05:15.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.533+0000 7f1c68ff9700 1 -- 172.21.15.150:0/3755202569 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c70021070 con 0x55c3a2795570 2024-05-10T04:05:15.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.533+0000 7f1c68ff9700 1 -- 172.21.15.150:0/3755202569 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70003c70 con 0x55c3a2795570 2024-05-10T04:05:15.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.534+0000 7f1c91064b80 1 -- 172.21.15.150:0/3755202569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 msgr2=0x55c3a2795980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:15.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.534+0000 7f1c91064b80 1 --2- 172.21.15.150:0/3755202569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 0x55c3a2795980 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1c70011d50 tx=0x7f1c70014000 comp rx=0 tx=0).stop 2024-05-10T04:05:15.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.534+0000 7f1c91064b80 1 -- 172.21.15.150:0/3755202569 shutdown_connections 2024-05-10T04:05:15.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.534+0000 7f1c91064b80 1 --2- 172.21.15.150:0/3755202569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a2795ec0 0x55c3a279c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.534+0000 7f1c91064b80 1 --2- 172.21.15.150:0/3755202569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a27946a0 0x55c3a2794af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.534+0000 7f1c91064b80 1 --2- 172.21.15.150:0/3755202569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 0x55c3a2795980 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.534+0000 7f1c91064b80 1 -- 172.21.15.150:0/3755202569 >> 172.21.15.150:0/3755202569 conn(0x55c3a26f6a60 msgr2=0x55c3a278efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:15.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.534+0000 7f1c91064b80 1 -- 172.21.15.150:0/3755202569 shutdown_connections 2024-05-10T04:05:15.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.534+0000 7f1c91064b80 1 -- 172.21.15.150:0/3755202569 wait complete. 2024-05-10T04:05:15.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.535+0000 7f1c91064b80 1 -- start start 2024-05-10T04:05:15.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.535+0000 7f1c91064b80 1 Processor -- start 2024-05-10T04:05:15.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.535+0000 7f1c91064b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a27946a0 0x55c3a27af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.535+0000 7f1c91064b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a2795570 0x55c3a27afc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.535+0000 7f1c91064b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 0x55c3a27b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.535+0000 7f1c91064b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a2650560 con 0x55c3a2795ec0 2024-05-10T04:05:15.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.535+0000 7f1c91064b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a2650200 con 0x55c3a2795570 2024-05-10T04:05:15.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.535+0000 7f1c91064b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a25376a0 con 0x55c3a27946a0 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.536+0000 7f1c91064b80 10 client.0 _ll_register_callbacks cb 0x55c3a2659b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.536+0000 7f1c7ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 0x55c3a27b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.536+0000 7f1c7ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 0x55c3a27b6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50446/0 (socket says 172.21.15.150:50446) 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.536+0000 7f1c7ad9d700 1 -- 172.21.15.150:0/463864895 learned_addr learned my addr 172.21.15.150:0/463864895 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.536+0000 7f1c7a59c700 1 --2- 172.21.15.150:0/463864895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a27946a0 0x55c3a27af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.536+0000 7f1c7ad9d700 1 -- 172.21.15.150:0/463864895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a27946a0 msgr2=0x55c3a27af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.536+0000 7f1c7ad9d700 1 --2- 172.21.15.150:0/463864895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a27946a0 0x55c3a27af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.536+0000 7f1c7ad9d700 1 -- 172.21.15.150:0/463864895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a2795570 msgr2=0x55c3a27afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.536+0000 7f1c7ad9d700 1 --2- 172.21.15.150:0/463864895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a2795570 0x55c3a27afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.536+0000 7f1c7ad9d700 1 -- 172.21.15.150:0/463864895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c70017070 con 0x55c3a2795ec0 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.536+0000 7f1c7a59c700 1 --2- 172.21.15.150:0/463864895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a27946a0 0x55c3a27af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.537+0000 7f1c7ad9d700 1 --2- 172.21.15.150:0/463864895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 0x55c3a27b6ba0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1c74011d70 tx=0x7f1c7400f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77a13ac015e83b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.537+0000 7f1c61ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.537+0000 7f1c61ffb700 1 -- 172.21.15.150:0/463864895 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7400fac0 con 0x55c3a2795ec0 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.537+0000 7f1c61ffb700 1 -- 172.21.15.150:0/463864895 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1c7400fc20 con 0x55c3a2795ec0 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.537+0000 7f1c91064b80 1 -- 172.21.15.150:0/463864895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3a260d270 con 0x55c3a2795ec0 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.537+0000 7f1c61ffb700 1 -- 172.21.15.150:0/463864895 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c7400fde0 con 0x55c3a2795ec0 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.537+0000 7f1c91064b80 10 client.15852 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.537+0000 7f1c91064b80 1 -- 172.21.15.150:0/463864895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c3a24e1b60 con 0x55c3a2795ec0 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.537+0000 7f1c61ffb700 1 -- 172.21.15.150:0/463864895 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3a260d270 con 0x55c3a2795ec0 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.538+0000 7f1c61ffb700 1 -- 172.21.15.150:0/463864895 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c740155f0 con 0x55c3a2795ec0 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.538+0000 7f1c91064b80 10 client.15852 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.538+0000 7f1c91064b80 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.538+0000 7f1c91064b80 20 client.15852 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.538+0000 7f1c91064b80 1 --2- 172.21.15.150:0/463864895 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c3a27bb4e0 0x55c3a27bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.538+0000 7f1c91064b80 4 client.15852 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.538+0000 7f1c91064b80 1 -- 172.21.15.150:0/463864895 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3a2763780 con 0x55c3a27bb4e0 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.539+0000 7f1c7a59c700 1 --2- 172.21.15.150:0/463864895 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c3a27bb4e0 0x55c3a27bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.539+0000 7f1c7a59c700 1 --2- 172.21.15.150:0/463864895 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c3a27bb4e0 0x55c3a27bd930 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=900af405be0bb962 server_cookie=8fdab3de7f484cc in_seq=0 out_seq=0 2024-05-10T04:05:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.539+0000 7f1c61ffb700 10 client.15852 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c61ffb700 1 -- 172.21.15.150:0/463864895 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3a2763780 con 0x55c3a27bb4e0 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c61ffb700 10 client.15852 handle_command_reply: tid=0 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c91064b80 1 client.15852 shutdown 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c91064b80 1 -- 172.21.15.150:0/463864895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 msgr2=0x55c3a27b6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c91064b80 1 --2- 172.21.15.150:0/463864895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 0x55c3a27b6ba0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1c74011d70 tx=0x7f1c7400f4b0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c91064b80 1 -- 172.21.15.150:0/463864895 shutdown_connections 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c91064b80 1 --2- 172.21.15.150:0/463864895 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c3a27bb4e0 0x55c3a27bd930 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c91064b80 1 --2- 172.21.15.150:0/463864895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 0x55c3a27b6ba0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c91064b80 1 --2- 172.21.15.150:0/463864895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a27946a0 0x55c3a27af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c91064b80 1 --2- 172.21.15.150:0/463864895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a2795570 0x55c3a27afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c61ffb700 0 client.15852 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c91064b80 1 -- 172.21.15.150:0/463864895 >> 172.21.15.150:0/463864895 conn(0x55c3a26f6a60 msgr2=0x55c3a278efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c91064b80 1 -- 172.21.15.150:0/463864895 shutdown_connections 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c91064b80 1 -- 172.21.15.150:0/463864895 wait complete. 2024-05-10T04:05:15.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.540+0000 7f1c91064b80 20 client.15852 trim_cache size 0 max 16384 2024-05-10T04:05:15.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.573+0000 7f1c91064b80 1 Processor -- start 2024-05-10T04:05:15.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.573+0000 7f1c91064b80 1 -- start start 2024-05-10T04:05:15.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.573+0000 7f1c91064b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a27946a0 0x55c3a27b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.573+0000 7f1c91064b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a2795570 0x55c3a27b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.573+0000 7f1c91064b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 0x55c3a27993b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.573+0000 7f1c91064b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a27bb160 con 0x55c3a2795ec0 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.573+0000 7f1c91064b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a279cac0 con 0x55c3a2795570 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.573+0000 7f1c91064b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a27bd900 con 0x55c3a27946a0 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.573+0000 7f1c7ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 0x55c3a27993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c7ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 0x55c3a27993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50460/0 (socket says 172.21.15.150:50460) 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c7ad9d700 1 -- 172.21.15.150:0/3524004260 learned_addr learned my addr 172.21.15.150:0/3524004260 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c7a59c700 1 --2- 172.21.15.150:0/3524004260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a27946a0 0x55c3a27b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c79d9b700 1 --2- 172.21.15.150:0/3524004260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a2795570 0x55c3a27b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c7ad9d700 1 -- 172.21.15.150:0/3524004260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a27946a0 msgr2=0x55c3a27b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c7ad9d700 1 --2- 172.21.15.150:0/3524004260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a27946a0 0x55c3a27b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c7ad9d700 1 -- 172.21.15.150:0/3524004260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a2795570 msgr2=0x55c3a27b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c7ad9d700 1 --2- 172.21.15.150:0/3524004260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a2795570 0x55c3a27b6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c7ad9d700 1 -- 172.21.15.150:0/3524004260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3a267a560 con 0x55c3a2795ec0 2024-05-10T04:05:15.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c7a59c700 1 --2- 172.21.15.150:0/3524004260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a27946a0 0x55c3a27b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:15.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c7ad9d700 1 --2- 172.21.15.150:0/3524004260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 0x55c3a27993b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1c74011d40 tx=0x7f1c7400f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27d7eadb01c1a84 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:15.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c697fa700 1 -- 172.21.15.150:0/3524004260 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7400b040 con 0x55c3a2795ec0 2024-05-10T04:05:15.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c697fa700 1 -- 172.21.15.150:0/3524004260 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c7400f8b0 con 0x55c3a2795ec0 2024-05-10T04:05:15.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.574+0000 7f1c697fa700 1 -- 172.21.15.150:0/3524004260 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c74003c60 con 0x55c3a2795ec0 2024-05-10T04:05:15.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.576+0000 7f1c91064b80 1 -- 172.21.15.150:0/3524004260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 msgr2=0x55c3a27993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:15.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.576+0000 7f1c91064b80 1 --2- 172.21.15.150:0/3524004260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 0x55c3a27993b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1c74011d40 tx=0x7f1c7400f130 comp rx=0 tx=0).stop 2024-05-10T04:05:15.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.576+0000 7f1c91064b80 1 -- 172.21.15.150:0/3524004260 shutdown_connections 2024-05-10T04:05:15.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.576+0000 7f1c91064b80 1 --2- 172.21.15.150:0/3524004260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795ec0 0x55c3a27993b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.576+0000 7f1c91064b80 1 --2- 172.21.15.150:0/3524004260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a27946a0 0x55c3a27b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.576+0000 7f1c91064b80 1 --2- 172.21.15.150:0/3524004260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a2795570 0x55c3a27b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.576+0000 7f1c91064b80 1 -- 172.21.15.150:0/3524004260 >> 172.21.15.150:0/3524004260 conn(0x55c3a26f6a60 msgr2=0x55c3a278b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:15.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.576+0000 7f1c91064b80 1 -- 172.21.15.150:0/3524004260 shutdown_connections 2024-05-10T04:05:15.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.576+0000 7f1c91064b80 1 -- 172.21.15.150:0/3524004260 wait complete. 2024-05-10T04:05:15.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.576+0000 7f1c91064b80 1 -- start start 2024-05-10T04:05:15.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.576+0000 7f1c91064b80 1 Processor -- start 2024-05-10T04:05:15.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.577+0000 7f1c91064b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a27946a0 0x55c3a27b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.577+0000 7f1c91064b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 0x55c3a27b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.577+0000 7f1c91064b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a2795ec0 0x55c3a27b5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.577+0000 7f1c91064b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a27b5b80 con 0x55c3a2795570 2024-05-10T04:05:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.577+0000 7f1c91064b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a27b5cf0 con 0x55c3a27946a0 2024-05-10T04:05:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.577+0000 7f1c91064b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a27b5eb0 con 0x55c3a2795ec0 2024-05-10T04:05:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.577+0000 7f1c91064b80 10 client.0 _ll_register_callbacks cb 0x55c3a2659b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.577+0000 7f1c79d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 0x55c3a27b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.577+0000 7f1c79d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 0x55c3a27b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50470/0 (socket says 172.21.15.150:50470) 2024-05-10T04:05:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.577+0000 7f1c79d9b700 1 -- 172.21.15.150:0/10162739 learned_addr learned my addr 172.21.15.150:0/10162739 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.578+0000 7f1c79d9b700 1 -- 172.21.15.150:0/10162739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a2795ec0 msgr2=0x55c3a27b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.578+0000 7f1c79d9b700 1 --2- 172.21.15.150:0/10162739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a2795ec0 0x55c3a27b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.578+0000 7f1c79d9b700 1 -- 172.21.15.150:0/10162739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a27946a0 msgr2=0x55c3a27b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.578+0000 7f1c79d9b700 1 --2- 172.21.15.150:0/10162739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a27946a0 0x55c3a27b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.578+0000 7f1c79d9b700 1 -- 172.21.15.150:0/10162739 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c1a6900 con 0x55c3a2795570 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.578+0000 7f1c79d9b700 1 --2- 172.21.15.150:0/10162739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 0x55c3a27b4710 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000edc0 tx=0x7f1c7000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f14b29cd362bc63 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.578+0000 7f1c61ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.578+0000 7f1c61ffb700 1 -- 172.21.15.150:0/10162739 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70003b40 con 0x55c3a2795570 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.578+0000 7f1c61ffb700 1 -- 172.21.15.150:0/10162739 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1c7002c070 con 0x55c3a2795570 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.578+0000 7f1c91064b80 1 -- 172.21.15.150:0/10162739 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c70021070 con 0x55c3a2795570 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.578+0000 7f1c61ffb700 1 -- 172.21.15.150:0/10162739 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c70003ca0 con 0x55c3a2795570 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.578+0000 7f1c91064b80 10 client.15858 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.578+0000 7f1c91064b80 1 -- 172.21.15.150:0/10162739 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c3a24e1b60 con 0x55c3a2795570 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.579+0000 7f1c61ffb700 1 -- 172.21.15.150:0/10162739 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c70021070 con 0x55c3a2795570 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.579+0000 7f1c61ffb700 1 -- 172.21.15.150:0/10162739 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c70026940 con 0x55c3a2795570 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.579+0000 7f1c91064b80 10 client.15858 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.579+0000 7f1c91064b80 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.579+0000 7f1c91064b80 20 client.15858 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.579+0000 7f1c91064b80 1 --2- 172.21.15.150:0/10162739 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c3a28120e0 0x55c3a2814500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.579+0000 7f1c91064b80 4 client.15858 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.579+0000 7f1c91064b80 1 -- 172.21.15.150:0/10162739 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c3a26acec0 con 0x55c3a28120e0 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.579+0000 7f1c7a59c700 1 --2- 172.21.15.150:0/10162739 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c3a28120e0 0x55c3a2814500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c61ffb700 10 client.15858 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c7a59c700 1 --2- 172.21.15.150:0/10162739 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c3a28120e0 0x55c3a2814500 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a824c75b6ca86e5 server_cookie=8bfb64311af2644b in_seq=0 out_seq=0 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c61ffb700 1 -- 172.21.15.150:0/10162739 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c3a26acec0 con 0x55c3a28120e0 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c61ffb700 10 client.15858 handle_command_reply: tid=0 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c91064b80 1 client.15858 shutdown 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c91064b80 1 -- 172.21.15.150:0/10162739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 msgr2=0x55c3a27b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c91064b80 1 --2- 172.21.15.150:0/10162739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 0x55c3a27b4710 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000edc0 tx=0x7f1c7000eea0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c91064b80 1 -- 172.21.15.150:0/10162739 shutdown_connections 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c91064b80 1 --2- 172.21.15.150:0/10162739 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55c3a28120e0 0x55c3a2814500 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c91064b80 1 --2- 172.21.15.150:0/10162739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3a2795ec0 0x55c3a27b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c91064b80 1 --2- 172.21.15.150:0/10162739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a27946a0 0x55c3a27b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c61ffb700 0 client.15858 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c91064b80 1 --2- 172.21.15.150:0/10162739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3a2795570 0x55c3a27b4710 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.580+0000 7f1c91064b80 1 -- 172.21.15.150:0/10162739 >> 172.21.15.150:0/10162739 conn(0x55c3a26f6a60 msgr2=0x55c3a278b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.581+0000 7f1c91064b80 1 -- 172.21.15.150:0/10162739 shutdown_connections 2024-05-10T04:05:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.581+0000 7f1c91064b80 1 -- 172.21.15.150:0/10162739 wait complete. 2024-05-10T04:05:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.581+0000 7f1c91064b80 20 client.15858 trim_cache size 0 max 16384 2024-05-10T04:05:15.586 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:05:15.586 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "no active scrubs running", 2024-05-10T04:05:15.586 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": {} 2024-05-10T04:05:15.586 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:05:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.581+0000 7f1c80d20700 1 -- 172.21.15.150:0/3755657547 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1c640a12e0 msgr2=0x7f1c640a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.581+0000 7f1c80d20700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1c640a12e0 0x7f1c640a37f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c19f5e0 tx=0x7f1c70009260 comp rx=0 tx=0).stop 2024-05-10T04:05:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.581+0000 7f1c80d20700 1 -- 172.21.15.150:0/3755657547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c7c101c00 msgr2=0x7f1c7c19e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.581+0000 7f1c80d20700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c7c101c00 0x7f1c7c19e540 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c005740 tx=0x7f1c6c009580 comp rx=0 tx=0).stop 2024-05-10T04:05:15.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.582+0000 7f1c80d20700 1 -- 172.21.15.150:0/3755657547 shutdown_connections 2024-05-10T04:05:15.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.582+0000 7f1c80d20700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1c640a12e0 0x7f1c640a37f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.582+0000 7f1c80d20700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c7c1036c0 0x7f1c7c1a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.582+0000 7f1c80d20700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c7c101c00 0x7f1c7c19e540 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.582+0000 7f1c80d20700 1 --2- 172.21.15.150:0/3755657547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c7c102d20 0x7f1c7c19ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.582+0000 7f1c80d20700 1 -- 172.21.15.150:0/3755657547 >> 172.21.15.150:0/3755657547 conn(0x7f1c7c0fd210 msgr2=0x7f1c7c103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:15.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.582+0000 7f1c80d20700 1 -- 172.21.15.150:0/3755657547 shutdown_connections 2024-05-10T04:05:15.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.582+0000 7f1c80d20700 1 -- 172.21.15.150:0/3755657547 wait complete. 2024-05-10T04:05:15.601 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T04:05:15.601 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:05:15.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.753+0000 7f1feb8a0700 1 -- 172.21.15.150:0/4144978345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4102580 msgr2=0x7f1fe4108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:15.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.753+0000 7f1feb8a0700 1 --2- 172.21.15.150:0/4144978345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4102580 0x7f1fe4108e30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00b0a0 tx=0x7f1fdc00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.753+0000 7f1feb8a0700 1 -- 172.21.15.150:0/4144978345 shutdown_connections 2024-05-10T04:05:15.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.753+0000 7f1feb8a0700 1 --2- 172.21.15.150:0/4144978345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4102580 0x7f1fe4108e30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.753+0000 7f1feb8a0700 1 --2- 172.21.15.150:0/4144978345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fe4100ac0 0x7f1fe4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.753+0000 7f1feb8a0700 1 --2- 172.21.15.150:0/4144978345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fe4101be0 0x7f1fe4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.753+0000 7f1feb8a0700 1 -- 172.21.15.150:0/4144978345 >> 172.21.15.150:0/4144978345 conn(0x7f1fe40fc0d0 msgr2=0x7f1fe40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:15.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.753+0000 7f1feb8a0700 1 -- 172.21.15.150:0/4144978345 shutdown_connections 2024-05-10T04:05:15.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.753+0000 7f1feb8a0700 1 -- 172.21.15.150:0/4144978345 wait complete. 2024-05-10T04:05:15.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.754+0000 7f1feb8a0700 1 Processor -- start 2024-05-10T04:05:15.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.754+0000 7f1feb8a0700 1 -- start start 2024-05-10T04:05:15.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.754+0000 7f1feb8a0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4100ac0 0x7f1fe4068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.754+0000 7f1feb8a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fe4101be0 0x7f1fe419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.754+0000 7f1feb8a0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fe4102580 0x7f1fe419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.754+0000 7f1feb8a0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe419fb90 con 0x7f1fe4100ac0 2024-05-10T04:05:15.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.754+0000 7f1feb8a0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe419fd00 con 0x7f1fe4102580 2024-05-10T04:05:15.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.754+0000 7f1feb8a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe419fec0 con 0x7f1fe4101be0 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fe963c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4100ac0 0x7f1fe4068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fe963c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4100ac0 0x7f1fe4068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50480/0 (socket says 172.21.15.150:50480) 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fe963c700 1 -- 172.21.15.150:0/2019472348 learned_addr learned my addr 172.21.15.150:0/2019472348 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fe8e3b700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fe4101be0 0x7f1fe419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fe9e3d700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fe4102580 0x7f1fe419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fe9e3d700 1 -- 172.21.15.150:0/2019472348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fe4101be0 msgr2=0x7f1fe419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fe9e3d700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fe4101be0 0x7f1fe419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fe9e3d700 1 -- 172.21.15.150:0/2019472348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4100ac0 msgr2=0x7f1fe4068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fe9e3d700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4100ac0 0x7f1fe4068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fe9e3d700 1 -- 172.21.15.150:0/2019472348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fdc009d00 con 0x7f1fe4102580 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fe8e3b700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fe4101be0 0x7f1fe419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fe963c700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4100ac0 0x7f1fe4068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fe9e3d700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fe4102580 0x7f1fe419f440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc006130 tx=0x7f1fdc0093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.755+0000 7f1fda7fc700 1 -- 172.21.15.150:0/2019472348 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00e040 con 0x7f1fe4102580 2024-05-10T04:05:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.756+0000 7f1feb8a0700 1 -- 172.21.15.150:0/2019472348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe41a00e0 con 0x7f1fe4102580 2024-05-10T04:05:15.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.756+0000 7f1fda7fc700 1 -- 172.21.15.150:0/2019472348 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fdc01d940 con 0x7f1fe4102580 2024-05-10T04:05:15.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.756+0000 7f1fda7fc700 1 -- 172.21.15.150:0/2019472348 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc01b8f0 con 0x7f1fe4102580 2024-05-10T04:05:15.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.756+0000 7f1feb8a0700 1 -- 172.21.15.150:0/2019472348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe41a05b0 con 0x7f1fe4102580 2024-05-10T04:05:15.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.757+0000 7f1feb8a0700 1 -- 172.21.15.150:0/2019472348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe4107cd0 con 0x7f1fe4102580 2024-05-10T04:05:15.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.758+0000 7f1fda7fc700 1 -- 172.21.15.150:0/2019472348 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1fdc01dab0 con 0x7f1fe4102580 2024-05-10T04:05:15.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.759+0000 7f1fda7fc700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1fd00a1230 0x7f1fd00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:15.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.759+0000 7f1fe963c700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1fd00a1230 0x7f1fd00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:15.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.759+0000 7f1fda7fc700 1 -- 172.21.15.150:0/2019472348 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1fdc0ce550 con 0x7f1fe4102580 2024-05-10T04:05:15.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.760+0000 7f1fe963c700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1fd00a1230 0x7f1fd00a3740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0005d90 tx=0x7f1fe000b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:15.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:15.765+0000 7f1fda7fc700 1 -- 172.21.15.150:0/2019472348 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1fdc097900 con 0x7f1fe4102580 2024-05-10T04:05:16.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.033+0000 7f1feb8a0700 1 -- 172.21.15.150:0/2019472348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fe40694d0 con 0x7f1fe4102580 2024-05-10T04:05:16.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.034+0000 7f1fda7fc700 1 -- 172.21.15.150:0/2019472348 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10886 (secure 0 0 0) 0x7f1fdc09af20 con 0x7f1fe4102580 2024-05-10T04:05:16.036 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:05:16.036 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:04:04.157819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:05:16.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.037+0000 7f1feb8a0700 1 -- 172.21.15.150:0/2019472348 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1fd00a1230 msgr2=0x7f1fd00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.037+0000 7f1feb8a0700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1fd00a1230 0x7f1fd00a3740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0005d90 tx=0x7f1fe000b480 comp rx=0 tx=0).stop 2024-05-10T04:05:16.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.037+0000 7f1feb8a0700 1 -- 172.21.15.150:0/2019472348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fe4102580 msgr2=0x7f1fe419f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.037+0000 7f1feb8a0700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fe4102580 0x7f1fe419f440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc006130 tx=0x7f1fdc0093f0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.037+0000 7f1feb8a0700 1 -- 172.21.15.150:0/2019472348 shutdown_connections 2024-05-10T04:05:16.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.037+0000 7f1feb8a0700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f1fd00a1230 0x7f1fd00a3740 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.037+0000 7f1feb8a0700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fe4102580 0x7f1fe419f440 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.037+0000 7f1feb8a0700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fe4100ac0 0x7f1fe4068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.037+0000 7f1feb8a0700 1 --2- 172.21.15.150:0/2019472348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fe4101be0 0x7f1fe419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.037+0000 7f1feb8a0700 1 -- 172.21.15.150:0/2019472348 >> 172.21.15.150:0/2019472348 conn(0x7f1fe40fc0d0 msgr2=0x7f1fe4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.037+0000 7f1feb8a0700 1 -- 172.21.15.150:0/2019472348 shutdown_connections 2024-05-10T04:05:16.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.037+0000 7f1feb8a0700 1 -- 172.21.15.150:0/2019472348 wait complete. 2024-05-10T04:05:16.041 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 25 2024-05-10T04:05:16.053 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T04:05:16.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.190+0000 7f05dfba1700 1 -- 172.21.15.150:0/2655699985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05d8108630 msgr2=0x7f05d8108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.190+0000 7f05dfba1700 1 --2- 172.21.15.150:0/2655699985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05d8108630 0x7f05d8108a40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009990 tx=0x7f05d4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.190+0000 7f05dfba1700 1 -- 172.21.15.150:0/2655699985 shutdown_connections 2024-05-10T04:05:16.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.190+0000 7f05dfba1700 1 --2- 172.21.15.150:0/2655699985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d8109c50 0x7f05d810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.190+0000 7f05dfba1700 1 --2- 172.21.15.150:0/2655699985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05d8108630 0x7f05d8108a40 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.190+0000 7f05dfba1700 1 --2- 172.21.15.150:0/2655699985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05d8109300 0x7f05d8109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.190+0000 7f05dfba1700 1 -- 172.21.15.150:0/2655699985 >> 172.21.15.150:0/2655699985 conn(0x7f05d806ea00 msgr2=0x7f05d8070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.190+0000 7f05dfba1700 1 -- 172.21.15.150:0/2655699985 shutdown_connections 2024-05-10T04:05:16.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.190+0000 7f05dfba1700 1 -- 172.21.15.150:0/2655699985 wait complete. 2024-05-10T04:05:16.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.191+0000 7f05dfba1700 1 Processor -- start 2024-05-10T04:05:16.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.191+0000 7f05dfba1700 1 -- start start 2024-05-10T04:05:16.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.191+0000 7f05dfba1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05d8108630 0x7f05d81a6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.191+0000 7f05dfba1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05d8109300 0x7f05d81a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.191+0000 7f05dfba1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d8109c50 0x7f05d81ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.191+0000 7f05dfba1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d81ae840 con 0x7f05d8108630 2024-05-10T04:05:16.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.191+0000 7f05dfba1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d81ae9b0 con 0x7f05d8109c50 2024-05-10T04:05:16.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.191+0000 7f05dfba1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d81aeb70 con 0x7f05d8109300 2024-05-10T04:05:16.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.191+0000 7f05dd93d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05d8108630 0x7f05d81a6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.191+0000 7f05dd93d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05d8108630 0x7f05d81a6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50498/0 (socket says 172.21.15.150:50498) 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.191+0000 7f05dd93d700 1 -- 172.21.15.150:0/2809035886 learned_addr learned my addr 172.21.15.150:0/2809035886 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.192+0000 7f05de13e700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d8109c50 0x7f05d81ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.192+0000 7f05dd13c700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05d8109300 0x7f05d81a72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.192+0000 7f05de13e700 1 -- 172.21.15.150:0/2809035886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05d8109300 msgr2=0x7f05d81a72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.192+0000 7f05de13e700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05d8109300 0x7f05d81a72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.192+0000 7f05de13e700 1 -- 172.21.15.150:0/2809035886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05d8108630 msgr2=0x7f05d81a6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.192+0000 7f05de13e700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05d8108630 0x7f05d81a6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.192+0000 7f05de13e700 1 -- 172.21.15.150:0/2809035886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d40095f0 con 0x7f05d8109c50 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.192+0000 7f05de13e700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d8109c50 0x7f05d81ae200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f05d0007f10 tx=0x7f05d000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.192+0000 7f05ceffd700 1 -- 172.21.15.150:0/2809035886 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0004db0 con 0x7f05d8109c50 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.192+0000 7f05ceffd700 1 -- 172.21.15.150:0/2809035886 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05d000f040 con 0x7f05d8109c50 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.192+0000 7f05dfba1700 1 -- 172.21.15.150:0/2809035886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d81aee50 con 0x7f05d8109c50 2024-05-10T04:05:16.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.193+0000 7f05dfba1700 1 -- 172.21.15.150:0/2809035886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d81af270 con 0x7f05d8109c50 2024-05-10T04:05:16.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.194+0000 7f05ceffd700 1 -- 172.21.15.150:0/2809035886 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0005990 con 0x7f05d8109c50 2024-05-10T04:05:16.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.196+0000 7f05ceffd700 1 -- 172.21.15.150:0/2809035886 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f05d0015400 con 0x7f05d8109c50 2024-05-10T04:05:16.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.197+0000 7f05efee5b80 1 Processor -- start 2024-05-10T04:05:16.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.197+0000 7f05ceffd700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f05c40a1260 0x7f05c40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.197+0000 7f05ceffd700 1 -- 172.21.15.150:0/2809035886 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f05d00cc4b0 con 0x7f05d8109c50 2024-05-10T04:05:16.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.197+0000 7f05efee5b80 1 -- start start 2024-05-10T04:05:16.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05efee5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99a6a0 0x56000a99aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05efee5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 0x56000a99b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05efee5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99bec0 0x56000a9a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05efee5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000a8c1150 con 0x56000a99a6a0 2024-05-10T04:05:16.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05efee5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000a8beb10 con 0x56000a99b570 2024-05-10T04:05:16.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05efee5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000a8ba400 con 0x56000a99bec0 2024-05-10T04:05:16.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05dd93d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99a6a0 0x56000a99aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05dd93d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99a6a0 0x56000a99aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50508/0 (socket says 172.21.15.150:50508) 2024-05-10T04:05:16.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05dd93d700 1 -- 172.21.15.150:0/741600804 learned_addr learned my addr 172.21.15.150:0/741600804 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05dd13c700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f05c40a1260 0x7f05c40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05dd93d700 1 -- 172.21.15.150:0/741600804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99bec0 msgr2=0x56000a9a27c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:05:16.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05dd93d700 1 --2- 172.21.15.150:0/741600804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99bec0 0x56000a9a27c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05dd93d700 1 -- 172.21.15.150:0/741600804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 msgr2=0x56000a99b980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:05:16.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05dd93d700 1 --2- 172.21.15.150:0/741600804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 0x56000a99b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.198+0000 7f05dd93d700 1 -- 172.21.15.150:0/741600804 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56000a73b3b0 con 0x56000a99a6a0 2024-05-10T04:05:16.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.199+0000 7f05dd93d700 1 --2- 172.21.15.150:0/741600804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99a6a0 0x56000a99aaf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f05d40095c0 tx=0x7f05d4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af96782cc120d51 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.199+0000 7f05dd13c700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f05c40a1260 0x7f05c40a3770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009d00 tx=0x7f05c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.199+0000 7f05c3fff700 1 -- 172.21.15.150:0/741600804 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d400bbd0 con 0x56000a99a6a0 2024-05-10T04:05:16.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.199+0000 7f05c3fff700 1 -- 172.21.15.150:0/741600804 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05d401e040 con 0x56000a99a6a0 2024-05-10T04:05:16.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.199+0000 7f05c3fff700 1 -- 172.21.15.150:0/741600804 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d40223f0 con 0x56000a99a6a0 2024-05-10T04:05:16.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.200+0000 7f05efee5b80 1 -- 172.21.15.150:0/741600804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99a6a0 msgr2=0x56000a99aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.200+0000 7f05efee5b80 1 --2- 172.21.15.150:0/741600804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99a6a0 0x56000a99aaf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f05d40095c0 tx=0x7f05d4005fd0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.200+0000 7f05efee5b80 1 -- 172.21.15.150:0/741600804 shutdown_connections 2024-05-10T04:05:16.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.200+0000 7f05efee5b80 1 --2- 172.21.15.150:0/741600804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99bec0 0x56000a9a27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.200+0000 7f05efee5b80 1 --2- 172.21.15.150:0/741600804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99a6a0 0x56000a99aaf0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.200+0000 7f05efee5b80 1 --2- 172.21.15.150:0/741600804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 0x56000a99b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.200+0000 7f05efee5b80 1 -- 172.21.15.150:0/741600804 >> 172.21.15.150:0/741600804 conn(0x56000a8fca60 msgr2=0x56000a994fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.200+0000 7f05efee5b80 1 -- 172.21.15.150:0/741600804 shutdown_connections 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.200+0000 7f05efee5b80 1 -- 172.21.15.150:0/741600804 wait complete. 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.201+0000 7f05efee5b80 1 -- start start 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.201+0000 7f05efee5b80 1 Processor -- start 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05efee5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 0x56000a9b5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05efee5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 0x56000a9b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05efee5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99bec0 0x56000a9bcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05efee5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000a856560 con 0x56000a99bec0 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05efee5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000a856200 con 0x56000a99b570 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05efee5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000a73d6a0 con 0x56000a99a6a0 2024-05-10T04:05:16.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05efee5b80 10 client.0 _ll_register_callbacks cb 0x56000a85fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:16.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05dd93d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 0x56000a9b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05dd93d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 0x56000a9b5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:57886/0 (socket says 172.21.15.150:57886) 2024-05-10T04:05:16.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05dd93d700 1 -- 172.21.15.150:0/1864104239 learned_addr learned my addr 172.21.15.150:0/1864104239 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05de13e700 1 --2- 172.21.15.150:0/1864104239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99bec0 0x56000a9bcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05dd13c700 1 --2- 172.21.15.150:0/1864104239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 0x56000a9b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05dd93d700 1 -- 172.21.15.150:0/1864104239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 msgr2=0x56000a9b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05dd93d700 1 --2- 172.21.15.150:0/1864104239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 0x56000a9b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05dd93d700 1 -- 172.21.15.150:0/1864104239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99bec0 msgr2=0x56000a9bcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05dd93d700 1 --2- 172.21.15.150:0/1864104239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99bec0 0x56000a9bcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.202+0000 7f05dd93d700 1 -- 172.21.15.150:0/1864104239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d810efc0 con 0x56000a99a6a0 2024-05-10T04:05:16.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.203+0000 7f05dd93d700 1 --2- 172.21.15.150:0/1864104239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 0x56000a9b5720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f05d400b2e0 tx=0x7f05d4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6892fc8cf1badb45 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.203+0000 7f05c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:05:16.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.203+0000 7f05c17fa700 1 -- 172.21.15.150:0/1864104239 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4005230 con 0x56000a99a6a0 2024-05-10T04:05:16.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.203+0000 7f05c17fa700 1 -- 172.21.15.150:0/1864104239 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f05d4005390 con 0x56000a99a6a0 2024-05-10T04:05:16.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.203+0000 7f05efee5b80 1 -- 172.21.15.150:0/1864104239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56000a813270 con 0x56000a99a6a0 2024-05-10T04:05:16.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.209+0000 7f05c17fa700 1 -- 172.21.15.150:0/1864104239 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05d4005550 con 0x56000a99a6a0 2024-05-10T04:05:16.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.209+0000 7f05efee5b80 10 client.25504 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:16.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.209+0000 7f05efee5b80 1 -- 172.21.15.150:0/1864104239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56000a6e7b60 con 0x56000a99a6a0 2024-05-10T04:05:16.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.210+0000 7f05c17fa700 1 -- 172.21.15.150:0/1864104239 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05d403c030 con 0x56000a99a6a0 2024-05-10T04:05:16.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.210+0000 7f05c17fa700 1 -- 172.21.15.150:0/1864104239 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f05d4021070 con 0x56000a99a6a0 2024-05-10T04:05:16.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.210+0000 7f05efee5b80 10 client.25504 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:16.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.210+0000 7f05efee5b80 10 client.25504 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T04:05:16.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.210+0000 7f05efee5b80 20 client.25504 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:16.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.210+0000 7f05efee5b80 1 --2- 172.21.15.150:0/1864104239 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x56000a9c0f10 0x56000a9c3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.210+0000 7f05efee5b80 4 client.25504 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:16.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.210+0000 7f05efee5b80 1 -- 172.21.15.150:0/1864104239 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56000a969780 con 0x56000a9c0f10 2024-05-10T04:05:16.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.210+0000 7f05dd13c700 1 --2- 172.21.15.150:0/1864104239 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x56000a9c0f10 0x56000a9c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.211+0000 7f05c17fa700 10 client.25504 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:05:16.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.211+0000 7f05dd13c700 1 --2- 172.21.15.150:0/1864104239 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x56000a9c0f10 0x56000a9c3330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e001d59cb0da8ca6 server_cookie=2b919c331b05efbb in_seq=0 out_seq=0 2024-05-10T04:05:16.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05c17fa700 1 -- 172.21.15.150:0/1864104239 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56000a969780 con 0x56000a9c0f10 2024-05-10T04:05:16.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05c17fa700 10 client.25504 handle_command_reply: tid=0 2024-05-10T04:05:16.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05efee5b80 1 client.25504 shutdown 2024-05-10T04:05:16.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05efee5b80 1 -- 172.21.15.150:0/1864104239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 msgr2=0x56000a9b5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05efee5b80 1 --2- 172.21.15.150:0/1864104239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 0x56000a9b5720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f05d400b2e0 tx=0x7f05d4009310 comp rx=0 tx=0).stop 2024-05-10T04:05:16.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05efee5b80 1 -- 172.21.15.150:0/1864104239 shutdown_connections 2024-05-10T04:05:16.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05efee5b80 1 --2- 172.21.15.150:0/1864104239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99bec0 0x56000a9bcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05efee5b80 1 --2- 172.21.15.150:0/1864104239 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x56000a9c0f10 0x56000a9c3330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05efee5b80 1 --2- 172.21.15.150:0/1864104239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 0x56000a9b5720 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05efee5b80 1 --2- 172.21.15.150:0/1864104239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 0x56000a9b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05efee5b80 1 -- 172.21.15.150:0/1864104239 >> 172.21.15.150:0/1864104239 conn(0x56000a8fca60 msgr2=0x56000a994fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05c17fa700 0 client.25504 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:05:16.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05efee5b80 1 -- 172.21.15.150:0/1864104239 shutdown_connections 2024-05-10T04:05:16.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05efee5b80 1 -- 172.21.15.150:0/1864104239 wait complete. 2024-05-10T04:05:16.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.212+0000 7f05efee5b80 20 client.25504 trim_cache size 0 max 16384 2024-05-10T04:05:16.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.236+0000 7f05efee5b80 1 Processor -- start 2024-05-10T04:05:16.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.236+0000 7f05efee5b80 1 -- start start 2024-05-10T04:05:16.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.236+0000 7f05efee5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 0x56000a99e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.236+0000 7f05efee5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99b570 0x56000a9bcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05efee5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99bec0 0x56000a99f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05efee5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000a99b3c0 con 0x56000a99b570 2024-05-10T04:05:16.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05efee5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000a996770 con 0x56000a99bec0 2024-05-10T04:05:16.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05efee5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000a9c0c90 con 0x56000a99a6a0 2024-05-10T04:05:16.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05dd13c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99b570 0x56000a9bcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05dd13c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99b570 0x56000a9bcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50526/0 (socket says 172.21.15.150:50526) 2024-05-10T04:05:16.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05dd13c700 1 -- 172.21.15.150:0/2861890608 learned_addr learned my addr 172.21.15.150:0/2861890608 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05dd93d700 1 --2- 172.21.15.150:0/2861890608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 0x56000a99e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05de13e700 1 --2- 172.21.15.150:0/2861890608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99bec0 0x56000a99f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05dd13c700 1 -- 172.21.15.150:0/2861890608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 msgr2=0x56000a99e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05dd13c700 1 --2- 172.21.15.150:0/2861890608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 0x56000a99e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05dd13c700 1 -- 172.21.15.150:0/2861890608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99bec0 msgr2=0x56000a99f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05dd13c700 1 --2- 172.21.15.150:0/2861890608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99bec0 0x56000a99f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05dd13c700 1 -- 172.21.15.150:0/2861890608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56000a880560 con 0x56000a99b570 2024-05-10T04:05:16.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.237+0000 7f05dd13c700 1 --2- 172.21.15.150:0/2861890608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99b570 0x56000a9bcb80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f05c8011ab0 tx=0x7f05c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3050b90946060e1b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.238+0000 7f05ccd7f700 1 -- 172.21.15.150:0/2861890608 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8025b00 con 0x56000a99b570 2024-05-10T04:05:16.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.238+0000 7f05ccd7f700 1 -- 172.21.15.150:0/2861890608 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05c8024040 con 0x56000a99b570 2024-05-10T04:05:16.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.238+0000 7f05ccd7f700 1 -- 172.21.15.150:0/2861890608 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c80256f0 con 0x56000a99b570 2024-05-10T04:05:16.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.239+0000 7f05efee5b80 1 -- 172.21.15.150:0/2861890608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99b570 msgr2=0x56000a9bcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.239+0000 7f05efee5b80 1 --2- 172.21.15.150:0/2861890608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99b570 0x56000a9bcb80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f05c8011ab0 tx=0x7f05c8014000 comp rx=0 tx=0).stop 2024-05-10T04:05:16.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.239+0000 7f05efee5b80 1 -- 172.21.15.150:0/2861890608 shutdown_connections 2024-05-10T04:05:16.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.239+0000 7f05efee5b80 1 --2- 172.21.15.150:0/2861890608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99bec0 0x56000a99f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.239+0000 7f05efee5b80 1 --2- 172.21.15.150:0/2861890608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 0x56000a99e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.239+0000 7f05efee5b80 1 --2- 172.21.15.150:0/2861890608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99b570 0x56000a9bcb80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.239+0000 7f05efee5b80 1 -- 172.21.15.150:0/2861890608 >> 172.21.15.150:0/2861890608 conn(0x56000a8fca60 msgr2=0x56000a994f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.239+0000 7f05efee5b80 1 -- 172.21.15.150:0/2861890608 shutdown_connections 2024-05-10T04:05:16.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.240+0000 7f05efee5b80 1 -- 172.21.15.150:0/2861890608 wait complete. 2024-05-10T04:05:16.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.241+0000 7f05efee5b80 1 -- start start 2024-05-10T04:05:16.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.241+0000 7f05efee5b80 1 Processor -- start 2024-05-10T04:05:16.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.241+0000 7f05efee5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 0x56000a9bb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.241+0000 7f05efee5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 0x56000a9bbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.241+0000 7f05efee5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99bec0 0x56000a9bc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.241+0000 7f05efee5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000aa15460 con 0x56000a99bec0 2024-05-10T04:05:16.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.241+0000 7f05efee5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000aa155d0 con 0x56000a99b570 2024-05-10T04:05:16.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.241+0000 7f05efee5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000aa15790 con 0x56000a99a6a0 2024-05-10T04:05:16.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.241+0000 7f05efee5b80 10 client.0 _ll_register_callbacks cb 0x56000a85fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:16.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.242+0000 7f05de13e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99bec0 0x56000a9bc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.242+0000 7f05de13e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99bec0 0x56000a9bc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50530/0 (socket says 172.21.15.150:50530) 2024-05-10T04:05:16.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.242+0000 7f05de13e700 1 -- 172.21.15.150:0/3507100641 learned_addr learned my addr 172.21.15.150:0/3507100641 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.242+0000 7f05dd13c700 1 --2- 172.21.15.150:0/3507100641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 0x56000a9bbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.242+0000 7f05de13e700 1 -- 172.21.15.150:0/3507100641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 msgr2=0x56000a9bb5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:05:16.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.242+0000 7f05de13e700 1 --2- 172.21.15.150:0/3507100641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 0x56000a9bb5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.242+0000 7f05de13e700 1 -- 172.21.15.150:0/3507100641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 msgr2=0x56000a9bbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.242+0000 7f05de13e700 1 --2- 172.21.15.150:0/3507100641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 0x56000a9bbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.242+0000 7f05de13e700 1 -- 172.21.15.150:0/3507100641 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c8017070 con 0x56000a99bec0 2024-05-10T04:05:16.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.242+0000 7f05de13e700 1 --2- 172.21.15.150:0/3507100641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99bec0 0x56000a9bc040 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f05d0099420 tx=0x7f05d0098e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20713f0353fb6471 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.242+0000 7f05c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:16.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.242+0000 7f05c17fa700 1 -- 172.21.15.150:0/3507100641 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0009e20 con 0x56000a99bec0 2024-05-10T04:05:16.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.243+0000 7f05c17fa700 1 -- 172.21.15.150:0/3507100641 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f05d0099490 con 0x56000a99bec0 2024-05-10T04:05:16.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.243+0000 7f05efee5b80 1 -- 172.21.15.150:0/3507100641 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05d401e040 con 0x56000a99bec0 2024-05-10T04:05:16.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.243+0000 7f05c17fa700 1 -- 172.21.15.150:0/3507100641 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05d009d2c0 con 0x56000a99bec0 2024-05-10T04:05:16.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.243+0000 7f05efee5b80 10 client.15882 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:16.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.243+0000 7f05efee5b80 1 -- 172.21.15.150:0/3507100641 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56000a73fcd0 con 0x56000a99bec0 2024-05-10T04:05:16.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.243+0000 7f05c17fa700 1 -- 172.21.15.150:0/3507100641 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05d401e040 con 0x56000a99bec0 2024-05-10T04:05:16.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.243+0000 7f05c17fa700 1 -- 172.21.15.150:0/3507100641 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f05d0098980 con 0x56000a99bec0 2024-05-10T04:05:16.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.243+0000 7f05efee5b80 10 client.15882 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:16.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.243+0000 7f05efee5b80 10 client.15882 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T04:05:16.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.243+0000 7f05efee5b80 20 client.15882 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:16.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.243+0000 7f05efee5b80 1 --2- 172.21.15.150:0/3507100641 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x56000aa17190 0x56000aa195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.243+0000 7f05efee5b80 4 client.15882 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:05:16.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.243+0000 7f05efee5b80 1 -- 172.21.15.150:0/3507100641 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56000a8b2ec0 con 0x56000aa17190 2024-05-10T04:05:16.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.244+0000 7f05dd93d700 1 --2- 172.21.15.150:0/3507100641 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x56000aa17190 0x56000aa195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05dd93d700 1 --2- 172.21.15.150:0/3507100641 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x56000aa17190 0x56000aa195b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=922a43f8610230ce server_cookie=8d781239560b0dc7 in_seq=0 out_seq=0 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05c17fa700 10 client.15882 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05c17fa700 1 -- 172.21.15.150:0/3507100641 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56000a8b2ec0 con 0x56000aa17190 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05c17fa700 10 client.15882 handle_command_reply: tid=0 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05efee5b80 1 client.15882 shutdown 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05efee5b80 1 -- 172.21.15.150:0/3507100641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99bec0 msgr2=0x56000a9bc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05efee5b80 1 --2- 172.21.15.150:0/3507100641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99bec0 0x56000a9bc040 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f05d0099420 tx=0x7f05d0098e70 comp rx=0 tx=0).stop 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05efee5b80 1 -- 172.21.15.150:0/3507100641 shutdown_connections 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05efee5b80 1 --2- 172.21.15.150:0/3507100641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56000a99bec0 0x56000a9bc040 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05efee5b80 1 --2- 172.21.15.150:0/3507100641 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x56000aa17190 0x56000aa195b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05efee5b80 1 --2- 172.21.15.150:0/3507100641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56000a99a6a0 0x56000a9bb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05efee5b80 1 --2- 172.21.15.150:0/3507100641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56000a99b570 0x56000a9bbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05c17fa700 0 client.15882 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:05:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.245+0000 7f05efee5b80 1 -- 172.21.15.150:0/3507100641 >> 172.21.15.150:0/3507100641 conn(0x56000a8fca60 msgr2=0x56000a9c2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.246+0000 7f05efee5b80 1 -- 172.21.15.150:0/3507100641 shutdown_connections 2024-05-10T04:05:16.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.246+0000 7f05efee5b80 1 -- 172.21.15.150:0/3507100641 wait complete. 2024-05-10T04:05:16.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.246+0000 7f05efee5b80 20 client.15882 trim_cache size 0 max 16384 2024-05-10T04:05:16.250 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:05:16.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.246+0000 7f05dfba1700 1 -- 172.21.15.150:0/2809035886 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f05c40a1260 msgr2=0x7f05c40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.246+0000 7f05dfba1700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f05c40a1260 0x7f05c40a3770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009d00 tx=0x7f05c8009260 comp rx=0 tx=0).stop 2024-05-10T04:05:16.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.246+0000 7f05dfba1700 1 -- 172.21.15.150:0/2809035886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d8109c50 msgr2=0x7f05d81ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.246+0000 7f05dfba1700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d8109c50 0x7f05d81ae200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f05d0007f10 tx=0x7f05d000d3b0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.247+0000 7f05dfba1700 1 -- 172.21.15.150:0/2809035886 shutdown_connections 2024-05-10T04:05:16.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.247+0000 7f05dfba1700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f05c40a1260 0x7f05c40a3770 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.247+0000 7f05dfba1700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d8109c50 0x7f05d81ae200 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.247+0000 7f05dfba1700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05d8108630 0x7f05d81a6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.247+0000 7f05dfba1700 1 --2- 172.21.15.150:0/2809035886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05d8109300 0x7f05d81a72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.247+0000 7f05dfba1700 1 -- 172.21.15.150:0/2809035886 >> 172.21.15.150:0/2809035886 conn(0x7f05d806ea00 msgr2=0x7f05d806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.247+0000 7f05dfba1700 1 -- 172.21.15.150:0/2809035886 shutdown_connections 2024-05-10T04:05:16.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.247+0000 7f05dfba1700 1 -- 172.21.15.150:0/2809035886 wait complete. 2024-05-10T04:05:16.265 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T04:05:16.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.441+0000 7f2b1e62d700 1 -- 172.21.15.150:0/2985706753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b181008e0 msgr2=0x7f2b18100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.441+0000 7f2b1e62d700 1 --2- 172.21.15.150:0/2985706753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b181008e0 0x7f2b18100db0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009990 tx=0x7f2b0c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.441+0000 7f2b1e62d700 1 -- 172.21.15.150:0/2985706753 shutdown_connections 2024-05-10T04:05:16.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.441+0000 7f2b1e62d700 1 --2- 172.21.15.150:0/2985706753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b18101440 0x7f2b18105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.441+0000 7f2b1e62d700 1 --2- 172.21.15.150:0/2985706753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b180ffe80 0x7f2b181002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.441+0000 7f2b1e62d700 1 --2- 172.21.15.150:0/2985706753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b181008e0 0x7f2b18100db0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.441+0000 7f2b1e62d700 1 -- 172.21.15.150:0/2985706753 >> 172.21.15.150:0/2985706753 conn(0x7f2b180fb8b0 msgr2=0x7f2b180fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.442+0000 7f2b1e62d700 1 -- 172.21.15.150:0/2985706753 shutdown_connections 2024-05-10T04:05:16.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.442+0000 7f2b1e62d700 1 -- 172.21.15.150:0/2985706753 wait complete. 2024-05-10T04:05:16.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.442+0000 7f2b1e62d700 1 Processor -- start 2024-05-10T04:05:16.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.442+0000 7f2b1e62d700 1 -- start start 2024-05-10T04:05:16.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b1e62d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b180ffe80 0x7f2b1819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b1e62d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b181008e0 0x7f2b1819ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b1e62d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b18101440 0x7f2b181a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b1e62d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b181a5fb0 con 0x7f2b18101440 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b1e62d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b181a6120 con 0x7f2b180ffe80 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b1e62d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b181a62e0 con 0x7f2b181008e0 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b17fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b180ffe80 0x7f2b1819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b1cbca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b18101440 0x7f2b181a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b17fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b180ffe80 0x7f2b1819e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:43898/0 (socket says 172.21.15.150:43898) 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b1cbca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b18101440 0x7f2b181a59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50554/0 (socket says 172.21.15.150:50554) 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b17fff700 1 -- 172.21.15.150:0/3846603279 learned_addr learned my addr 172.21.15.150:0/3846603279 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b177fe700 1 --2- 172.21.15.150:0/3846603279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b181008e0 0x7f2b1819ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b1cbca700 1 -- 172.21.15.150:0/3846603279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b181008e0 msgr2=0x7f2b1819ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b1cbca700 1 --2- 172.21.15.150:0/3846603279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b181008e0 0x7f2b1819ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b1cbca700 1 -- 172.21.15.150:0/3846603279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b180ffe80 msgr2=0x7f2b1819e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b1cbca700 1 --2- 172.21.15.150:0/3846603279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b180ffe80 0x7f2b1819e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.443+0000 7f2b1cbca700 1 -- 172.21.15.150:0/3846603279 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b08009520 con 0x7f2b18101440 2024-05-10T04:05:16.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.444+0000 7f2b1cbca700 1 --2- 172.21.15.150:0/3846603279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b18101440 0x7f2b181a59a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000ab40 tx=0x7f2b1000ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.444+0000 7f2b157fa700 1 -- 172.21.15.150:0/3846603279 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1000f070 con 0x7f2b18101440 2024-05-10T04:05:16.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.444+0000 7f2b1e62d700 1 -- 172.21.15.150:0/3846603279 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b0c0095f0 con 0x7f2b18101440 2024-05-10T04:05:16.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.444+0000 7f2b157fa700 1 -- 172.21.15.150:0/3846603279 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b10007a70 con 0x7f2b18101440 2024-05-10T04:05:16.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.444+0000 7f2b157fa700 1 -- 172.21.15.150:0/3846603279 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10005280 con 0x7f2b18101440 2024-05-10T04:05:16.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.444+0000 7f2b1e62d700 1 -- 172.21.15.150:0/3846603279 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b181a6940 con 0x7f2b18101440 2024-05-10T04:05:16.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.448+0000 7f2b157fa700 1 -- 172.21.15.150:0/3846603279 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f2b10004ad0 con 0x7f2b18101440 2024-05-10T04:05:16.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.448+0000 7f2b157fa700 1 --2- 172.21.15.150:0/3846603279 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2b000a1260 0x7f2b000a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.448+0000 7f2b2e971b80 1 Processor -- start 2024-05-10T04:05:16.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b157fa700 1 -- 172.21.15.150:0/3846603279 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2b100cc140 con 0x7f2b18101440 2024-05-10T04:05:16.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b2e971b80 1 -- start start 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b17fff700 1 --2- 172.21.15.150:0/3846603279 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2b000a1260 0x7f2b000a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b2e971b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325e6a0 0x55d98325eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b2e971b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325f570 0x55d98325f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b2e971b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325fec0 0x55d9832667c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b2e971b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d983185150 con 0x55d98325f570 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b2e971b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d983182b10 con 0x55d98325e6a0 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b2e971b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98317e400 con 0x55d98325fec0 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b177fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325f570 0x55d98325f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b177fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325f570 0x55d98325f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50560/0 (socket says 172.21.15.150:50560) 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b177fe700 1 -- 172.21.15.150:0/4165296343 learned_addr learned my addr 172.21.15.150:0/4165296343 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b17fff700 1 --2- 172.21.15.150:0/4165296343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325e6a0 0x55d98325eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.449+0000 7f2b17fff700 1 --2- 172.21.15.150:0/3846603279 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2b000a1260 0x7f2b000a3770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009f40 tx=0x7f2b08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.450+0000 7f2b177fe700 1 -- 172.21.15.150:0/4165296343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325fec0 msgr2=0x55d9832667c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.450+0000 7f2b177fe700 1 --2- 172.21.15.150:0/4165296343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325fec0 0x55d9832667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.450+0000 7f2b177fe700 1 -- 172.21.15.150:0/4165296343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325e6a0 msgr2=0x55d98325eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.450+0000 7f2b177fe700 1 --2- 172.21.15.150:0/4165296343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325e6a0 0x55d98325eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.450+0000 7f2b177fe700 1 -- 172.21.15.150:0/4165296343 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d982fff3b0 con 0x55d98325f570 2024-05-10T04:05:16.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.450+0000 7f2b17fff700 1 --2- 172.21.15.150:0/4165296343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325e6a0 0x55d98325eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:16.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.450+0000 7f2b177fe700 1 --2- 172.21.15.150:0/4165296343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325f570 0x55d98325f980 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c015000 tx=0x7f2b0c00b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d533aa29112c90a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.450+0000 7f2afe7fc700 1 -- 172.21.15.150:0/4165296343 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0058e0 con 0x55d98325f570 2024-05-10T04:05:16.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.450+0000 7f2afe7fc700 1 -- 172.21.15.150:0/4165296343 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b0c01e040 con 0x55d98325f570 2024-05-10T04:05:16.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.450+0000 7f2afe7fc700 1 -- 172.21.15.150:0/4165296343 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c022650 con 0x55d98325f570 2024-05-10T04:05:16.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.451+0000 7f2b2e971b80 1 -- 172.21.15.150:0/4165296343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325f570 msgr2=0x55d98325f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.451+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/4165296343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325f570 0x55d98325f980 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c015000 tx=0x7f2b0c00b750 comp rx=0 tx=0).stop 2024-05-10T04:05:16.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.451+0000 7f2b2e971b80 1 -- 172.21.15.150:0/4165296343 shutdown_connections 2024-05-10T04:05:16.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.451+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/4165296343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325fec0 0x55d9832667c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.451+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/4165296343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325e6a0 0x55d98325eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.451+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/4165296343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325f570 0x55d98325f980 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.451+0000 7f2b2e971b80 1 -- 172.21.15.150:0/4165296343 >> 172.21.15.150:0/4165296343 conn(0x55d9831c0a60 msgr2=0x55d983258fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.452+0000 7f2b2e971b80 1 -- 172.21.15.150:0/4165296343 shutdown_connections 2024-05-10T04:05:16.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.452+0000 7f2b2e971b80 1 -- 172.21.15.150:0/4165296343 wait complete. 2024-05-10T04:05:16.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.452+0000 7f2b2e971b80 1 -- start start 2024-05-10T04:05:16.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.452+0000 7f2b2e971b80 1 Processor -- start 2024-05-10T04:05:16.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b2e971b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325e6a0 0x55d983268630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b2e971b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325f570 0x55d983268b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b2e971b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325fec0 0x55d98326fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b2e971b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98311a560 con 0x55d98325e6a0 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b2e971b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98311a200 con 0x55d98325fec0 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b2e971b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9830016a0 con 0x55d98325f570 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b2e971b80 10 client.0 _ll_register_callbacks cb 0x55d983123b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b1cbca700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325fec0 0x55d98326fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b1cbca700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325fec0 0x55d98326fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:43918/0 (socket says 172.21.15.150:43918) 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b1cbca700 1 -- 172.21.15.150:0/50450129 learned_addr learned my addr 172.21.15.150:0/50450129 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b177fe700 1 --2- 172.21.15.150:0/50450129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325f570 0x55d983268b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b1cbca700 1 -- 172.21.15.150:0/50450129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325f570 msgr2=0x55d983268b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b1cbca700 1 --2- 172.21.15.150:0/50450129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325f570 0x55d983268b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b1cbca700 1 -- 172.21.15.150:0/50450129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325e6a0 msgr2=0x55d983268630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b1cbca700 1 --2- 172.21.15.150:0/50450129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325e6a0 0x55d983268630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.453+0000 7f2b1cbca700 1 -- 172.21.15.150:0/50450129 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b18106880 con 0x55d98325fec0 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2b1cbca700 1 --2- 172.21.15.150:0/50450129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325fec0 0x55d98326fad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2b100a1070 tx=0x7f2b10098c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966e028b471c84ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2af77fe700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2af77fe700 1 -- 172.21.15.150:0/50450129 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1009cfe0 con 0x55d98325fec0 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2b2e971b80 1 -- 172.21.15.150:0/50450129 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9830d7270 con 0x55d98325fec0 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2af77fe700 1 -- 172.21.15.150:0/50450129 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2b100a15a0 con 0x55d98325fec0 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2af77fe700 1 -- 172.21.15.150:0/50450129 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b10004dd0 con 0x55d98325fec0 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2b2e971b80 10 client.25568 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2b2e971b80 1 -- 172.21.15.150:0/50450129 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d982fabb60 con 0x55d98325fec0 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2af77fe700 1 -- 172.21.15.150:0/50450129 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d9830d7270 con 0x55d98325fec0 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2af77fe700 1 -- 172.21.15.150:0/50450129 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2b100a1110 con 0x55d98325fec0 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2b2e971b80 10 client.25568 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2b2e971b80 10 client.25568 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:05:16.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2b2e971b80 20 client.25568 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:16.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/50450129 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x55d9832853b0 0x55d983287800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2b2e971b80 4 client.25568 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:16.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.454+0000 7f2b2e971b80 1 -- 172.21.15.150:0/50450129 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d98322d780 con 0x55d9832853b0 2024-05-10T04:05:16.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.455+0000 7f2b17fff700 1 --2- 172.21.15.150:0/50450129 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x55d9832853b0 0x55d983287800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.455+0000 7f2af77fe700 10 client.25568 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:05:16.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.455+0000 7f2b17fff700 1 --2- 172.21.15.150:0/50450129 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x55d9832853b0 0x55d983287800 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f657f076b57c3639 server_cookie=caec41198d9e3e82 in_seq=0 out_seq=0 2024-05-10T04:05:16.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2af77fe700 1 -- 172.21.15.150:0/50450129 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d98322d780 con 0x55d9832853b0 2024-05-10T04:05:16.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2af77fe700 10 client.25568 handle_command_reply: tid=0 2024-05-10T04:05:16.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2b2e971b80 1 client.25568 shutdown 2024-05-10T04:05:16.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2b2e971b80 1 -- 172.21.15.150:0/50450129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325fec0 msgr2=0x55d98326fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/50450129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325fec0 0x55d98326fad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2b100a1070 tx=0x7f2b10098c40 comp rx=0 tx=0).stop 2024-05-10T04:05:16.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2b2e971b80 1 -- 172.21.15.150:0/50450129 shutdown_connections 2024-05-10T04:05:16.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/50450129 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x55d9832853b0 0x55d983287800 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/50450129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325fec0 0x55d98326fad0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/50450129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325e6a0 0x55d983268630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2af77fe700 0 client.25568 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:05:16.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/50450129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325f570 0x55d983268b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2b2e971b80 1 -- 172.21.15.150:0/50450129 >> 172.21.15.150:0/50450129 conn(0x55d9831c0a60 msgr2=0x55d983258fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2b2e971b80 1 -- 172.21.15.150:0/50450129 shutdown_connections 2024-05-10T04:05:16.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2b2e971b80 1 -- 172.21.15.150:0/50450129 wait complete. 2024-05-10T04:05:16.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.456+0000 7f2b2e971b80 20 client.25568 trim_cache size 0 max 16384 2024-05-10T04:05:16.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:16 smithi150 ceph-mon[42477]: pgmap v310: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T04:05:16.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:16 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2019472348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:05:16.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.470+0000 7f2b2e971b80 1 Processor -- start 2024-05-10T04:05:16.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.470+0000 7f2b2e971b80 1 -- start start 2024-05-10T04:05:16.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.470+0000 7f2b2e971b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325e6a0 0x55d98325f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.470+0000 7f2b2e971b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325f570 0x55d98326fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.470+0000 7f2b2e971b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325fec0 0x55d983259b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.470+0000 7f2b2e971b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9832537c0 con 0x55d98325fec0 2024-05-10T04:05:16.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.470+0000 7f2b2e971b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98325ef50 con 0x55d98325e6a0 2024-05-10T04:05:16.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.470+0000 7f2b2e971b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9832850b0 con 0x55d98325f570 2024-05-10T04:05:16.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.470+0000 7f2b1cbca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325fec0 0x55d983259b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.470+0000 7f2b1cbca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325fec0 0x55d983259b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50568/0 (socket says 172.21.15.150:50568) 2024-05-10T04:05:16.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.470+0000 7f2b1cbca700 1 -- 172.21.15.150:0/1131176683 learned_addr learned my addr 172.21.15.150:0/1131176683 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.470+0000 7f2b17fff700 1 --2- 172.21.15.150:0/1131176683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325e6a0 0x55d98325f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.470+0000 7f2b177fe700 1 --2- 172.21.15.150:0/1131176683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325f570 0x55d98326fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.471+0000 7f2b1cbca700 1 -- 172.21.15.150:0/1131176683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325f570 msgr2=0x55d98326fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.471+0000 7f2b1cbca700 1 --2- 172.21.15.150:0/1131176683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325f570 0x55d98326fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.471+0000 7f2b1cbca700 1 -- 172.21.15.150:0/1131176683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325e6a0 msgr2=0x55d98325f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.471+0000 7f2b1cbca700 1 --2- 172.21.15.150:0/1131176683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325e6a0 0x55d98325f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.471+0000 7f2b1cbca700 1 -- 172.21.15.150:0/1131176683 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d983144560 con 0x55d98325fec0 2024-05-10T04:05:16.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.471+0000 7f2b1cbca700 1 --2- 172.21.15.150:0/1131176683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325fec0 0x55d983259b70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2b10095bd0 tx=0x7f2b100954f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4869e48b9e90c3b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.471+0000 7f2afeffd700 1 -- 172.21.15.150:0/1131176683 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b100a15a0 con 0x55d98325fec0 2024-05-10T04:05:16.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.471+0000 7f2afeffd700 1 -- 172.21.15.150:0/1131176683 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b1009cfe0 con 0x55d98325fec0 2024-05-10T04:05:16.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.471+0000 7f2afeffd700 1 -- 172.21.15.150:0/1131176683 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10099410 con 0x55d98325fec0 2024-05-10T04:05:16.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.472+0000 7f2b2e971b80 1 -- 172.21.15.150:0/1131176683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325fec0 msgr2=0x55d983259b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.472+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/1131176683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325fec0 0x55d983259b70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2b10095bd0 tx=0x7f2b100954f0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.472+0000 7f2b2e971b80 1 -- 172.21.15.150:0/1131176683 shutdown_connections 2024-05-10T04:05:16.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.472+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/1131176683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325fec0 0x55d983259b70 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.472+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/1131176683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325e6a0 0x55d98325f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.472+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/1131176683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325f570 0x55d98326fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.472+0000 7f2b2e971b80 1 -- 172.21.15.150:0/1131176683 >> 172.21.15.150:0/1131176683 conn(0x55d9831c0a60 msgr2=0x55d983255180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.472+0000 7f2b2e971b80 1 -- 172.21.15.150:0/1131176683 shutdown_connections 2024-05-10T04:05:16.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.472+0000 7f2b2e971b80 1 -- 172.21.15.150:0/1131176683 wait complete. 2024-05-10T04:05:16.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.472+0000 7f2b2e971b80 1 -- start start 2024-05-10T04:05:16.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b2e971b80 1 Processor -- start 2024-05-10T04:05:16.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b2e971b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325e6a0 0x55d98326cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b2e971b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325f570 0x55d98326d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b2e971b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325fec0 0x55d98326e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b2e971b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98326e610 con 0x55d98325e6a0 2024-05-10T04:05:16.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b2e971b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98326e780 con 0x55d98325f570 2024-05-10T04:05:16.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b2e971b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98326e940 con 0x55d98325fec0 2024-05-10T04:05:16.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b2e971b80 10 client.0 _ll_register_callbacks cb 0x55d983123b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:16.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b177fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325f570 0x55d98326d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b1cbca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325fec0 0x55d98326e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b177fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325f570 0x55d98326d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:43926/0 (socket says 172.21.15.150:43926) 2024-05-10T04:05:16.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b177fe700 1 -- 172.21.15.150:0/3760046298 learned_addr learned my addr 172.21.15.150:0/3760046298 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b17fff700 1 --2- 172.21.15.150:0/3760046298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325e6a0 0x55d98326cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b177fe700 1 -- 172.21.15.150:0/3760046298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325fec0 msgr2=0x55d98326e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.473+0000 7f2b177fe700 1 --2- 172.21.15.150:0/3760046298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325fec0 0x55d98326e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.474+0000 7f2b177fe700 1 -- 172.21.15.150:0/3760046298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325e6a0 msgr2=0x55d98326cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.474+0000 7f2b177fe700 1 --2- 172.21.15.150:0/3760046298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325e6a0 0x55d98326cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.474+0000 7f2b177fe700 1 -- 172.21.15.150:0/3760046298 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d982fabb60 con 0x55d98325f570 2024-05-10T04:05:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.474+0000 7f2b1cbca700 1 --2- 172.21.15.150:0/3760046298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325fec0 0x55d98326e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.474+0000 7f2b177fe700 1 --2- 172.21.15.150:0/3760046298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325f570 0x55d98326d1a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009990 tx=0x7f2b0c00b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8384096a14fe4916 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.474+0000 7f2af77fe700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:05:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.474+0000 7f2af77fe700 1 -- 172.21.15.150:0/3760046298 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c031070 con 0x55d98325f570 2024-05-10T04:05:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.474+0000 7f2b2e971b80 1 -- 172.21.15.150:0/3760046298 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b0c01e040 con 0x55d98325f570 2024-05-10T04:05:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.474+0000 7f2af77fe700 1 -- 172.21.15.150:0/3760046298 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2b0c00f730 con 0x55d98325f570 2024-05-10T04:05:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.474+0000 7f2af77fe700 1 -- 172.21.15.150:0/3760046298 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b0c00f8f0 con 0x55d98325f570 2024-05-10T04:05:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.474+0000 7f2b2e971b80 10 client.25574 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.474+0000 7f2b2e971b80 1 -- 172.21.15.150:0/3760046298 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d983003cd0 con 0x55d98325f570 2024-05-10T04:05:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.474+0000 7f2af77fe700 1 -- 172.21.15.150:0/3760046298 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b0c01e040 con 0x55d98325f570 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.475+0000 7f2af77fe700 1 -- 172.21.15.150:0/3760046298 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2b0c021070 con 0x55d98325f570 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.475+0000 7f2b2e971b80 10 client.25574 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.475+0000 7f2b2e971b80 10 client.25574 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.475+0000 7f2b2e971b80 20 client.25574 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.475+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/3760046298 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x55d9832db880 0x55d9832ddca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.475+0000 7f2b2e971b80 4 client.25574 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.475+0000 7f2b2e971b80 1 -- 172.21.15.150:0/3760046298 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d983176ec0 con 0x55d9832db880 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.475+0000 7f2b17fff700 1 --2- 172.21.15.150:0/3760046298 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x55d9832db880 0x55d9832ddca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2b17fff700 1 --2- 172.21.15.150:0/3760046298 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x55d9832db880 0x55d9832ddca0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9eaf678d20b3c5b8 server_cookie=1425917c8c4f19ce in_seq=0 out_seq=0 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2af77fe700 10 client.25574 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2af77fe700 1 -- 172.21.15.150:0/3760046298 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d983176ec0 con 0x55d9832db880 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2af77fe700 10 client.25574 handle_command_reply: tid=0 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2b2e971b80 1 client.25574 shutdown 2024-05-10T04:05:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2b2e971b80 1 -- 172.21.15.150:0/3760046298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325f570 msgr2=0x55d98326d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/3760046298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325f570 0x55d98326d1a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009990 tx=0x7f2b0c00b400 comp rx=0 tx=0).stop 2024-05-10T04:05:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2b2e971b80 1 -- 172.21.15.150:0/3760046298 shutdown_connections 2024-05-10T04:05:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/3760046298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98325fec0 0x55d98326e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/3760046298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d98325e6a0 0x55d98326cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/3760046298 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x55d9832db880 0x55d9832ddca0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2b2e971b80 1 --2- 172.21.15.150:0/3760046298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d98325f570 0x55d98326d1a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2b2e971b80 1 -- 172.21.15.150:0/3760046298 >> 172.21.15.150:0/3760046298 conn(0x55d9831c0a60 msgr2=0x55d983255180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2af77fe700 0 client.25574 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:05:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2b2e971b80 1 -- 172.21.15.150:0/3760046298 shutdown_connections 2024-05-10T04:05:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.476+0000 7f2b2e971b80 1 -- 172.21.15.150:0/3760046298 wait complete. 2024-05-10T04:05:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.477+0000 7f2b2e971b80 20 client.25574 trim_cache size 0 max 16384 2024-05-10T04:05:16.482 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:05:16.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.477+0000 7f2b1e62d700 1 -- 172.21.15.150:0/3846603279 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2b000a1260 msgr2=0x7f2b000a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.477+0000 7f2b1e62d700 1 --2- 172.21.15.150:0/3846603279 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2b000a1260 0x7f2b000a3770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009f40 tx=0x7f2b08009260 comp rx=0 tx=0).stop 2024-05-10T04:05:16.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.477+0000 7f2b1e62d700 1 -- 172.21.15.150:0/3846603279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b18101440 msgr2=0x7f2b181a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.477+0000 7f2b1e62d700 1 --2- 172.21.15.150:0/3846603279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b18101440 0x7f2b181a59a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000ab40 tx=0x7f2b1000ae50 comp rx=0 tx=0).stop 2024-05-10T04:05:16.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.477+0000 7f2b1e62d700 1 -- 172.21.15.150:0/3846603279 shutdown_connections 2024-05-10T04:05:16.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.477+0000 7f2b1e62d700 1 --2- 172.21.15.150:0/3846603279 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2b000a1260 0x7f2b000a3770 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.477+0000 7f2b1e62d700 1 --2- 172.21.15.150:0/3846603279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b18101440 0x7f2b181a59a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.477+0000 7f2b1e62d700 1 --2- 172.21.15.150:0/3846603279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b180ffe80 0x7f2b1819e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.477+0000 7f2b1e62d700 1 --2- 172.21.15.150:0/3846603279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b181008e0 0x7f2b1819ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.477+0000 7f2b1e62d700 1 -- 172.21.15.150:0/3846603279 >> 172.21.15.150:0/3846603279 conn(0x7f2b180fb8b0 msgr2=0x7f2b18101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.478+0000 7f2b1e62d700 1 -- 172.21.15.150:0/3846603279 shutdown_connections 2024-05-10T04:05:16.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.478+0000 7f2b1e62d700 1 -- 172.21.15.150:0/3846603279 wait complete. 2024-05-10T04:05:16.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:16 smithi181 ceph-mon[46602]: pgmap v310: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T04:05:16.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:16 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2019472348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:05:16.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:16 smithi161 ceph-mon[46918]: pgmap v310: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T04:05:16.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:16 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2019472348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:05:16.496 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T04:05:16.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.650+0000 7f857628d700 1 -- 172.21.15.150:0/3096538754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8570101680 msgr2=0x7f8570108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.650+0000 7f857628d700 1 --2- 172.21.15.150:0/3096538754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8570101680 0x7f8570108f90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f856800b0a0 tx=0x7f856800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.650+0000 7f857628d700 1 -- 172.21.15.150:0/3096538754 shutdown_connections 2024-05-10T04:05:16.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.650+0000 7f857628d700 1 --2- 172.21.15.150:0/3096538754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8570101680 0x7f8570108f90 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.650+0000 7f857628d700 1 --2- 172.21.15.150:0/3096538754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8570102030 0x7f8570102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.651+0000 7f857628d700 1 --2- 172.21.15.150:0/3096538754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8570100d30 0x7f8570101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.651+0000 7f857628d700 1 -- 172.21.15.150:0/3096538754 >> 172.21.15.150:0/3096538754 conn(0x7f85700fb8f0 msgr2=0x7f85700fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.651+0000 7f857628d700 1 -- 172.21.15.150:0/3096538754 shutdown_connections 2024-05-10T04:05:16.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.651+0000 7f857628d700 1 -- 172.21.15.150:0/3096538754 wait complete. 2024-05-10T04:05:16.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.651+0000 7f857628d700 1 Processor -- start 2024-05-10T04:05:16.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.651+0000 7f857628d700 1 -- start start 2024-05-10T04:05:16.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857628d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8570100d30 0x7f8570198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857628d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8570101680 0x7f8570198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857628d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8570102030 0x7f857019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857628d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857019faf0 con 0x7f8570102030 2024-05-10T04:05:16.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857628d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857019fc60 con 0x7f8570101680 2024-05-10T04:05:16.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857628d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857019fe20 con 0x7f8570100d30 2024-05-10T04:05:16.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857482a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8570102030 0x7f857019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857482a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8570102030 0x7f857019f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50602/0 (socket says 172.21.15.150:50602) 2024-05-10T04:05:16.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857482a700 1 -- 172.21.15.150:0/2849930337 learned_addr learned my addr 172.21.15.150:0/2849930337 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f856f7fe700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8570101680 0x7f8570198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f856ffff700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8570100d30 0x7f8570198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857482a700 1 -- 172.21.15.150:0/2849930337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8570100d30 msgr2=0x7f8570198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857482a700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8570100d30 0x7f8570198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857482a700 1 -- 172.21.15.150:0/2849930337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8570101680 msgr2=0x7f8570198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857482a700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8570101680 0x7f8570198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.652+0000 7f857482a700 1 -- 172.21.15.150:0/2849930337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8564009520 con 0x7f8570102030 2024-05-10T04:05:16.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.653+0000 7f856f7fe700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8570101680 0x7f8570198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:16.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.653+0000 7f857482a700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8570102030 0x7f857019f4e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f856800b070 tx=0x7f85680078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.653+0000 7f856d7fa700 1 -- 172.21.15.150:0/2849930337 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856800e040 con 0x7f8570102030 2024-05-10T04:05:16.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.653+0000 7f857628d700 1 -- 172.21.15.150:0/2849930337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8568009d00 con 0x7f8570102030 2024-05-10T04:05:16.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.653+0000 7f856d7fa700 1 -- 172.21.15.150:0/2849930337 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8568003c00 con 0x7f8570102030 2024-05-10T04:05:16.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.653+0000 7f856d7fa700 1 -- 172.21.15.150:0/2849930337 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856801b6c0 con 0x7f8570102030 2024-05-10T04:05:16.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.653+0000 7f857628d700 1 -- 172.21.15.150:0/2849930337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8570109bb0 con 0x7f8570102030 2024-05-10T04:05:16.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.656+0000 7f856d7fa700 1 -- 172.21.15.150:0/2849930337 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f856801d470 con 0x7f8570102030 2024-05-10T04:05:16.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.657+0000 7f856d7fa700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f85580a1260 0x7f85580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.657+0000 7f856d7fa700 1 -- 172.21.15.150:0/2849930337 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f85680cd8a0 con 0x7f8570102030 2024-05-10T04:05:16.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.661+0000 7f85865d1b80 1 Processor -- start 2024-05-10T04:05:16.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.661+0000 7f85865d1b80 1 -- start start 2024-05-10T04:05:16.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.661+0000 7f85865d1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e2eb0 0x563fab9e3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.661+0000 7f85865d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e3d80 0x563fab9e4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.661+0000 7f85865d1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9eafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.661+0000 7f85865d1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fab901150 con 0x563fab9e46d0 2024-05-10T04:05:16.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.661+0000 7f85865d1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fab8feb10 con 0x563fab9e2eb0 2024-05-10T04:05:16.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.661+0000 7f85865d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fab8fa400 con 0x563fab9e3d80 2024-05-10T04:05:16.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.661+0000 7f856ffff700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f85580a1260 0x7f85580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.662+0000 7f856f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e3d80 0x563fab9e4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.662+0000 7f856ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e2eb0 0x563fab9e3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.662+0000 7f856ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e2eb0 0x563fab9e3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:43936/0 (socket says 172.21.15.150:43936) 2024-05-10T04:05:16.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.662+0000 7f856ffff700 1 -- 172.21.15.150:0/814263413 learned_addr learned my addr 172.21.15.150:0/814263413 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.662+0000 7f857482a700 1 --2- 172.21.15.150:0/814263413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9eafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.662+0000 7f856ffff700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f85580a1260 0x7f85580a3770 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8560014410 tx=0x7f85600147d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.662+0000 7f856ffff700 1 -- 172.21.15.150:0/814263413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e3d80 msgr2=0x563fab9e4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.662+0000 7f856ffff700 1 --2- 172.21.15.150:0/814263413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e3d80 0x563fab9e4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.662+0000 7f856ffff700 1 -- 172.21.15.150:0/814263413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 msgr2=0x563fab9eafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.662+0000 7f856ffff700 1 --2- 172.21.15.150:0/814263413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9eafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.662+0000 7f856ffff700 1 -- 172.21.15.150:0/814263413 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fab77b3b0 con 0x563fab9e2eb0 2024-05-10T04:05:16.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.663+0000 7f856ffff700 1 --2- 172.21.15.150:0/814263413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e2eb0 0x563fab9e3300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8560014fd0 tx=0x7f856001e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1ced7835e15ac27 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.663+0000 7f85567fc700 1 -- 172.21.15.150:0/814263413 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560012de0 con 0x563fab9e2eb0 2024-05-10T04:05:16.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.663+0000 7f85567fc700 1 -- 172.21.15.150:0/814263413 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f856001a070 con 0x563fab9e2eb0 2024-05-10T04:05:16.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.663+0000 7f85567fc700 1 -- 172.21.15.150:0/814263413 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560005020 con 0x563fab9e2eb0 2024-05-10T04:05:16.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.664+0000 7f85865d1b80 1 -- 172.21.15.150:0/814263413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e2eb0 msgr2=0x563fab9e3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.664+0000 7f85865d1b80 1 --2- 172.21.15.150:0/814263413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e2eb0 0x563fab9e3300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8560014fd0 tx=0x7f856001e000 comp rx=0 tx=0).stop 2024-05-10T04:05:16.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.664+0000 7f85865d1b80 1 -- 172.21.15.150:0/814263413 shutdown_connections 2024-05-10T04:05:16.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.664+0000 7f85865d1b80 1 --2- 172.21.15.150:0/814263413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9eafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.664+0000 7f85865d1b80 1 --2- 172.21.15.150:0/814263413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e2eb0 0x563fab9e3300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.664+0000 7f85865d1b80 1 --2- 172.21.15.150:0/814263413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e3d80 0x563fab9e4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.664+0000 7f85865d1b80 1 -- 172.21.15.150:0/814263413 >> 172.21.15.150:0/814263413 conn(0x563fab93ca60 msgr2=0x563fab9d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.664+0000 7f85865d1b80 1 -- 172.21.15.150:0/814263413 shutdown_connections 2024-05-10T04:05:16.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.664+0000 7f85865d1b80 1 -- 172.21.15.150:0/814263413 wait complete. 2024-05-10T04:05:16.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.665+0000 7f85865d1b80 1 -- start start 2024-05-10T04:05:16.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.665+0000 7f85865d1b80 1 Processor -- start 2024-05-10T04:05:16.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.666+0000 7f85865d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e2eb0 0x563fab9ece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.666+0000 7f85865d1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e3d80 0x563fab9ed390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.666+0000 7f85865d1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.666+0000 7f85865d1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fab896560 con 0x563fab9e46d0 2024-05-10T04:05:16.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.666+0000 7f85865d1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fab896200 con 0x563fab9e3d80 2024-05-10T04:05:16.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.666+0000 7f85865d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fab77d6a0 con 0x563fab9e2eb0 2024-05-10T04:05:16.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.667+0000 7f85865d1b80 10 client.0 _ll_register_callbacks cb 0x563fab89fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.667+0000 7f857482a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.667+0000 7f857482a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9f42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50622/0 (socket says 172.21.15.150:50622) 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.667+0000 7f857482a700 1 -- 172.21.15.150:0/2094739701 learned_addr learned my addr 172.21.15.150:0/2094739701 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.667+0000 7f856f7fe700 1 --2- 172.21.15.150:0/2094739701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e3d80 0x563fab9ed390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.667+0000 7f856ffff700 1 --2- 172.21.15.150:0/2094739701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e2eb0 0x563fab9ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.667+0000 7f856f7fe700 1 -- 172.21.15.150:0/2094739701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e2eb0 msgr2=0x563fab9ece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.667+0000 7f856f7fe700 1 --2- 172.21.15.150:0/2094739701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e2eb0 0x563fab9ece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.667+0000 7f856f7fe700 1 -- 172.21.15.150:0/2094739701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 msgr2=0x563fab9f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.667+0000 7f856f7fe700 1 --2- 172.21.15.150:0/2094739701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.667+0000 7f856f7fe700 1 -- 172.21.15.150:0/2094739701 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8560014e40 con 0x563fab9e3d80 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.667+0000 7f856f7fe700 1 --2- 172.21.15.150:0/2094739701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e3d80 0x563fab9ed390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f85701990b0 tx=0x7f856400f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f92d0de617f8ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.667+0000 7f854f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.668+0000 7f854f7fe700 1 -- 172.21.15.150:0/2094739701 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564017070 con 0x563fab9e3d80 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.668+0000 7f85865d1b80 1 -- 172.21.15.150:0/2094739701 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fab853270 con 0x563fab9e3d80 2024-05-10T04:05:16.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.668+0000 7f854f7fe700 1 -- 172.21.15.150:0/2094739701 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f856400fa10 con 0x563fab9e3d80 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.668+0000 7f854f7fe700 1 -- 172.21.15.150:0/2094739701 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f856400fbd0 con 0x563fab9e3d80 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.668+0000 7f85865d1b80 10 client.25586 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.668+0000 7f85865d1b80 1 -- 172.21.15.150:0/2094739701 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563fab727b60 con 0x563fab9e3d80 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.668+0000 7f854f7fe700 1 -- 172.21.15.150:0/2094739701 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563fab853270 con 0x563fab9e3d80 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.668+0000 7f854f7fe700 1 -- 172.21.15.150:0/2094739701 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8564003a30 con 0x563fab9e3d80 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.669+0000 7f85865d1b80 10 client.25586 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.669+0000 7f85865d1b80 10 client.25586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.669+0000 7f85865d1b80 20 client.25586 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.669+0000 7f85865d1b80 1 --2- 172.21.15.150:0/2094739701 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563fab9f8aa0 0x563fab9faef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.669+0000 7f85865d1b80 4 client.25586 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.669+0000 7f85865d1b80 1 -- 172.21.15.150:0/2094739701 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fab9a9780 con 0x563fab9f8aa0 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.670+0000 7f856ffff700 1 --2- 172.21.15.150:0/2094739701 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563fab9f8aa0 0x563fab9faef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.670+0000 7f856ffff700 1 --2- 172.21.15.150:0/2094739701 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563fab9f8aa0 0x563fab9faef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65cd4f5a7cf5f054 server_cookie=8ab82014d1a61d42 in_seq=0 out_seq=0 2024-05-10T04:05:16.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.670+0000 7f854f7fe700 10 client.25586 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:16.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f854f7fe700 1 -- 172.21.15.150:0/2094739701 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563fab9a9780 con 0x563fab9f8aa0 2024-05-10T04:05:16.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f854f7fe700 10 client.25586 handle_command_reply: tid=0 2024-05-10T04:05:16.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f85865d1b80 1 client.25586 shutdown 2024-05-10T04:05:16.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f85865d1b80 1 -- 172.21.15.150:0/2094739701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e3d80 msgr2=0x563fab9ed390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f85865d1b80 1 --2- 172.21.15.150:0/2094739701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e3d80 0x563fab9ed390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f85701990b0 tx=0x7f856400f3c0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f85865d1b80 1 -- 172.21.15.150:0/2094739701 shutdown_connections 2024-05-10T04:05:16.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f85865d1b80 1 --2- 172.21.15.150:0/2094739701 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563fab9f8aa0 0x563fab9faef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f85865d1b80 1 --2- 172.21.15.150:0/2094739701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f854f7fe700 0 client.25586 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:16.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f85865d1b80 1 --2- 172.21.15.150:0/2094739701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e2eb0 0x563fab9ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f85865d1b80 1 --2- 172.21.15.150:0/2094739701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e3d80 0x563fab9ed390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f85865d1b80 1 -- 172.21.15.150:0/2094739701 >> 172.21.15.150:0/2094739701 conn(0x563fab93ca60 msgr2=0x563fab9d4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f85865d1b80 1 -- 172.21.15.150:0/2094739701 shutdown_connections 2024-05-10T04:05:16.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.671+0000 7f85865d1b80 1 -- 172.21.15.150:0/2094739701 wait complete. 2024-05-10T04:05:16.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.672+0000 7f85865d1b80 20 client.25586 trim_cache size 0 max 16384 2024-05-10T04:05:16.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.696+0000 7f85865d1b80 1 Processor -- start 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.696+0000 7f85865d1b80 1 -- start start 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.696+0000 7f85865d1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e2eb0 0x563fab9faec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.696+0000 7f85865d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e3d80 0x563fab9d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.696+0000 7f85865d1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.696+0000 7f85865d1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fab9d6770 con 0x563fab9e46d0 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.696+0000 7f85865d1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fab9cf7c0 con 0x563fab9e2eb0 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.696+0000 7f85865d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fab9eae80 con 0x563fab9e3d80 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.696+0000 7f856f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e3d80 0x563fab9d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.697+0000 7f856f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e3d80 0x563fab9d1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:57976/0 (socket says 172.21.15.150:57976) 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.697+0000 7f856f7fe700 1 -- 172.21.15.150:0/2528923825 learned_addr learned my addr 172.21.15.150:0/2528923825 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.697+0000 7f857482a700 1 --2- 172.21.15.150:0/2528923825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.697+0000 7f856f7fe700 1 -- 172.21.15.150:0/2528923825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e2eb0 msgr2=0x563fab9faec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.697+0000 7f856f7fe700 1 --2- 172.21.15.150:0/2528923825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e2eb0 0x563fab9faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.697+0000 7f856f7fe700 1 -- 172.21.15.150:0/2528923825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 msgr2=0x563fab9e7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.697+0000 7f856f7fe700 1 --2- 172.21.15.150:0/2528923825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9e7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.697+0000 7f856f7fe700 1 -- 172.21.15.150:0/2528923825 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fab8c0560 con 0x563fab9e3d80 2024-05-10T04:05:16.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.697+0000 7f857482a700 1 --2- 172.21.15.150:0/2528923825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9e7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:16.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.697+0000 7f856f7fe700 1 --2- 172.21.15.150:0/2528923825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e3d80 0x563fab9d1180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8564001eb0 tx=0x7f856400f7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5188d8561843e517 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.697+0000 7f8556ffd700 1 -- 172.21.15.150:0/2528923825 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564017040 con 0x563fab9e3d80 2024-05-10T04:05:16.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.697+0000 7f8556ffd700 1 -- 172.21.15.150:0/2528923825 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8564003b40 con 0x563fab9e3d80 2024-05-10T04:05:16.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.697+0000 7f8556ffd700 1 -- 172.21.15.150:0/2528923825 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564010890 con 0x563fab9e3d80 2024-05-10T04:05:16.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.698+0000 7f85865d1b80 1 -- 172.21.15.150:0/2528923825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e3d80 msgr2=0x563fab9d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.698+0000 7f85865d1b80 1 --2- 172.21.15.150:0/2528923825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e3d80 0x563fab9d1180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8564001eb0 tx=0x7f856400f7c0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.698+0000 7f85865d1b80 1 -- 172.21.15.150:0/2528923825 shutdown_connections 2024-05-10T04:05:16.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.698+0000 7f85865d1b80 1 --2- 172.21.15.150:0/2528923825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e46d0 0x563fab9e7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.698+0000 7f85865d1b80 1 --2- 172.21.15.150:0/2528923825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e2eb0 0x563fab9faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.698+0000 7f85865d1b80 1 --2- 172.21.15.150:0/2528923825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e3d80 0x563fab9d1180 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.698+0000 7f85865d1b80 1 -- 172.21.15.150:0/2528923825 >> 172.21.15.150:0/2528923825 conn(0x563fab93ca60 msgr2=0x563fab9d4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.698+0000 7f85865d1b80 1 -- 172.21.15.150:0/2528923825 shutdown_connections 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.698+0000 7f85865d1b80 1 -- 172.21.15.150:0/2528923825 wait complete. 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.699+0000 7f85865d1b80 1 -- start start 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.699+0000 7f85865d1b80 1 Processor -- start 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.699+0000 7f85865d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e2eb0 0x563fab9f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.699+0000 7f85865d1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e3d80 0x563fab9f2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.699+0000 7f85865d1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e46d0 0x563fab9f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.699+0000 7f85865d1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fab9f37f0 con 0x563fab9e3d80 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.699+0000 7f85865d1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fab9f3960 con 0x563fab9e46d0 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.699+0000 7f85865d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fab9f3b20 con 0x563fab9e2eb0 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.699+0000 7f85865d1b80 10 client.0 _ll_register_callbacks cb 0x563fab89fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.699+0000 7f856ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e2eb0 0x563fab9f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.699+0000 7f856ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e2eb0 0x563fab9f1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:57988/0 (socket says 172.21.15.150:57988) 2024-05-10T04:05:16.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.699+0000 7f856ffff700 1 -- 172.21.15.150:0/133549637 learned_addr learned my addr 172.21.15.150:0/133549637 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f857482a700 1 --2- 172.21.15.150:0/133549637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e46d0 0x563fab9f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f856ffff700 1 -- 172.21.15.150:0/133549637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e46d0 msgr2=0x563fab9f32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f856ffff700 1 --2- 172.21.15.150:0/133549637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e46d0 0x563fab9f32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f856ffff700 1 -- 172.21.15.150:0/133549637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e3d80 msgr2=0x563fab9f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f856ffff700 1 --2- 172.21.15.150:0/133549637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e3d80 0x563fab9f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f856ffff700 1 -- 172.21.15.150:0/133549637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fab727b60 con 0x563fab9e2eb0 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f857482a700 1 --2- 172.21.15.150:0/133549637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e46d0 0x563fab9f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f856ffff700 1 --2- 172.21.15.150:0/133549637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e2eb0 0x563fab9f1e40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8560012480 tx=0x7f8560005c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c738fa11be8f116 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f854f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f854f7fe700 1 -- 172.21.15.150:0/133549637 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856002d040 con 0x563fab9e2eb0 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f85865d1b80 1 -- 172.21.15.150:0/133549637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f856001a070 con 0x563fab9e2eb0 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f854f7fe700 1 -- 172.21.15.150:0/133549637 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8560012de0 con 0x563fab9e2eb0 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f854f7fe700 1 -- 172.21.15.150:0/133549637 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8560009990 con 0x563fab9e2eb0 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.700+0000 7f85865d1b80 10 client.25534 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.701+0000 7f85865d1b80 1 -- 172.21.15.150:0/133549637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563fab77fcd0 con 0x563fab9e2eb0 2024-05-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.701+0000 7f854f7fe700 1 -- 172.21.15.150:0/133549637 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f856001a070 con 0x563fab9e2eb0 2024-05-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.701+0000 7f854f7fe700 1 -- 172.21.15.150:0/133549637 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f856003a020 con 0x563fab9e2eb0 2024-05-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.701+0000 7f85865d1b80 10 client.25534 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.701+0000 7f85865d1b80 10 client.25534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.701+0000 7f85865d1b80 20 client.25534 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.701+0000 7f85865d1b80 1 --2- 172.21.15.150:0/133549637 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563faba4f240 0x563faba51660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.701+0000 7f85865d1b80 4 client.25534 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.701+0000 7f85865d1b80 1 -- 172.21.15.150:0/133549637 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563fab8f2ec0 con 0x563faba4f240 2024-05-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.701+0000 7f856f7fe700 1 --2- 172.21.15.150:0/133549637 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563faba4f240 0x563faba51660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f856f7fe700 1 --2- 172.21.15.150:0/133549637 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563faba4f240 0x563faba51660 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2de249260f1551b server_cookie=1ef12698d875d49e in_seq=0 out_seq=0 2024-05-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f854f7fe700 10 client.25534 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f854f7fe700 1 -- 172.21.15.150:0/133549637 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563fab8f2ec0 con 0x563faba4f240 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f854f7fe700 10 client.25534 handle_command_reply: tid=0 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f85865d1b80 1 client.25534 shutdown 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f85865d1b80 1 -- 172.21.15.150:0/133549637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e2eb0 msgr2=0x563fab9f1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f85865d1b80 1 --2- 172.21.15.150:0/133549637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e2eb0 0x563fab9f1e40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8560012480 tx=0x7f8560005c40 comp rx=0 tx=0).stop 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f85865d1b80 1 -- 172.21.15.150:0/133549637 shutdown_connections 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f85865d1b80 1 --2- 172.21.15.150:0/133549637 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563faba4f240 0x563faba51660 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f85865d1b80 1 --2- 172.21.15.150:0/133549637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fab9e46d0 0x563fab9f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f85865d1b80 1 --2- 172.21.15.150:0/133549637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563fab9e2eb0 0x563fab9f1e40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f85865d1b80 1 --2- 172.21.15.150:0/133549637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fab9e3d80 0x563fab9f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f85865d1b80 1 -- 172.21.15.150:0/133549637 >> 172.21.15.150:0/133549637 conn(0x563fab93ca60 msgr2=0x563fab9ec9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f854f7fe700 0 client.25534 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f85865d1b80 1 -- 172.21.15.150:0/133549637 shutdown_connections 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.702+0000 7f85865d1b80 1 -- 172.21.15.150:0/133549637 wait complete. 2024-05-10T04:05:16.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.703+0000 7f85865d1b80 20 client.25534 trim_cache size 0 max 16384 2024-05-10T04:05:16.710 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:05:16.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.703+0000 7f857628d700 1 -- 172.21.15.150:0/2849930337 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f85580a1260 msgr2=0x7f85580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.703+0000 7f857628d700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f85580a1260 0x7f85580a3770 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8560014410 tx=0x7f85600147d0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.703+0000 7f857628d700 1 -- 172.21.15.150:0/2849930337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8570102030 msgr2=0x7f857019f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:16.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.703+0000 7f857628d700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8570102030 0x7f857019f4e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f856800b070 tx=0x7f85680078e0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.703+0000 7f857628d700 1 -- 172.21.15.150:0/2849930337 shutdown_connections 2024-05-10T04:05:16.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.703+0000 7f857628d700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f85580a1260 0x7f85580a3770 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.703+0000 7f857628d700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8570102030 0x7f857019f4e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.703+0000 7f857628d700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8570100d30 0x7f8570198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.703+0000 7f857628d700 1 --2- 172.21.15.150:0/2849930337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8570101680 0x7f8570198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:16.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.703+0000 7f857628d700 1 -- 172.21.15.150:0/2849930337 >> 172.21.15.150:0/2849930337 conn(0x7f85700fb8f0 msgr2=0x7f8570104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:16.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.703+0000 7f857628d700 1 -- 172.21.15.150:0/2849930337 shutdown_connections 2024-05-10T04:05:16.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:16.703+0000 7f857628d700 1 -- 172.21.15.150:0/2849930337 wait complete. 2024-05-10T04:05:17.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:17 smithi150 ceph-mon[42477]: pgmap v311: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T04:05:17.718 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T04:05:17.719 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T04:05:17.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:17 smithi181 ceph-mon[46602]: pgmap v311: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T04:05:17.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:17 smithi161 ceph-mon[46918]: pgmap v311: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T04:05:17.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.875+0000 7f13d0e54700 1 -- 172.21.15.150:0/4185822175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13cc100ad0 msgr2=0x7f13cc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:17.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.875+0000 7f13d0e54700 1 --2- 172.21.15.150:0/4185822175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13cc100ad0 0x7f13cc100ee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009990 tx=0x7f13bc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.875+0000 7f13d0e54700 1 -- 172.21.15.150:0/4185822175 shutdown_connections 2024-05-10T04:05:17.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.875+0000 7f13d0e54700 1 --2- 172.21.15.150:0/4185822175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13cc102590 0x7f13cc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.875+0000 7f13d0e54700 1 --2- 172.21.15.150:0/4185822175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13cc100ad0 0x7f13cc100ee0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.875+0000 7f13d0e54700 1 --2- 172.21.15.150:0/4185822175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13cc101bf0 0x7f13cc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.875+0000 7f13d0e54700 1 -- 172.21.15.150:0/4185822175 >> 172.21.15.150:0/4185822175 conn(0x7f13cc0fc0c0 msgr2=0x7f13cc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:17.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.875+0000 7f13d0e54700 1 -- 172.21.15.150:0/4185822175 shutdown_connections 2024-05-10T04:05:17.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.875+0000 7f13d0e54700 1 -- 172.21.15.150:0/4185822175 wait complete. 2024-05-10T04:05:17.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.876+0000 7f13d0e54700 1 Processor -- start 2024-05-10T04:05:17.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.876+0000 7f13d0e54700 1 -- start start 2024-05-10T04:05:17.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.876+0000 7f13d0e54700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13cc100ad0 0x7f13cc104890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.876+0000 7f13d0e54700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13cc101bf0 0x7f13cc102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.876+0000 7f13d0e54700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13cc102590 0x7f13cc103470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.876+0000 7f13d0e54700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc19be40 con 0x7f13cc101bf0 2024-05-10T04:05:17.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.876+0000 7f13d0e54700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc19bfb0 con 0x7f13cc102590 2024-05-10T04:05:17.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.876+0000 7f13d0e54700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc19c170 con 0x7f13cc100ad0 2024-05-10T04:05:17.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.877+0000 7f13c9d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13cc101bf0 0x7f13cc102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.877+0000 7f13c9d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13cc101bf0 0x7f13cc102f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50660/0 (socket says 172.21.15.150:50660) 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.877+0000 7f13c9d9b700 1 -- 172.21.15.150:0/1890496891 learned_addr learned my addr 172.21.15.150:0/1890496891 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.877+0000 7f13ca59c700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13cc100ad0 0x7f13cc104890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.877+0000 7f13cad9d700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13cc102590 0x7f13cc103470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.877+0000 7f13c9d9b700 1 -- 172.21.15.150:0/1890496891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13cc100ad0 msgr2=0x7f13cc104890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.877+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13cc100ad0 0x7f13cc104890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.877+0000 7f13c9d9b700 1 -- 172.21.15.150:0/1890496891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13cc102590 msgr2=0x7f13cc103470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.877+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13cc102590 0x7f13cc103470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.877+0000 7f13c9d9b700 1 -- 172.21.15.150:0/1890496891 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc0095f0 con 0x7f13cc101bf0 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.877+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13cc101bf0 0x7f13cc102f30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f13c000e9a0 tx=0x7f13c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.878+0000 7f13bb7fe700 1 -- 172.21.15.150:0/1890496891 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0009790 con 0x7f13cc101bf0 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.878+0000 7f13bb7fe700 1 -- 172.21.15.150:0/1890496891 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13c0010430 con 0x7f13cc101bf0 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.878+0000 7f13bb7fe700 1 -- 172.21.15.150:0/1890496891 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0018680 con 0x7f13cc101bf0 2024-05-10T04:05:17.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.878+0000 7f13d0e54700 1 -- 172.21.15.150:0/1890496891 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13cc19c450 con 0x7f13cc101bf0 2024-05-10T04:05:17.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.878+0000 7f13d0e54700 1 -- 172.21.15.150:0/1890496891 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13cc19c910 con 0x7f13cc101bf0 2024-05-10T04:05:17.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.882+0000 7f13e1198b80 1 Processor -- start 2024-05-10T04:05:17.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.882+0000 7f13e1198b80 1 -- start start 2024-05-10T04:05:17.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.882+0000 7f13bb7fe700 1 -- 172.21.15.150:0/1890496891 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f13c000ca10 con 0x7f13cc101bf0 2024-05-10T04:05:17.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.882+0000 7f13e1198b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2fe720 0x5639bb2feb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.882+0000 7f13e1198b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2ff5f0 0x5639bb2ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.882+0000 7f13e1198b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fff40 0x5639bb306840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.882+0000 7f13e1198b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bb2251d0 con 0x5639bb2ff5f0 2024-05-10T04:05:17.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.882+0000 7f13e1198b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bb222b90 con 0x5639bb2fe720 2024-05-10T04:05:17.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.882+0000 7f13e1198b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bb21e480 con 0x5639bb2fff40 2024-05-10T04:05:17.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.882+0000 7f13c9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fff40 0x5639bb306840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.883+0000 7f13c9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fff40 0x5639bb306840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:58016/0 (socket says 172.21.15.150:58016) 2024-05-10T04:05:17.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.883+0000 7f13c9d9b700 1 -- 172.21.15.150:0/244396975 learned_addr learned my addr 172.21.15.150:0/244396975 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:17.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.883+0000 7f13bb7fe700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13b40a1260 0x7f13b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.883+0000 7f13cad9d700 1 --2- 172.21.15.150:0/244396975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2fe720 0x5639bb2feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.883+0000 7f13c9d9b700 1 -- 172.21.15.150:0/244396975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2fe720 msgr2=0x5639bb2feb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:17.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.883+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/244396975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2fe720 0x5639bb2feb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.883+0000 7f13c9d9b700 1 -- 172.21.15.150:0/244396975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2ff5f0 msgr2=0x5639bb2ffa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:17.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.883+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/244396975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2ff5f0 0x5639bb2ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.883+0000 7f13c9d9b700 1 -- 172.21.15.150:0/244396975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639bb09f430 con 0x5639bb2fff40 2024-05-10T04:05:17.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.883+0000 7f13cad9d700 1 --2- 172.21.15.150:0/244396975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2fe720 0x5639bb2feb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:05:17.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.883+0000 7f13cad9d700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13b40a1260 0x7f13b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.883+0000 7f13bb7fe700 1 -- 172.21.15.150:0/1890496891 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f13c0014070 con 0x7f13cc101bf0 2024-05-10T04:05:17.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.883+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/244396975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fff40 0x5639bb306840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f13c00ca520 tx=0x7f13c0024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510f45e03f2d6e24 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:17.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.884+0000 7f13cad9d700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13b40a1260 0x7f13b40a3770 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009550 tx=0x7f13c4004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:17.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.884+0000 7f13b8ff9700 1 -- 172.21.15.150:0/244396975 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0099cf0 con 0x5639bb2fff40 2024-05-10T04:05:17.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.884+0000 7f13b8ff9700 1 -- 172.21.15.150:0/244396975 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13c0099a60 con 0x5639bb2fff40 2024-05-10T04:05:17.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.884+0000 7f13b8ff9700 1 -- 172.21.15.150:0/244396975 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c001b6b0 con 0x5639bb2fff40 2024-05-10T04:05:17.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.885+0000 7f13e1198b80 1 -- 172.21.15.150:0/244396975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fff40 msgr2=0x5639bb306840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:17.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.885+0000 7f13e1198b80 1 --2- 172.21.15.150:0/244396975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fff40 0x5639bb306840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f13c00ca520 tx=0x7f13c0024000 comp rx=0 tx=0).stop 2024-05-10T04:05:17.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.885+0000 7f13e1198b80 1 -- 172.21.15.150:0/244396975 shutdown_connections 2024-05-10T04:05:17.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.885+0000 7f13e1198b80 1 --2- 172.21.15.150:0/244396975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fff40 0x5639bb306840 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.885+0000 7f13e1198b80 1 --2- 172.21.15.150:0/244396975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2fe720 0x5639bb2feb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.885+0000 7f13e1198b80 1 --2- 172.21.15.150:0/244396975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2ff5f0 0x5639bb2ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.885+0000 7f13e1198b80 1 -- 172.21.15.150:0/244396975 >> 172.21.15.150:0/244396975 conn(0x5639bb260ae0 msgr2=0x5639bb2f9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:17.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.885+0000 7f13e1198b80 1 -- 172.21.15.150:0/244396975 shutdown_connections 2024-05-10T04:05:17.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.885+0000 7f13e1198b80 1 -- 172.21.15.150:0/244396975 wait complete. 2024-05-10T04:05:17.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.885+0000 7f13e1198b80 1 -- start start 2024-05-10T04:05:17.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.885+0000 7f13e1198b80 1 Processor -- start 2024-05-10T04:05:17.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13e1198b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fe720 0x5639bb3086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13e1198b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2ff5f0 0x5639bb308c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13e1198b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fff40 0x5639bb30fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13e1198b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bb1ba5e0 con 0x5639bb2fe720 2024-05-10T04:05:17.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13e1198b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bb1ba280 con 0x5639bb2ff5f0 2024-05-10T04:05:17.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13e1198b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bb0a1720 con 0x5639bb2fff40 2024-05-10T04:05:17.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13e1198b80 10 client.0 _ll_register_callbacks cb 0x5639bb1c3bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:17.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13ca59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fe720 0x5639bb3086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13ca59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fe720 0x5639bb3086e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50676/0 (socket says 172.21.15.150:50676) 2024-05-10T04:05:17.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13ca59c700 1 -- 172.21.15.150:0/453403435 learned_addr learned my addr 172.21.15.150:0/453403435 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:17.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13cad9d700 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fff40 0x5639bb30fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2ff5f0 0x5639bb308c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13ca59c700 1 -- 172.21.15.150:0/453403435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fff40 msgr2=0x5639bb30fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:17.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13ca59c700 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fff40 0x5639bb30fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13ca59c700 1 -- 172.21.15.150:0/453403435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2ff5f0 msgr2=0x5639bb308c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:17.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13ca59c700 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2ff5f0 0x5639bb308c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13ca59c700 1 -- 172.21.15.150:0/453403435 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13cc19c910 con 0x5639bb2fe720 2024-05-10T04:05:17.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13cad9d700 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fff40 0x5639bb30fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:17.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.886+0000 7f13ca59c700 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fe720 0x5639bb3086e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f13bc015000 tx=0x7f13bc00baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b60d380f652ac3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:17.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.887+0000 7f13b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:17.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.887+0000 7f13b1ffb700 1 -- 172.21.15.150:0/453403435 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc00f870 con 0x5639bb2fe720 2024-05-10T04:05:17.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.887+0000 7f13b1ffb700 1 -- 172.21.15.150:0/453403435 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f13bc00f9d0 con 0x5639bb2fe720 2024-05-10T04:05:17.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.887+0000 7f13e1198b80 1 -- 172.21.15.150:0/453403435 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639bb1772f0 con 0x5639bb2fe720 2024-05-10T04:05:17.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.887+0000 7f13b1ffb700 1 -- 172.21.15.150:0/453403435 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13bc00fb90 con 0x5639bb2fe720 2024-05-10T04:05:17.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.887+0000 7f13e1198b80 10 client.15921 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:17.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.887+0000 7f13e1198b80 1 -- 172.21.15.150:0/453403435 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5639bb04bbe0 con 0x5639bb2fe720 2024-05-10T04:05:17.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.888+0000 7f13b1ffb700 1 -- 172.21.15.150:0/453403435 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639bb1772f0 con 0x5639bb2fe720 2024-05-10T04:05:17.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.889+0000 7f13b1ffb700 1 -- 172.21.15.150:0/453403435 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f13bc021070 con 0x5639bb2fe720 2024-05-10T04:05:17.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.889+0000 7f13e1198b80 10 client.15921 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:17.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.889+0000 7f13e1198b80 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:17.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.889+0000 7f13e1198b80 20 client.15921 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:17.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.889+0000 7f13e1198b80 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639bb3142d0 0x5639bb316720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.889+0000 7f13e1198b80 4 client.15921 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:17.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.889+0000 7f13e1198b80 1 -- 172.21.15.150:0/453403435 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639bb2cd800 con 0x5639bb3142d0 2024-05-10T04:05:17.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.889+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639bb3142d0 0x5639bb316720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.890+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639bb3142d0 0x5639bb316720 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b0fb0afd9f58d79 server_cookie=bf749d98845daace in_seq=0 out_seq=0 2024-05-10T04:05:17.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.890+0000 7f13b1ffb700 10 client.15921 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:17.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.890+0000 7f13b1ffb700 1 -- 172.21.15.150:0/453403435 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639bb2cd800 con 0x5639bb3142d0 2024-05-10T04:05:17.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.890+0000 7f13b1ffb700 10 client.15921 handle_command_reply: tid=0 2024-05-10T04:05:17.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.890+0000 7f13e1198b80 1 client.15921 shutdown 2024-05-10T04:05:17.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.890+0000 7f13e1198b80 1 -- 172.21.15.150:0/453403435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fe720 msgr2=0x5639bb3086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:17.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.890+0000 7f13e1198b80 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fe720 0x5639bb3086e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f13bc015000 tx=0x7f13bc00baf0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.890+0000 7f13e1198b80 1 -- 172.21.15.150:0/453403435 shutdown_connections 2024-05-10T04:05:17.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.890+0000 7f13e1198b80 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639bb3142d0 0x5639bb316720 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.890+0000 7f13e1198b80 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fff40 0x5639bb30fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.890+0000 7f13e1198b80 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fe720 0x5639bb3086e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.890+0000 7f13e1198b80 1 --2- 172.21.15.150:0/453403435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2ff5f0 0x5639bb308c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.890+0000 7f13e1198b80 1 -- 172.21.15.150:0/453403435 >> 172.21.15.150:0/453403435 conn(0x5639bb260ae0 msgr2=0x5639bb2f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:17.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.891+0000 7f13b1ffb700 0 client.15921 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:17.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.891+0000 7f13e1198b80 1 -- 172.21.15.150:0/453403435 shutdown_connections 2024-05-10T04:05:17.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.891+0000 7f13e1198b80 1 -- 172.21.15.150:0/453403435 wait complete. 2024-05-10T04:05:17.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.891+0000 7f13e1198b80 20 client.15921 trim_cache size 0 max 16384 2024-05-10T04:05:17.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.908+0000 7f13e1198b80 1 Processor -- start 2024-05-10T04:05:17.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.908+0000 7f13e1198b80 1 -- start start 2024-05-10T04:05:17.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13e1198b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fe720 0x5639bb307140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13e1198b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2ff5f0 0x5639bb30fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13e1198b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fff40 0x5639bb307c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13e1198b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bb1772f0 con 0x5639bb2fff40 2024-05-10T04:05:17.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13e1198b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bb300350 con 0x5639bb2ff5f0 2024-05-10T04:05:17.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13e1198b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bb2f3860 con 0x5639bb2fe720 2024-05-10T04:05:17.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13cad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fff40 0x5639bb307c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13cad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fff40 0x5639bb307c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50686/0 (socket says 172.21.15.150:50686) 2024-05-10T04:05:17.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13cad9d700 1 -- 172.21.15.150:0/2263185503 learned_addr learned my addr 172.21.15.150:0/2263185503 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:17.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/2263185503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2ff5f0 0x5639bb30fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13ca59c700 1 --2- 172.21.15.150:0/2263185503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fe720 0x5639bb307140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13c9d9b700 1 -- 172.21.15.150:0/2263185503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fe720 msgr2=0x5639bb307140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:17.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/2263185503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fe720 0x5639bb307140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13c9d9b700 1 -- 172.21.15.150:0/2263185503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fff40 msgr2=0x5639bb307c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:17.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/2263185503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fff40 0x5639bb307c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13c9d9b700 1 -- 172.21.15.150:0/2263185503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639bb1e45e0 con 0x5639bb2ff5f0 2024-05-10T04:05:17.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.909+0000 7f13cad9d700 1 --2- 172.21.15.150:0/2263185503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fff40 0x5639bb307c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:17.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.910+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/2263185503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2ff5f0 0x5639bb30fb40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f13c00ca550 tx=0x7f13c00961c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28e3a3a7ce8cb228 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:17.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.910+0000 7f13b97fa700 1 -- 172.21.15.150:0/2263185503 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0099880 con 0x5639bb2ff5f0 2024-05-10T04:05:17.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.910+0000 7f13b97fa700 1 -- 172.21.15.150:0/2263185503 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13c009a620 con 0x5639bb2ff5f0 2024-05-10T04:05:17.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.910+0000 7f13b97fa700 1 -- 172.21.15.150:0/2263185503 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00983e0 con 0x5639bb2ff5f0 2024-05-10T04:05:17.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.911+0000 7f13e1198b80 1 -- 172.21.15.150:0/2263185503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2ff5f0 msgr2=0x5639bb30fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:17.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.911+0000 7f13e1198b80 1 --2- 172.21.15.150:0/2263185503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2ff5f0 0x5639bb30fb40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f13c00ca550 tx=0x7f13c00961c0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.911+0000 7f13e1198b80 1 -- 172.21.15.150:0/2263185503 shutdown_connections 2024-05-10T04:05:17.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.911+0000 7f13e1198b80 1 --2- 172.21.15.150:0/2263185503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fff40 0x5639bb307c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.911+0000 7f13e1198b80 1 --2- 172.21.15.150:0/2263185503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2fe720 0x5639bb307140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.911+0000 7f13e1198b80 1 --2- 172.21.15.150:0/2263185503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2ff5f0 0x5639bb30fb40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.911+0000 7f13e1198b80 1 -- 172.21.15.150:0/2263185503 >> 172.21.15.150:0/2263185503 conn(0x5639bb260ae0 msgr2=0x5639bb306a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:17.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.911+0000 7f13e1198b80 1 -- 172.21.15.150:0/2263185503 shutdown_connections 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.911+0000 7f13e1198b80 1 -- 172.21.15.150:0/2263185503 wait complete. 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.911+0000 7f13e1198b80 1 -- start start 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.911+0000 7f13e1198b80 1 Processor -- start 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.912+0000 7f13e1198b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fe720 0x5639bb30db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.912+0000 7f13e1198b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2ff5f0 0x5639bb30e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.912+0000 7f13e1198b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2fff40 0x5639bb30ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.912+0000 7f13e1198b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bb30f4b0 con 0x5639bb2fe720 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.912+0000 7f13e1198b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bb30f620 con 0x5639bb2fff40 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.912+0000 7f13e1198b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bb30f7e0 con 0x5639bb2ff5f0 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.912+0000 7f13e1198b80 10 client.0 _ll_register_callbacks cb 0x5639bb1c3bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.912+0000 7f13c9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2ff5f0 0x5639bb30e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.912+0000 7f13c9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2ff5f0 0x5639bb30e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:58046/0 (socket says 172.21.15.150:58046) 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.912+0000 7f13c9d9b700 1 -- 172.21.15.150:0/4063481624 learned_addr learned my addr 172.21.15.150:0/4063481624 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:17.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.912+0000 7f13cad9d700 1 --2- 172.21.15.150:0/4063481624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2fff40 0x5639bb30ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13c9d9b700 1 -- 172.21.15.150:0/4063481624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2fff40 msgr2=0x5639bb30ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:17.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/4063481624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2fff40 0x5639bb30ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13c9d9b700 1 -- 172.21.15.150:0/4063481624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fe720 msgr2=0x5639bb30db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:17.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/4063481624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fe720 0x5639bb30db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13c9d9b700 1 -- 172.21.15.150:0/4063481624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c4004760 con 0x5639bb2ff5f0 2024-05-10T04:05:17.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13c9d9b700 1 --2- 172.21.15.150:0/4063481624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2ff5f0 0x5639bb30e040 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f13c0000c00 tx=0x7f13c009a790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d81c54f977afaac9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:17.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:05:17.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13b1ffb700 1 -- 172.21.15.150:0/4063481624 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0099f20 con 0x5639bb2ff5f0 2024-05-10T04:05:17.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13b1ffb700 1 -- 172.21.15.150:0/4063481624 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f13c009dd50 con 0x5639bb2ff5f0 2024-05-10T04:05:17.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13e1198b80 1 -- 172.21.15.150:0/4063481624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13c0099a60 con 0x5639bb2ff5f0 2024-05-10T04:05:17.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13b1ffb700 1 -- 172.21.15.150:0/4063481624 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13c00ca830 con 0x5639bb2ff5f0 2024-05-10T04:05:17.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13e1198b80 10 client.25552 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:17.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13e1198b80 1 -- 172.21.15.150:0/4063481624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5639bb09f430 con 0x5639bb2ff5f0 2024-05-10T04:05:17.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.913+0000 7f13b1ffb700 1 -- 172.21.15.150:0/4063481624 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13c0081080 con 0x5639bb2ff5f0 2024-05-10T04:05:17.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.914+0000 7f13b1ffb700 1 -- 172.21.15.150:0/4063481624 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f13c00b8170 con 0x5639bb2ff5f0 2024-05-10T04:05:17.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.914+0000 7f13e1198b80 10 client.25552 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:17.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.914+0000 7f13e1198b80 10 client.25552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:17.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.914+0000 7f13e1198b80 20 client.25552 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:17.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.914+0000 7f13e1198b80 1 --2- 172.21.15.150:0/4063481624 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639bb37c360 0x5639bb37e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:17.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.914+0000 7f13e1198b80 4 client.25552 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T04:05:17.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.914+0000 7f13e1198b80 1 -- 172.21.15.150:0/4063481624 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5639bb216f40 con 0x5639bb37c360 2024-05-10T04:05:17.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.914+0000 7f13ca59c700 1 --2- 172.21.15.150:0/4063481624 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639bb37c360 0x5639bb37e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:17.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.915+0000 7f13b1ffb700 10 client.25552 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:17.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.915+0000 7f13ca59c700 1 --2- 172.21.15.150:0/4063481624 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639bb37c360 0x5639bb37e780 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49226bf841346f4d server_cookie=7fa4478af61a9a5a in_seq=0 out_seq=0 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13b1ffb700 1 -- 172.21.15.150:0/4063481624 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5639bb216f40 con 0x5639bb37c360 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13b1ffb700 10 client.25552 handle_command_reply: tid=0 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13e1198b80 1 client.25552 shutdown 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13e1198b80 1 -- 172.21.15.150:0/4063481624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2ff5f0 msgr2=0x5639bb30e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13e1198b80 1 --2- 172.21.15.150:0/4063481624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2ff5f0 0x5639bb30e040 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f13c0000c00 tx=0x7f13c009a790 comp rx=0 tx=0).stop 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13e1198b80 1 -- 172.21.15.150:0/4063481624 shutdown_connections 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13e1198b80 1 --2- 172.21.15.150:0/4063481624 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5639bb37c360 0x5639bb37e780 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13e1198b80 1 --2- 172.21.15.150:0/4063481624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639bb2fff40 0x5639bb30ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13b1ffb700 0 client.25552 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13e1198b80 1 --2- 172.21.15.150:0/4063481624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639bb2fe720 0x5639bb30db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13e1198b80 1 --2- 172.21.15.150:0/4063481624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5639bb2ff5f0 0x5639bb30e040 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13e1198b80 1 -- 172.21.15.150:0/4063481624 >> 172.21.15.150:0/4063481624 conn(0x5639bb260ae0 msgr2=0x5639bb306620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13e1198b80 1 -- 172.21.15.150:0/4063481624 shutdown_connections 2024-05-10T04:05:17.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.916+0000 7f13e1198b80 1 -- 172.21.15.150:0/4063481624 wait complete. 2024-05-10T04:05:17.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.917+0000 7f13e1198b80 20 client.25552 trim_cache size 0 max 16384 2024-05-10T04:05:17.921 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:05:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: "return_code": 0, 2024-05-10T04:05:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_tag": "2bb74934-2970-4854-bae0-cee4c11f39f6", 2024-05-10T04:05:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": "asynchronous" 2024-05-10T04:05:17.921 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:05:17.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.917+0000 7f13d0e54700 1 -- 172.21.15.150:0/1890496891 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13b40a1260 msgr2=0x7f13b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:17.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.917+0000 7f13d0e54700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13b40a1260 0x7f13b40a3770 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009550 tx=0x7f13c4004ad0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.917+0000 7f13d0e54700 1 -- 172.21.15.150:0/1890496891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13cc101bf0 msgr2=0x7f13cc102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:17.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.917+0000 7f13d0e54700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13cc101bf0 0x7f13cc102f30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f13c000e9a0 tx=0x7f13c000ecb0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.917+0000 7f13d0e54700 1 -- 172.21.15.150:0/1890496891 shutdown_connections 2024-05-10T04:05:17.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.917+0000 7f13d0e54700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13b40a1260 0x7f13b40a3770 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.917+0000 7f13d0e54700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13cc102590 0x7f13cc103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.917+0000 7f13d0e54700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13cc100ad0 0x7f13cc104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.917+0000 7f13d0e54700 1 --2- 172.21.15.150:0/1890496891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13cc101bf0 0x7f13cc102f30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:17.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.917+0000 7f13d0e54700 1 -- 172.21.15.150:0/1890496891 >> 172.21.15.150:0/1890496891 conn(0x7f13cc0fc0c0 msgr2=0x7f13cc0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:17.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.917+0000 7f13d0e54700 1 -- 172.21.15.150:0/1890496891 shutdown_connections 2024-05-10T04:05:17.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:17.917+0000 7f13d0e54700 1 -- 172.21.15.150:0/1890496891 wait complete. 2024-05-10T04:05:17.932 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:05:18.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.074+0000 7f0403dbe700 1 -- 172.21.15.150:0/2933603678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03fc1026d0 msgr2=0x7f03fc104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.074+0000 7f0403dbe700 1 --2- 172.21.15.150:0/2933603678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03fc1026d0 0x7f03fc104af0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009990 tx=0x7f03f8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.074+0000 7f0403dbe700 1 -- 172.21.15.150:0/2933603678 shutdown_connections 2024-05-10T04:05:18.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.074+0000 7f0403dbe700 1 --2- 172.21.15.150:0/2933603678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03fc107990 0x7f03fc109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.074+0000 7f0403dbe700 1 --2- 172.21.15.150:0/2933603678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03fc1026d0 0x7f03fc104af0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.074+0000 7f0403dbe700 1 --2- 172.21.15.150:0/2933603678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03fc105030 0x7f03fc107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.074+0000 7f0403dbe700 1 -- 172.21.15.150:0/2933603678 >> 172.21.15.150:0/2933603678 conn(0x7f03fc0fc0f0 msgr2=0x7f03fc0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:18.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.074+0000 7f0403dbe700 1 -- 172.21.15.150:0/2933603678 shutdown_connections 2024-05-10T04:05:18.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.074+0000 7f0403dbe700 1 -- 172.21.15.150:0/2933603678 wait complete. 2024-05-10T04:05:18.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.075+0000 7f0403dbe700 1 Processor -- start 2024-05-10T04:05:18.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.075+0000 7f0403dbe700 1 -- start start 2024-05-10T04:05:18.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.075+0000 7f0403dbe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03fc1026d0 0x7f03fc101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.075+0000 7f0403dbe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03fc105030 0x7f03fc1005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.075+0000 7f0403dbe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03fc107990 0x7f03fc100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.075+0000 7f0403dbe700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc101200 con 0x7f03fc105030 2024-05-10T04:05:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.075+0000 7f0403dbe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc101370 con 0x7f03fc107990 2024-05-10T04:05:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.075+0000 7f0403dbe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1a1fa0 con 0x7f03fc1026d0 2024-05-10T04:05:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.075+0000 7f0401359700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03fc105030 0x7f03fc1005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f0401359700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03fc105030 0x7f03fc1005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50710/0 (socket says 172.21.15.150:50710) 2024-05-10T04:05:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f0401359700 1 -- 172.21.15.150:0/1566052452 learned_addr learned my addr 172.21.15.150:0/1566052452 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f040235b700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03fc107990 0x7f03fc100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f0401b5a700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03fc1026d0 0x7f03fc101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f0401359700 1 -- 172.21.15.150:0/1566052452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03fc1026d0 msgr2=0x7f03fc101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f0401359700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03fc1026d0 0x7f03fc101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f0401359700 1 -- 172.21.15.150:0/1566052452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03fc107990 msgr2=0x7f03fc100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f0401359700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03fc107990 0x7f03fc100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f0401359700 1 -- 172.21.15.150:0/1566052452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f80095f0 con 0x7f03fc105030 2024-05-10T04:05:18.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f0401359700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03fc105030 0x7f03fc1005e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00e9a0 tx=0x7f03ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:18.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f03f2ffd700 1 -- 172.21.15.150:0/1566052452 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec009790 con 0x7f03fc105030 2024-05-10T04:05:18.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f0403dbe700 1 -- 172.21.15.150:0/1566052452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc1a2200 con 0x7f03fc105030 2024-05-10T04:05:18.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f03f2ffd700 1 -- 172.21.15.150:0/1566052452 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03ec00cbd0 con 0x7f03fc105030 2024-05-10T04:05:18.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.076+0000 7f03f2ffd700 1 -- 172.21.15.150:0/1566052452 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec018680 con 0x7f03fc105030 2024-05-10T04:05:18.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.077+0000 7f0403dbe700 1 -- 172.21.15.150:0/1566052452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc1a2730 con 0x7f03fc105030 2024-05-10T04:05:18.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.081+0000 7f03f2ffd700 1 -- 172.21.15.150:0/1566052452 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f03ec00cd40 con 0x7f03fc105030 2024-05-10T04:05:18.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.081+0000 7f03f2ffd700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03e80a1260 0x7f03e80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.081+0000 7f03f2ffd700 1 -- 172.21.15.150:0/1566052452 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f03ec014070 con 0x7f03fc105030 2024-05-10T04:05:18.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f0414102b80 1 Processor -- start 2024-05-10T04:05:18.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f0401b5a700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03e80a1260 0x7f03e80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f0414102b80 1 -- start start 2024-05-10T04:05:18.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f0414102b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faa9eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f0414102b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 0x55d3faa9f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f0414102b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaa67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f0414102b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3fa9c5150 con 0x55d3faa9fec0 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f0414102b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3fa9c2b10 con 0x55d3faa9e6a0 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f0414102b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3fa9be400 con 0x55d3faa9f570 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f040235b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaa67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f040235b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaa67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50720/0 (socket says 172.21.15.150:50720) 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f040235b700 1 -- 172.21.15.150:0/1903304386 learned_addr learned my addr 172.21.15.150:0/1903304386 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f0401b5a700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03e80a1260 0x7f03e80a3770 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f03f800b2e0 tx=0x7f03f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.082+0000 7f0401b5a700 1 --2- 172.21.15.150:0/1903304386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faa9eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.083+0000 7f0401359700 1 --2- 172.21.15.150:0/1903304386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 0x55d3faa9f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.083+0000 7f040235b700 1 -- 172.21.15.150:0/1903304386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 msgr2=0x55d3faa9f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.083+0000 7f040235b700 1 --2- 172.21.15.150:0/1903304386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 0x55d3faa9f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.083+0000 7f040235b700 1 -- 172.21.15.150:0/1903304386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 msgr2=0x55d3faa9eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.083+0000 7f040235b700 1 --2- 172.21.15.150:0/1903304386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faa9eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.083+0000 7f040235b700 1 -- 172.21.15.150:0/1903304386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3fa83f3b0 con 0x55d3faa9fec0 2024-05-10T04:05:18.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.083+0000 7f0401359700 1 --2- 172.21.15.150:0/1903304386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 0x55d3faa9f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:18.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.083+0000 7f040235b700 1 --2- 172.21.15.150:0/1903304386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaa67c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f03f4011580 tx=0x7f03f4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1437303b95a2ba16 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:18.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.083+0000 7f03e7fff700 1 -- 172.21.15.150:0/1903304386 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f400e040 con 0x55d3faa9fec0 2024-05-10T04:05:18.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.083+0000 7f03e7fff700 1 -- 172.21.15.150:0/1903304386 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03f400f040 con 0x55d3faa9fec0 2024-05-10T04:05:18.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.083+0000 7f03e7fff700 1 -- 172.21.15.150:0/1903304386 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4004460 con 0x55d3faa9fec0 2024-05-10T04:05:18.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.084+0000 7f0414102b80 1 -- 172.21.15.150:0/1903304386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 msgr2=0x55d3faaa67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.084+0000 7f0414102b80 1 --2- 172.21.15.150:0/1903304386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaa67c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f03f4011580 tx=0x7f03f4009920 comp rx=0 tx=0).stop 2024-05-10T04:05:18.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.085+0000 7f0414102b80 1 -- 172.21.15.150:0/1903304386 shutdown_connections 2024-05-10T04:05:18.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.085+0000 7f0414102b80 1 --2- 172.21.15.150:0/1903304386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaa67c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.085+0000 7f0414102b80 1 --2- 172.21.15.150:0/1903304386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faa9eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.085+0000 7f0414102b80 1 --2- 172.21.15.150:0/1903304386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 0x55d3faa9f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.085+0000 7f0414102b80 1 -- 172.21.15.150:0/1903304386 >> 172.21.15.150:0/1903304386 conn(0x55d3faa00a60 msgr2=0x55d3faa98fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:18.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.085+0000 7f0414102b80 1 -- 172.21.15.150:0/1903304386 shutdown_connections 2024-05-10T04:05:18.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.085+0000 7f0414102b80 1 -- 172.21.15.150:0/1903304386 wait complete. 2024-05-10T04:05:18.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.085+0000 7f0414102b80 1 -- start start 2024-05-10T04:05:18.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.085+0000 7f0414102b80 1 Processor -- start 2024-05-10T04:05:18.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0414102b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faaa8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0414102b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 0x55d3faaa8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0414102b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaafb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0414102b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3fa95a560 con 0x55d3faa9fec0 2024-05-10T04:05:18.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0414102b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3fa95a200 con 0x55d3faa9e6a0 2024-05-10T04:05:18.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0414102b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3fa8416a0 con 0x55d3faa9f570 2024-05-10T04:05:18.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0414102b80 10 client.0 _ll_register_callbacks cb 0x55d3fa963b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:18.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f040235b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaafb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f040235b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaafb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50730/0 (socket says 172.21.15.150:50730) 2024-05-10T04:05:18.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0401b5a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faaa8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f040235b700 1 -- 172.21.15.150:0/660435509 learned_addr learned my addr 172.21.15.150:0/660435509 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:18.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0401359700 1 --2- 172.21.15.150:0/660435509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 0x55d3faaa8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0401b5a700 1 -- 172.21.15.150:0/660435509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 msgr2=0x55d3faaa8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0401b5a700 1 --2- 172.21.15.150:0/660435509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 0x55d3faaa8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0401b5a700 1 -- 172.21.15.150:0/660435509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 msgr2=0x55d3faaafb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0401b5a700 1 --2- 172.21.15.150:0/660435509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaafb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.086+0000 7f0401b5a700 1 -- 172.21.15.150:0/660435509 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f4011230 con 0x55d3faa9e6a0 2024-05-10T04:05:18.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.087+0000 7f0401b5a700 1 --2- 172.21.15.150:0/660435509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faaa8690 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f03f80098e0 tx=0x7f03f801e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7530400a10b96157 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:18.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.087+0000 7f03e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:05:18.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.087+0000 7f03e57fa700 1 -- 172.21.15.150:0/660435509 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f802c040 con 0x55d3faa9e6a0 2024-05-10T04:05:18.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.087+0000 7f0414102b80 1 -- 172.21.15.150:0/660435509 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3fa917270 con 0x55d3faa9e6a0 2024-05-10T04:05:18.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.087+0000 7f03e57fa700 1 -- 172.21.15.150:0/660435509 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f03f800f660 con 0x55d3faa9e6a0 2024-05-10T04:05:18.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.087+0000 7f03e57fa700 1 -- 172.21.15.150:0/660435509 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03f801d040 con 0x55d3faa9e6a0 2024-05-10T04:05:18.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.087+0000 7f0414102b80 10 client.25616 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:18.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.087+0000 7f0414102b80 1 -- 172.21.15.150:0/660435509 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d3fa7ebb60 con 0x55d3faa9e6a0 2024-05-10T04:05:18.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.087+0000 7f03e57fa700 1 -- 172.21.15.150:0/660435509 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3fa917270 con 0x55d3faa9e6a0 2024-05-10T04:05:18.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.088+0000 7f03e57fa700 1 -- 172.21.15.150:0/660435509 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f03f80239b0 con 0x55d3faa9e6a0 2024-05-10T04:05:18.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.088+0000 7f0414102b80 10 client.25616 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:18.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.088+0000 7f0414102b80 10 client.25616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:18.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.088+0000 7f0414102b80 20 client.25616 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:18.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.088+0000 7f0414102b80 1 --2- 172.21.15.150:0/660435509 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55d3faab4330 0x55d3faab6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.088+0000 7f0414102b80 4 client.25616 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:18.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.088+0000 7f0414102b80 1 -- 172.21.15.150:0/660435509 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3faa6d780 con 0x55d3faab4330 2024-05-10T04:05:18.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.088+0000 7f0401359700 1 --2- 172.21.15.150:0/660435509 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55d3faab4330 0x55d3faab6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.089+0000 7f03e57fa700 10 client.25616 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:18.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.089+0000 7f0401359700 1 --2- 172.21.15.150:0/660435509 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55d3faab4330 0x55d3faab6780 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=883211f6fdaceda5 server_cookie=a8ec97d2661754d0 in_seq=0 out_seq=0 2024-05-10T04:05:18.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.089+0000 7f03e57fa700 1 -- 172.21.15.150:0/660435509 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d3faa6d780 con 0x55d3faab4330 2024-05-10T04:05:18.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.089+0000 7f03e57fa700 10 client.25616 handle_command_reply: tid=0 2024-05-10T04:05:18.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.089+0000 7f0414102b80 1 client.25616 shutdown 2024-05-10T04:05:18.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.089+0000 7f0414102b80 1 -- 172.21.15.150:0/660435509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 msgr2=0x55d3faaa8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.089+0000 7f0414102b80 1 --2- 172.21.15.150:0/660435509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faaa8690 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f03f80098e0 tx=0x7f03f801e000 comp rx=0 tx=0).stop 2024-05-10T04:05:18.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.089+0000 7f0414102b80 1 -- 172.21.15.150:0/660435509 shutdown_connections 2024-05-10T04:05:18.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.089+0000 7f0414102b80 1 --2- 172.21.15.150:0/660435509 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55d3faab4330 0x55d3faab6780 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.089+0000 7f0414102b80 1 --2- 172.21.15.150:0/660435509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaafb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.089+0000 7f0414102b80 1 --2- 172.21.15.150:0/660435509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faaa8690 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.089+0000 7f0414102b80 1 --2- 172.21.15.150:0/660435509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 0x55d3faaa8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.089+0000 7f0414102b80 1 -- 172.21.15.150:0/660435509 >> 172.21.15.150:0/660435509 conn(0x55d3faa00a60 msgr2=0x55d3faa98fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.090+0000 7f03e57fa700 0 client.25616 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.090+0000 7f0414102b80 1 -- 172.21.15.150:0/660435509 shutdown_connections 2024-05-10T04:05:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.090+0000 7f0414102b80 1 -- 172.21.15.150:0/660435509 wait complete. 2024-05-10T04:05:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.090+0000 7f0414102b80 20 client.25616 trim_cache size 0 max 16384 2024-05-10T04:05:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.107+0000 7f0414102b80 1 Processor -- start 2024-05-10T04:05:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.107+0000 7f0414102b80 1 -- start start 2024-05-10T04:05:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.107+0000 7f0414102b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faaa27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.107+0000 7f0414102b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 0x55d3faaafaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.107+0000 7f0414102b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaa3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.107+0000 7f0414102b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3faaa2d20 con 0x55d3faa9fec0 2024-05-10T04:05:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.107+0000 7f0414102b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3faa9ed00 con 0x55d3faa9e6a0 2024-05-10T04:05:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.107+0000 7f0414102b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3faaa3750 con 0x55d3faa9f570 2024-05-10T04:05:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.108+0000 7f0401b5a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faaa27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.108+0000 7f0401b5a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faaa27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:44096/0 (socket says 172.21.15.150:44096) 2024-05-10T04:05:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.108+0000 7f0401b5a700 1 -- 172.21.15.150:0/548311037 learned_addr learned my addr 172.21.15.150:0/548311037 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.108+0000 7f040235b700 1 --2- 172.21.15.150:0/548311037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaa3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.108+0000 7f0401359700 1 --2- 172.21.15.150:0/548311037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 0x55d3faaafaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.108+0000 7f0401b5a700 1 -- 172.21.15.150:0/548311037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 msgr2=0x55d3faaafaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.108+0000 7f0401b5a700 1 --2- 172.21.15.150:0/548311037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 0x55d3faaafaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.108+0000 7f0401b5a700 1 -- 172.21.15.150:0/548311037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 msgr2=0x55d3faaa3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.108+0000 7f0401b5a700 1 --2- 172.21.15.150:0/548311037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaa3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.108+0000 7f0401b5a700 1 -- 172.21.15.150:0/548311037 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3fa984560 con 0x55d3faa9e6a0 2024-05-10T04:05:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.108+0000 7f040235b700 1 --2- 172.21.15.150:0/548311037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaa3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:18.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.108+0000 7f0401b5a700 1 --2- 172.21.15.150:0/548311037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faaa27a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f03f800b6e0 tx=0x7f03f8005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c308354793b9916 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:18.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.108+0000 7f03f0ff9700 1 -- 172.21.15.150:0/548311037 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f801d070 con 0x55d3faa9e6a0 2024-05-10T04:05:18.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.109+0000 7f03f0ff9700 1 -- 172.21.15.150:0/548311037 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03f800e3e0 con 0x55d3faa9e6a0 2024-05-10T04:05:18.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.109+0000 7f03f0ff9700 1 -- 172.21.15.150:0/548311037 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f802c040 con 0x55d3faa9e6a0 2024-05-10T04:05:18.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.109+0000 7f0414102b80 1 -- 172.21.15.150:0/548311037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 msgr2=0x55d3faaa27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.109+0000 7f0414102b80 1 --2- 172.21.15.150:0/548311037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faaa27a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f03f800b6e0 tx=0x7f03f8005fb0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.110+0000 7f0414102b80 1 -- 172.21.15.150:0/548311037 shutdown_connections 2024-05-10T04:05:18.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.110+0000 7f0414102b80 1 --2- 172.21.15.150:0/548311037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaa3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.110+0000 7f0414102b80 1 --2- 172.21.15.150:0/548311037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9e6a0 0x55d3faaa27a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.110+0000 7f0414102b80 1 --2- 172.21.15.150:0/548311037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9f570 0x55d3faaafaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.110+0000 7f0414102b80 1 -- 172.21.15.150:0/548311037 >> 172.21.15.150:0/548311037 conn(0x55d3faa00a60 msgr2=0x55d3faa98f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.110+0000 7f0414102b80 1 -- 172.21.15.150:0/548311037 shutdown_connections 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.110+0000 7f0414102b80 1 -- 172.21.15.150:0/548311037 wait complete. 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.110+0000 7f0414102b80 1 -- start start 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.110+0000 7f0414102b80 1 Processor -- start 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.111+0000 7f0414102b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9e6a0 0x55d3faaad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.111+0000 7f0414102b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9f570 0x55d3faaadf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.111+0000 7f0414102b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaaeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.111+0000 7f0414102b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3faaaf2e0 con 0x55d3faa9fec0 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.111+0000 7f0414102b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3faaaf450 con 0x55d3faa9f570 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.111+0000 7f0414102b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3faaaf610 con 0x55d3faa9e6a0 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.111+0000 7f0414102b80 10 client.0 _ll_register_callbacks cb 0x55d3fa963b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.111+0000 7f040235b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaaeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.111+0000 7f040235b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaaeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50760/0 (socket says 172.21.15.150:50760) 2024-05-10T04:05:18.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.111+0000 7f040235b700 1 -- 172.21.15.150:0/2919626923 learned_addr learned my addr 172.21.15.150:0/2919626923 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:18.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.111+0000 7f0401359700 1 --2- 172.21.15.150:0/2919626923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9f570 0x55d3faaadf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f0401b5a700 1 --2- 172.21.15.150:0/2919626923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9e6a0 0x55d3faaad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f0401359700 1 -- 172.21.15.150:0/2919626923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9e6a0 msgr2=0x55d3faaad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f0401359700 1 --2- 172.21.15.150:0/2919626923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9e6a0 0x55d3faaad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f0401359700 1 -- 172.21.15.150:0/2919626923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 msgr2=0x55d3faaaeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f0401359700 1 --2- 172.21.15.150:0/2919626923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaaeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f0401359700 1 -- 172.21.15.150:0/2919626923 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3fa7ebb60 con 0x55d3faa9f570 2024-05-10T04:05:18.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f0401359700 1 --2- 172.21.15.150:0/2919626923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9f570 0x55d3faaadf10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0ca630 tx=0x7f03ec0cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcd97d4e620139b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:18.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f03e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:05:18.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f03e57fa700 1 -- 172.21.15.150:0/2919626923 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0a23f0 con 0x55d3faa9f570 2024-05-10T04:05:18.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f03e57fa700 1 -- 172.21.15.150:0/2919626923 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f03ec0ade40 con 0x55d3faa9f570 2024-05-10T04:05:18.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f0414102b80 1 -- 172.21.15.150:0/2919626923 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03f400f040 con 0x55d3faa9f570 2024-05-10T04:05:18.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f03e57fa700 1 -- 172.21.15.150:0/2919626923 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03ec099460 con 0x55d3faa9f570 2024-05-10T04:05:18.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f0414102b80 10 client.25622 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:18.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.112+0000 7f0414102b80 1 -- 172.21.15.150:0/2919626923 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d3fa843cd0 con 0x55d3faa9f570 2024-05-10T04:05:18.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.113+0000 7f03e57fa700 1 -- 172.21.15.150:0/2919626923 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03f400f040 con 0x55d3faa9f570 2024-05-10T04:05:18.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.113+0000 7f03e57fa700 1 -- 172.21.15.150:0/2919626923 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f03ec0a1ff0 con 0x55d3faa9f570 2024-05-10T04:05:18.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.114+0000 7f0414102b80 10 client.25622 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:18.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.114+0000 7f0414102b80 10 client.25622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:18.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.114+0000 7f0414102b80 20 client.25622 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:18.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.114+0000 7f0414102b80 1 --2- 172.21.15.150:0/2919626923 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55d3fab0b730 0x55d3fab0db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.114+0000 7f0414102b80 4 client.25622 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:05:18.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.114+0000 7f0414102b80 1 -- 172.21.15.150:0/2919626923 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d3fa9b6ec0 con 0x55d3fab0b730 2024-05-10T04:05:18.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.114+0000 7f0401b5a700 1 --2- 172.21.15.150:0/2919626923 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55d3fab0b730 0x55d3fab0db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.114+0000 7f03e57fa700 10 client.25622 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:18.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.114+0000 7f0401b5a700 1 --2- 172.21.15.150:0/2919626923 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55d3fab0b730 0x55d3fab0db50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15db9eac33ae7b70 server_cookie=208a2b109bfa88f2 in_seq=0 out_seq=0 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f03e57fa700 1 -- 172.21.15.150:0/2919626923 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d3fa9b6ec0 con 0x55d3fab0b730 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f03e57fa700 10 client.25622 handle_command_reply: tid=0 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f0414102b80 1 client.25622 shutdown 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f0414102b80 1 -- 172.21.15.150:0/2919626923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9f570 msgr2=0x55d3faaadf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f0414102b80 1 --2- 172.21.15.150:0/2919626923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9f570 0x55d3faaadf10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0ca630 tx=0x7f03ec0cc040 comp rx=0 tx=0).stop 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f0414102b80 1 -- 172.21.15.150:0/2919626923 shutdown_connections 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f0414102b80 1 --2- 172.21.15.150:0/2919626923 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55d3fab0b730 0x55d3fab0db50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f0414102b80 1 --2- 172.21.15.150:0/2919626923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3faa9fec0 0x55d3faaaeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f0414102b80 1 --2- 172.21.15.150:0/2919626923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3faa9e6a0 0x55d3faaad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f0414102b80 1 --2- 172.21.15.150:0/2919626923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3faa9f570 0x55d3faaadf10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f0414102b80 1 -- 172.21.15.150:0/2919626923 >> 172.21.15.150:0/2919626923 conn(0x55d3faa00a60 msgr2=0x55d3faa98f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f03e57fa700 0 client.25622 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f0414102b80 1 -- 172.21.15.150:0/2919626923 shutdown_connections 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f0414102b80 1 -- 172.21.15.150:0/2919626923 wait complete. 2024-05-10T04:05:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.115+0000 7f0414102b80 20 client.25622 trim_cache size 0 max 16384 2024-05-10T04:05:18.120 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:05:18.120 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T04:05:18.120 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": { 2024-05-10T04:05:18.120 INFO:teuthology.orchestra.run.smithi150.stdout: "2bb74934-2970-4854-bae0-cee4c11f39f6": { 2024-05-10T04:05:18.120 INFO:teuthology.orchestra.run.smithi150.stdout: "path": "/", 2024-05-10T04:05:18.120 INFO:teuthology.orchestra.run.smithi150.stdout: "tag": "2bb74934-2970-4854-bae0-cee4c11f39f6", 2024-05-10T04:05:18.120 INFO:teuthology.orchestra.run.smithi150.stdout: "options": "recursive,force" 2024-05-10T04:05:18.120 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:05:18.120 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:05:18.120 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:05:18.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.116+0000 7f0403dbe700 1 -- 172.21.15.150:0/1566052452 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03e80a1260 msgr2=0x7f03e80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.116+0000 7f0403dbe700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03e80a1260 0x7f03e80a3770 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f03f800b2e0 tx=0x7f03f801a040 comp rx=0 tx=0).stop 2024-05-10T04:05:18.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.116+0000 7f0403dbe700 1 -- 172.21.15.150:0/1566052452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03fc105030 msgr2=0x7f03fc1005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.116+0000 7f0403dbe700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03fc105030 0x7f03fc1005e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00e9a0 tx=0x7f03ec00ecb0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.116+0000 7f0403dbe700 1 -- 172.21.15.150:0/1566052452 shutdown_connections 2024-05-10T04:05:18.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.116+0000 7f0403dbe700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03e80a1260 0x7f03e80a3770 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.116+0000 7f0403dbe700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03fc107990 0x7f03fc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.116+0000 7f0403dbe700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03fc1026d0 0x7f03fc101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.116+0000 7f0403dbe700 1 --2- 172.21.15.150:0/1566052452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03fc105030 0x7f03fc1005e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.116+0000 7f0403dbe700 1 -- 172.21.15.150:0/1566052452 >> 172.21.15.150:0/1566052452 conn(0x7f03fc0fc0f0 msgr2=0x7f03fc103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:18.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.117+0000 7f0403dbe700 1 -- 172.21.15.150:0/1566052452 shutdown_connections 2024-05-10T04:05:18.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:18.117+0000 7f0403dbe700 1 -- 172.21.15.150:0/1566052452 wait complete. 2024-05-10T04:05:18.136 INFO:tasks.cephfs.filesystem:scrub status for tag:2bb74934-2970-4854-bae0-cee4c11f39f6 - {'path': '/', 'tag': '2bb74934-2970-4854-bae0-cee4c11f39f6', 'options': 'recursive,force'} 2024-05-10T04:05:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.282+0000 7f21418e0700 1 -- 172.21.15.150:0/2148711254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f213c0ffc00 msgr2=0x7f213c100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.282+0000 7f21418e0700 1 --2- 172.21.15.150:0/2148711254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f213c0ffc00 0x7f213c100010 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2130009990 tx=0x7f2130009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.282+0000 7f21418e0700 1 -- 172.21.15.150:0/2148711254 shutdown_connections 2024-05-10T04:05:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.282+0000 7f21418e0700 1 --2- 172.21.15.150:0/2148711254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c100550 0x7f213c10a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.282+0000 7f21418e0700 1 --2- 172.21.15.150:0/2148711254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f213c1031c0 0x7f213c1035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.282+0000 7f21418e0700 1 --2- 172.21.15.150:0/2148711254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f213c0ffc00 0x7f213c100010 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.282+0000 7f21418e0700 1 -- 172.21.15.150:0/2148711254 >> 172.21.15.150:0/2148711254 conn(0x7f213c0fb8d0 msgr2=0x7f213c0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.282+0000 7f21418e0700 1 -- 172.21.15.150:0/2148711254 shutdown_connections 2024-05-10T04:05:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.282+0000 7f21418e0700 1 -- 172.21.15.150:0/2148711254 wait complete. 2024-05-10T04:05:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.283+0000 7f21418e0700 1 Processor -- start 2024-05-10T04:05:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f21418e0700 1 -- start start 2024-05-10T04:05:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f21418e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f213c0ffc00 0x7f213c102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f21418e0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f213c100550 0x7f213c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f21418e0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c1031c0 0x7f213c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f21418e0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c19be00 con 0x7f213c100550 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f21418e0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c19bf70 con 0x7f213c1031c0 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f21418e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c101930 con 0x7f213c0ffc00 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f213a7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f213c100550 0x7f213c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f213a7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f213c100550 0x7f213c100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50786/0 (socket says 172.21.15.150:50786) 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f213a7fc700 1 -- 172.21.15.150:0/1871646327 learned_addr learned my addr 172.21.15.150:0/1871646327 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f213b7fe700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c1031c0 0x7f213c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f213affd700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f213c0ffc00 0x7f213c102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f213b7fe700 1 -- 172.21.15.150:0/1871646327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f213c0ffc00 msgr2=0x7f213c102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f213b7fe700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f213c0ffc00 0x7f213c102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f213b7fe700 1 -- 172.21.15.150:0/1871646327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f213c100550 msgr2=0x7f213c100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.284+0000 7f213b7fe700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f213c100550 0x7f213c100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.285+0000 7f213b7fe700 1 -- 172.21.15.150:0/1871646327 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2134009c40 con 0x7f213c1031c0 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.285+0000 7f213b7fe700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c1031c0 0x7f213c101370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f213400efb0 tx=0x7f213400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.285+0000 7f21408de700 1 -- 172.21.15.150:0/1871646327 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134003a00 con 0x7f213c1031c0 2024-05-10T04:05:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.285+0000 7f21418e0700 1 -- 172.21.15.150:0/1871646327 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21300095f0 con 0x7f213c1031c0 2024-05-10T04:05:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.285+0000 7f21408de700 1 -- 172.21.15.150:0/1871646327 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f213400f040 con 0x7f213c1031c0 2024-05-10T04:05:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.285+0000 7f21408de700 1 -- 172.21.15.150:0/1871646327 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134004540 con 0x7f213c1031c0 2024-05-10T04:05:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.285+0000 7f21418e0700 1 -- 172.21.15.150:0/1871646327 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c19c290 con 0x7f213c1031c0 2024-05-10T04:05:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.287+0000 7f21408de700 1 -- 172.21.15.150:0/1871646327 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f2134004780 con 0x7f213c1031c0 2024-05-10T04:05:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.288+0000 7f21408de700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f21240a11b0 0x7f21240a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.288+0000 7f21408de700 1 -- 172.21.15.150:0/1871646327 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f21340cd960 con 0x7f213c1031c0 2024-05-10T04:05:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.288+0000 7f213affd700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f21240a11b0 0x7f21240a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.291+0000 7f213affd700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f21240a11b0 0x7f21240a36c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f213c069920 tx=0x7f212c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.291+0000 7f2151c24b80 1 Processor -- start 2024-05-10T04:05:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.291+0000 7f2151c24b80 1 -- start start 2024-05-10T04:05:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.291+0000 7f2151c24b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x55886443a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.291+0000 7f2151c24b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b050 0x55886443b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.291+0000 7f2151c24b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b9a0 0x5588644422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.291+0000 7f2151c24b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588643610f0 con 0x55886443a180 2024-05-10T04:05:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.291+0000 7f2151c24b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55886435eab0 con 0x55886443b050 2024-05-10T04:05:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.291+0000 7f2151c24b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55886435a3a0 con 0x55886443b9a0 2024-05-10T04:05:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.292+0000 7f213affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x55886443a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.292+0000 7f213affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x55886443a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50802/0 (socket says 172.21.15.150:50802) 2024-05-10T04:05:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.292+0000 7f213affd700 1 -- 172.21.15.150:0/2274264399 learned_addr learned my addr 172.21.15.150:0/2274264399 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.292+0000 7f213a7fc700 1 --2- 172.21.15.150:0/2274264399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b050 0x55886443b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.292+0000 7f213b7fe700 1 --2- 172.21.15.150:0/2274264399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b9a0 0x5588644422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.292+0000 7f213affd700 1 -- 172.21.15.150:0/2274264399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b9a0 msgr2=0x5588644422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.292+0000 7f213affd700 1 --2- 172.21.15.150:0/2274264399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b9a0 0x5588644422a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.292+0000 7f213affd700 1 -- 172.21.15.150:0/2274264399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b050 msgr2=0x55886443b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.292+0000 7f213affd700 1 --2- 172.21.15.150:0/2274264399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b050 0x55886443b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.292+0000 7f213affd700 1 -- 172.21.15.150:0/2274264399 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588641db350 con 0x55886443a180 2024-05-10T04:05:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.292+0000 7f213affd700 1 --2- 172.21.15.150:0/2274264399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x55886443a5b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f212c011d20 tx=0x7f212c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebe66b8f19d1ce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.292+0000 7f2121ffb700 1 -- 172.21.15.150:0/2274264399 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c00eb20 con 0x55886443a180 2024-05-10T04:05:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.293+0000 7f2121ffb700 1 -- 172.21.15.150:0/2274264399 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f212c021070 con 0x55886443a180 2024-05-10T04:05:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.293+0000 7f2121ffb700 1 -- 172.21.15.150:0/2274264399 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c01d410 con 0x55886443a180 2024-05-10T04:05:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.294+0000 7f2151c24b80 1 -- 172.21.15.150:0/2274264399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 msgr2=0x55886443a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.294+0000 7f2151c24b80 1 --2- 172.21.15.150:0/2274264399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x55886443a5b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f212c011d20 tx=0x7f212c014000 comp rx=0 tx=0).stop 2024-05-10T04:05:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.294+0000 7f2151c24b80 1 -- 172.21.15.150:0/2274264399 shutdown_connections 2024-05-10T04:05:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.294+0000 7f2151c24b80 1 --2- 172.21.15.150:0/2274264399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b9a0 0x5588644422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.294+0000 7f2151c24b80 1 --2- 172.21.15.150:0/2274264399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x55886443a5b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.294+0000 7f2151c24b80 1 --2- 172.21.15.150:0/2274264399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b050 0x55886443b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.294+0000 7f2151c24b80 1 -- 172.21.15.150:0/2274264399 >> 172.21.15.150:0/2274264399 conn(0x55886439ca00 msgr2=0x558864434a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.294+0000 7f2151c24b80 1 -- 172.21.15.150:0/2274264399 shutdown_connections 2024-05-10T04:05:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.294+0000 7f2151c24b80 1 -- 172.21.15.150:0/2274264399 wait complete. 2024-05-10T04:05:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.294+0000 7f2151c24b80 1 -- start start 2024-05-10T04:05:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.295+0000 7f2151c24b80 1 Processor -- start 2024-05-10T04:05:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.295+0000 7f2151c24b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x558864444220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.295+0000 7f2151c24b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b050 0x558864444760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.295+0000 7f2151c24b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b9a0 0x55886444b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.295+0000 7f2151c24b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588642f6500 con 0x55886443a180 2024-05-10T04:05:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.295+0000 7f2151c24b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588642f61a0 con 0x55886443b9a0 2024-05-10T04:05:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.295+0000 7f2151c24b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588641dd640 con 0x55886443b050 2024-05-10T04:05:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.295+0000 7f2151c24b80 10 client.0 _ll_register_callbacks cb 0x5588642ffad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.295+0000 7f213a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b050 0x558864444760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f213affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x558864444220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f213a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b050 0x558864444760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:58126/0 (socket says 172.21.15.150:58126) 2024-05-10T04:05:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f213a7fc700 1 -- 172.21.15.150:0/1447084753 learned_addr learned my addr 172.21.15.150:0/1447084753 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f213affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x558864444220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50804/0 (socket says 172.21.15.150:50804) 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f213b7fe700 1 --2- 172.21.15.150:0/1447084753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b9a0 0x55886444b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f213a7fc700 1 -- 172.21.15.150:0/1447084753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b9a0 msgr2=0x55886444b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f213a7fc700 1 --2- 172.21.15.150:0/1447084753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b9a0 0x55886444b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f213a7fc700 1 -- 172.21.15.150:0/1447084753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 msgr2=0x558864444220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f213a7fc700 1 --2- 172.21.15.150:0/1447084753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x558864444220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f213a7fc700 1 -- 172.21.15.150:0/1447084753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212c017070 con 0x55886443b050 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f213b7fe700 1 --2- 172.21.15.150:0/1447084753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b9a0 0x55886444b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f213a7fc700 1 --2- 172.21.15.150:0/1447084753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b050 0x558864444760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f213000bfd0 tx=0x7f2130005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c73e9ae8a1636c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f211affd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.296+0000 7f211affd700 1 -- 172.21.15.150:0/1447084753 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130005720 con 0x55886443b050 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.297+0000 7f211affd700 1 -- 172.21.15.150:0/1447084753 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2130004700 con 0x55886443b050 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.297+0000 7f2151c24b80 1 -- 172.21.15.150:0/1447084753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588642b3210 con 0x55886443b050 2024-05-10T04:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.297+0000 7f211affd700 1 -- 172.21.15.150:0/1447084753 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21300048c0 con 0x55886443b050 2024-05-10T04:05:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.297+0000 7f2151c24b80 10 client.25582 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.297+0000 7f2151c24b80 1 -- 172.21.15.150:0/1447084753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558864187b00 con 0x55886443b050 2024-05-10T04:05:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.297+0000 7f211affd700 1 -- 172.21.15.150:0/1447084753 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2130032030 con 0x55886443b050 2024-05-10T04:05:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.297+0000 7f211affd700 1 -- 172.21.15.150:0/1447084753 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2130024b30 con 0x55886443b050 2024-05-10T04:05:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.298+0000 7f2151c24b80 10 client.25582 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.298+0000 7f2151c24b80 10 client.25582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.298+0000 7f2151c24b80 20 client.25582 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.298+0000 7f2151c24b80 1 --2- 172.21.15.150:0/1447084753 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55886444fe90 0x5588644522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.298+0000 7f2151c24b80 4 client.25582 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.298+0000 7f2151c24b80 1 -- 172.21.15.150:0/1447084753 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558864409720 con 0x55886444fe90 2024-05-10T04:05:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.299+0000 7f213affd700 1 --2- 172.21.15.150:0/1447084753 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55886444fe90 0x5588644522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.299+0000 7f213affd700 1 --2- 172.21.15.150:0/1447084753 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55886444fe90 0x5588644522e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62ee37b87f61328e server_cookie=1c27159c235bb345 in_seq=0 out_seq=0 2024-05-10T04:05:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.299+0000 7f211affd700 10 client.25582 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.300+0000 7f211affd700 1 -- 172.21.15.150:0/1447084753 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558864409720 con 0x55886444fe90 2024-05-10T04:05:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.300+0000 7f211affd700 10 client.25582 handle_command_reply: tid=0 2024-05-10T04:05:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.300+0000 7f2151c24b80 1 client.25582 shutdown 2024-05-10T04:05:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.300+0000 7f2151c24b80 1 -- 172.21.15.150:0/1447084753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b050 msgr2=0x558864444760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.300+0000 7f2151c24b80 1 --2- 172.21.15.150:0/1447084753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b050 0x558864444760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f213000bfd0 tx=0x7f2130005e80 comp rx=0 tx=0).stop 2024-05-10T04:05:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.300+0000 7f2151c24b80 1 -- 172.21.15.150:0/1447084753 shutdown_connections 2024-05-10T04:05:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.300+0000 7f2151c24b80 1 --2- 172.21.15.150:0/1447084753 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55886444fe90 0x5588644522e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.300+0000 7f2151c24b80 1 --2- 172.21.15.150:0/1447084753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b9a0 0x55886444b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.300+0000 7f211affd700 0 client.25582 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.300+0000 7f2151c24b80 1 --2- 172.21.15.150:0/1447084753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x558864444220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.300+0000 7f2151c24b80 1 --2- 172.21.15.150:0/1447084753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b050 0x558864444760 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.300+0000 7f2151c24b80 1 -- 172.21.15.150:0/1447084753 >> 172.21.15.150:0/1447084753 conn(0x55886439ca00 msgr2=0x558864434a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.301+0000 7f2151c24b80 1 -- 172.21.15.150:0/1447084753 shutdown_connections 2024-05-10T04:05:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.301+0000 7f2151c24b80 1 -- 172.21.15.150:0/1447084753 wait complete. 2024-05-10T04:05:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.301+0000 7f2151c24b80 20 client.25582 trim_cache size 0 max 16384 2024-05-10T04:05:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.329+0000 7f2151c24b80 1 Processor -- start 2024-05-10T04:05:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.329+0000 7f2151c24b80 1 -- start start 2024-05-10T04:05:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f2151c24b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443a180 0x55886444bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f2151c24b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b050 0x55886444b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f2151c24b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443b9a0 0x558864443790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f2151c24b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55886442c9b0 con 0x55886443b9a0 2024-05-10T04:05:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f2151c24b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588644421e0 con 0x55886443b050 2024-05-10T04:05:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f2151c24b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55886443fab0 con 0x55886443a180 2024-05-10T04:05:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443b9a0 0x558864443790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443b9a0 0x558864443790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50816/0 (socket says 172.21.15.150:50816) 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f213b7fe700 1 -- 172.21.15.150:0/3795181527 learned_addr learned my addr 172.21.15.150:0/3795181527 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f213a7fc700 1 --2- 172.21.15.150:0/3795181527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b050 0x55886444b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f213affd700 1 --2- 172.21.15.150:0/3795181527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443a180 0x55886444bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f213b7fe700 1 -- 172.21.15.150:0/3795181527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443a180 msgr2=0x55886444bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f213b7fe700 1 --2- 172.21.15.150:0/3795181527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443a180 0x55886444bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f213b7fe700 1 -- 172.21.15.150:0/3795181527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b050 msgr2=0x55886444b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f213b7fe700 1 --2- 172.21.15.150:0/3795181527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b050 0x55886444b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f213b7fe700 1 -- 172.21.15.150:0/3795181527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558864320500 con 0x55886443b9a0 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.330+0000 7f213affd700 1 --2- 172.21.15.150:0/3795181527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443a180 0x55886444bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.331+0000 7f213b7fe700 1 --2- 172.21.15.150:0/3795181527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443b9a0 0x558864443790 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f21340973f0 tx=0x7f2134004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a205af69be81372 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.331+0000 7f21227fc700 1 -- 172.21.15.150:0/3795181527 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213409e800 con 0x55886443b9a0 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.331+0000 7f21227fc700 1 -- 172.21.15.150:0/3795181527 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21340a2dc0 con 0x55886443b9a0 2024-05-10T04:05:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.331+0000 7f21227fc700 1 -- 172.21.15.150:0/3795181527 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213409ae90 con 0x55886443b9a0 2024-05-10T04:05:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.332+0000 7f2151c24b80 1 -- 172.21.15.150:0/3795181527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443b9a0 msgr2=0x558864443790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.332+0000 7f2151c24b80 1 --2- 172.21.15.150:0/3795181527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443b9a0 0x558864443790 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f21340973f0 tx=0x7f2134004ad0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.332+0000 7f2151c24b80 1 -- 172.21.15.150:0/3795181527 shutdown_connections 2024-05-10T04:05:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.332+0000 7f2151c24b80 1 --2- 172.21.15.150:0/3795181527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443b9a0 0x558864443790 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.332+0000 7f2151c24b80 1 --2- 172.21.15.150:0/3795181527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443a180 0x55886444bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.332+0000 7f2151c24b80 1 --2- 172.21.15.150:0/3795181527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b050 0x55886444b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.332+0000 7f2151c24b80 1 -- 172.21.15.150:0/3795181527 >> 172.21.15.150:0/3795181527 conn(0x55886439ca00 msgr2=0x5588644441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.332+0000 7f2151c24b80 1 -- 172.21.15.150:0/3795181527 shutdown_connections 2024-05-10T04:05:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.332+0000 7f2151c24b80 1 -- 172.21.15.150:0/3795181527 wait complete. 2024-05-10T04:05:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.333+0000 7f2151c24b80 1 -- start start 2024-05-10T04:05:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.333+0000 7f2151c24b80 1 Processor -- start 2024-05-10T04:05:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.333+0000 7f2151c24b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x558864448cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.333+0000 7f2151c24b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b050 0x558864449230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.333+0000 7f2151c24b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b9a0 0x55886444a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.333+0000 7f2151c24b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55886444a600 con 0x55886443a180 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.333+0000 7f2151c24b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55886444a770 con 0x55886443b9a0 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.333+0000 7f2151c24b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55886444a930 con 0x55886443b050 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.333+0000 7f2151c24b80 10 client.0 _ll_register_callbacks cb 0x5588642ffad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.334+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b9a0 0x55886444a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.334+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b9a0 0x55886444a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:44174/0 (socket says 172.21.15.150:44174) 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.334+0000 7f213b7fe700 1 -- 172.21.15.150:0/3023201805 learned_addr learned my addr 172.21.15.150:0/3023201805 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.334+0000 7f213b7fe700 1 -- 172.21.15.150:0/3023201805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b050 msgr2=0x558864449230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.334+0000 7f213b7fe700 1 --2- 172.21.15.150:0/3023201805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b050 0x558864449230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.334+0000 7f213b7fe700 1 -- 172.21.15.150:0/3023201805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 msgr2=0x558864448cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.334+0000 7f213b7fe700 1 --2- 172.21.15.150:0/3023201805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x558864448cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.334+0000 7f213b7fe700 1 -- 172.21.15.150:0/3023201805 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c19c290 con 0x55886443b9a0 2024-05-10T04:05:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.334+0000 7f213b7fe700 1 --2- 172.21.15.150:0/3023201805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b9a0 0x55886444a0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f213400c6f0 tx=0x7f2134003b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a85103f99c768c76 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.334+0000 7f211affd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f211affd700 1 -- 172.21.15.150:0/3023201805 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213409e800 con 0x55886443b9a0 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f211affd700 1 -- 172.21.15.150:0/3023201805 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f213409ac30 con 0x55886443b9a0 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f2151c24b80 1 -- 172.21.15.150:0/3023201805 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f212c021070 con 0x55886443b9a0 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f211affd700 1 -- 172.21.15.150:0/3023201805 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f213409ae30 con 0x55886443b9a0 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f2151c24b80 10 client.25640 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f2151c24b80 1 -- 172.21.15.150:0/3023201805 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5588641dfc70 con 0x55886443b9a0 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f211affd700 1 -- 172.21.15.150:0/3023201805 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f212c021070 con 0x55886443b9a0 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f211affd700 1 -- 172.21.15.150:0/3023201805 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f21340967c0 con 0x55886443b9a0 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f2151c24b80 10 client.25640 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f2151c24b80 10 client.25640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f2151c24b80 20 client.25640 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f2151c24b80 1 --2- 172.21.15.150:0/3023201805 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5588644a6500 0x5588644a8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f2151c24b80 4 client.25640 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:05:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.335+0000 7f2151c24b80 1 -- 172.21.15.150:0/3023201805 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558864352e60 con 0x5588644a6500 2024-05-10T04:05:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.336+0000 7f213affd700 1 --2- 172.21.15.150:0/3023201805 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5588644a6500 0x5588644a8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.336+0000 7f211affd700 10 client.25640 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.336+0000 7f213affd700 1 --2- 172.21.15.150:0/3023201805 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5588644a6500 0x5588644a8920 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eacc8174055f8a83 server_cookie=827842f9410ce3f in_seq=0 out_seq=0 2024-05-10T04:05:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.340+0000 7f211affd700 1 -- 172.21.15.150:0/3023201805 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558864352e60 con 0x5588644a6500 2024-05-10T04:05:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.340+0000 7f211affd700 10 client.25640 handle_command_reply: tid=0 2024-05-10T04:05:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.340+0000 7f2151c24b80 1 client.25640 shutdown 2024-05-10T04:05:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.341+0000 7f2151c24b80 1 -- 172.21.15.150:0/3023201805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b9a0 msgr2=0x55886444a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.341+0000 7f2151c24b80 1 --2- 172.21.15.150:0/3023201805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b9a0 0x55886444a0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f213400c6f0 tx=0x7f2134003b60 comp rx=0 tx=0).stop 2024-05-10T04:05:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.341+0000 7f2151c24b80 1 -- 172.21.15.150:0/3023201805 shutdown_connections 2024-05-10T04:05:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.341+0000 7f2151c24b80 1 --2- 172.21.15.150:0/3023201805 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5588644a6500 0x5588644a8920 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.341+0000 7f2151c24b80 1 --2- 172.21.15.150:0/3023201805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55886443b9a0 0x55886444a0c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.342+0000 7f211affd700 0 client.25640 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.342+0000 7f2151c24b80 1 --2- 172.21.15.150:0/3023201805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55886443a180 0x558864448cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.342+0000 7f2151c24b80 1 --2- 172.21.15.150:0/3023201805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55886443b050 0x558864449230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.342+0000 7f2151c24b80 1 -- 172.21.15.150:0/3023201805 >> 172.21.15.150:0/3023201805 conn(0x55886439ca00 msgr2=0x5588644440f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.342+0000 7f2151c24b80 1 -- 172.21.15.150:0/3023201805 shutdown_connections 2024-05-10T04:05:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.342+0000 7f2151c24b80 1 -- 172.21.15.150:0/3023201805 wait complete. 2024-05-10T04:05:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.342+0000 7f2151c24b80 20 client.25640 trim_cache size 0 max 16384 2024-05-10T04:05:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:05:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:05:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:05:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:05:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.342+0000 7f21418e0700 1 -- 172.21.15.150:0/1871646327 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f21240a11b0 msgr2=0x7f21240a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.342+0000 7f21418e0700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f21240a11b0 0x7f21240a36c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f213c069920 tx=0x7f212c009260 comp rx=0 tx=0).stop 2024-05-10T04:05:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.342+0000 7f21418e0700 1 -- 172.21.15.150:0/1871646327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c1031c0 msgr2=0x7f213c101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.342+0000 7f21418e0700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c1031c0 0x7f213c101370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f213400efb0 tx=0x7f213400c3a0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.342+0000 7f21418e0700 1 -- 172.21.15.150:0/1871646327 shutdown_connections 2024-05-10T04:05:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.343+0000 7f21418e0700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f21240a11b0 0x7f21240a36c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.343+0000 7f21418e0700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c1031c0 0x7f213c101370 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.343+0000 7f21418e0700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f213c0ffc00 0x7f213c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.343+0000 7f21418e0700 1 --2- 172.21.15.150:0/1871646327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f213c100550 0x7f213c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.343+0000 7f21418e0700 1 -- 172.21.15.150:0/1871646327 >> 172.21.15.150:0/1871646327 conn(0x7f213c0fb8d0 msgr2=0x7f213c106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.344+0000 7f21418e0700 1 -- 172.21.15.150:0/1871646327 shutdown_connections 2024-05-10T04:05:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:18.344+0000 7f21418e0700 1 -- 172.21.15.150:0/1871646327 wait complete. 2024-05-10T04:05:18.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:18 smithi150 ceph-mon[42477]: scrub queued for path: / 2024-05-10T04:05:18.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:18 smithi150 ceph-mon[42477]: scrub summary: idle+waiting paths [/] 2024-05-10T04:05:18.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:18 smithi150 ceph-mon[42477]: scrub summary: active paths [/] 2024-05-10T04:05:18.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:18 smithi150 ceph-mon[42477]: scrub summary: idle+waiting paths [/] 2024-05-10T04:05:18.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:18 smithi181 ceph-mon[46602]: scrub queued for path: / 2024-05-10T04:05:18.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:18 smithi181 ceph-mon[46602]: scrub summary: idle+waiting paths [/] 2024-05-10T04:05:18.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:18 smithi181 ceph-mon[46602]: scrub summary: active paths [/] 2024-05-10T04:05:18.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:18 smithi181 ceph-mon[46602]: scrub summary: idle+waiting paths [/] 2024-05-10T04:05:18.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:18 smithi161 ceph-mon[46918]: scrub queued for path: / 2024-05-10T04:05:18.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:18 smithi161 ceph-mon[46918]: scrub summary: idle+waiting paths [/] 2024-05-10T04:05:18.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:18 smithi161 ceph-mon[46918]: scrub summary: active paths [/] 2024-05-10T04:05:18.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:18 smithi161 ceph-mon[46918]: scrub summary: idle+waiting paths [/] 2024-05-10T04:05:19.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:19 smithi150 ceph-mon[42477]: pgmap v312: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T04:05:19.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:19 smithi181 ceph-mon[46602]: pgmap v312: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T04:05:19.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:19 smithi161 ceph-mon[46918]: pgmap v312: 129 pgs: 129 active+clean; 669 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T04:05:21.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:21 smithi150 ceph-mon[42477]: scrub summary: idle 2024-05-10T04:05:21.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:21 smithi181 ceph-mon[46602]: scrub summary: idle 2024-05-10T04:05:21.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:21 smithi161 ceph-mon[46918]: scrub summary: idle 2024-05-10T04:05:22.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:22 smithi150 ceph-mon[42477]: pgmap v313: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-10T04:05:22.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:22 smithi181 ceph-mon[46602]: pgmap v313: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-10T04:05:22.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:22 smithi161 ceph-mon[46918]: pgmap v313: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-10T04:05:24.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:24 smithi150 ceph-mon[42477]: pgmap v314: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T04:05:24.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:24 smithi181 ceph-mon[46602]: pgmap v314: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T04:05:24.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:24 smithi161 ceph-mon[46918]: pgmap v314: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T04:05:26.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:26 smithi150 ceph-mon[42477]: pgmap v315: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T04:05:26.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:26 smithi181 ceph-mon[46602]: pgmap v315: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T04:05:26.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:26 smithi161 ceph-mon[46918]: pgmap v315: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T04:05:28.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:28 smithi150 ceph-mon[42477]: pgmap v316: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T04:05:28.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:28 smithi181 ceph-mon[46602]: pgmap v316: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T04:05:28.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:28 smithi161 ceph-mon[46918]: pgmap v316: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T04:05:30.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:30 smithi150 ceph-mon[42477]: pgmap v317: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T04:05:30.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:30 smithi181 ceph-mon[46602]: pgmap v317: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T04:05:30.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:30 smithi161 ceph-mon[46918]: pgmap v317: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T04:05:31.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:31 smithi150 ceph-mon[42477]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-10T04:05:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.465+0000 7fbbed0cc700 1 -- 172.21.15.150:0/3888088255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe8100ae0 msgr2=0x7fbbe8100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.465+0000 7fbbed0cc700 1 --2- 172.21.15.150:0/3888088255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe8100ae0 0x7fbbe8100ef0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009990 tx=0x7fbbd8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.465+0000 7fbbed0cc700 1 -- 172.21.15.150:0/3888088255 shutdown_connections 2024-05-10T04:05:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.465+0000 7fbbed0cc700 1 --2- 172.21.15.150:0/3888088255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe81025a0 0x7fbbe8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.465+0000 7fbbed0cc700 1 --2- 172.21.15.150:0/3888088255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe8100ae0 0x7fbbe8100ef0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.465+0000 7fbbed0cc700 1 --2- 172.21.15.150:0/3888088255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbe8101c00 0x7fbbe8102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.465+0000 7fbbed0cc700 1 -- 172.21.15.150:0/3888088255 >> 172.21.15.150:0/3888088255 conn(0x7fbbe80fc0d0 msgr2=0x7fbbe80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.465+0000 7fbbed0cc700 1 -- 172.21.15.150:0/3888088255 shutdown_connections 2024-05-10T04:05:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.465+0000 7fbbed0cc700 1 -- 172.21.15.150:0/3888088255 wait complete. 2024-05-10T04:05:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.466+0000 7fbbed0cc700 1 Processor -- start 2024-05-10T04:05:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.466+0000 7fbbed0cc700 1 -- start start 2024-05-10T04:05:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbed0cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbe8100ae0 0x7fbbe8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbed0cc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8101c00 0x7fbbe81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbed0cc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe81025a0 0x7fbbe819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbed0cc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe819fa50 con 0x7fbbe81025a0 2024-05-10T04:05:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbed0cc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe819fb90 con 0x7fbbe8101c00 2024-05-10T04:05:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbed0cc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe819fd20 con 0x7fbbe8100ae0 2024-05-10T04:05:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbe6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbe8100ae0 0x7fbbe8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbe759e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe81025a0 0x7fbbe819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbe6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbe8100ae0 0x7fbbe8197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:58950/0 (socket says 172.21.15.150:58950) 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbe759e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe81025a0 0x7fbbe819f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44296/0 (socket says 172.21.15.150:44296) 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbe6d9d700 1 -- 172.21.15.150:0/2953184182 learned_addr learned my addr 172.21.15.150:0/2953184182 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbe659c700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8101c00 0x7fbbe81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbe6d9d700 1 -- 172.21.15.150:0/2953184182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8101c00 msgr2=0x7fbbe81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbe6d9d700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8101c00 0x7fbbe81984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbe6d9d700 1 -- 172.21.15.150:0/2953184182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe81025a0 msgr2=0x7fbbe819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbe6d9d700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe81025a0 0x7fbbe819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbe6d9d700 1 -- 172.21.15.150:0/2953184182 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd80095f0 con 0x7fbbe8100ae0 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.467+0000 7fbbe759e700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe81025a0 0x7fbbe819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.468+0000 7fbbe6d9d700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbe8100ae0 0x7fbbe8197fb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8005740 tx=0x7fbbd8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.468+0000 7fbbcffff700 1 -- 172.21.15.150:0/2953184182 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd801d070 con 0x7fbbe8100ae0 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.468+0000 7fbbed0cc700 1 -- 172.21.15.150:0/2953184182 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe819ffa0 con 0x7fbbe8100ae0 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.468+0000 7fbbcffff700 1 -- 172.21.15.150:0/2953184182 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbd8021410 con 0x7fbbe8100ae0 2024-05-10T04:05:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.468+0000 7fbbcffff700 1 -- 172.21.15.150:0/2953184182 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8017650 con 0x7fbbe8100ae0 2024-05-10T04:05:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.468+0000 7fbbed0cc700 1 -- 172.21.15.150:0/2953184182 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe81a0310 con 0x7fbbe8100ae0 2024-05-10T04:05:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.472+0000 7fbbcffff700 1 -- 172.21.15.150:0/2953184182 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbbd80177b0 con 0x7fbbe8100ae0 2024-05-10T04:05:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.472+0000 7fbbcffff700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbbd00a11a0 0x7fbbd00a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.473+0000 7fbbcffff700 1 -- 172.21.15.150:0/2953184182 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbbd80cdd10 con 0x7fbbe8100ae0 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.473+0000 7fbbfd410b80 1 Processor -- start 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.473+0000 7fbbfd410b80 1 -- start start 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.473+0000 7fbbfd410b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46d0a0 0x557b0e46d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.473+0000 7fbbfd410b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46da50 0x557b0e47abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.473+0000 7fbbfd410b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e47d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.473+0000 7fbbfd410b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e3940f0 con 0x557b0e46da50 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.473+0000 7fbbfd410b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e391ab0 con 0x557b0e47b1c0 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.473+0000 7fbbfd410b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e38d3a0 con 0x557b0e46d0a0 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.473+0000 7fbbe6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46d0a0 0x557b0e46d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.473+0000 7fbbe759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e47d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.473+0000 7fbbe6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46d0a0 0x557b0e46d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:58962/0 (socket says 172.21.15.150:58962) 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.473+0000 7fbbe6d9d700 1 -- 172.21.15.150:0/1101169599 learned_addr learned my addr 172.21.15.150:0/1101169599 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.474+0000 7fbbe659c700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbbd00a11a0 0x7fbbd00a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.474+0000 7fbbe759e700 1 -- 172.21.15.150:0/1101169599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46d0a0 msgr2=0x557b0e46d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.474+0000 7fbbe759e700 1 --2- 172.21.15.150:0/1101169599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46d0a0 0x557b0e46d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.474+0000 7fbbe759e700 1 -- 172.21.15.150:0/1101169599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46da50 msgr2=0x557b0e47abc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:05:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.474+0000 7fbbe759e700 1 --2- 172.21.15.150:0/1101169599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46da50 0x557b0e47abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.474+0000 7fbbe759e700 1 -- 172.21.15.150:0/1101169599 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b0e20e350 con 0x557b0e47b1c0 2024-05-10T04:05:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.474+0000 7fbbe6d9d700 1 --2- 172.21.15.150:0/1101169599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46d0a0 0x557b0e46d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.474+0000 7fbbe759e700 1 --2- 172.21.15.150:0/1101169599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e47d6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0003db0 tx=0x7fbbe0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a895609d2ba07b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.474+0000 7fbbcd7fa700 1 -- 172.21.15.150:0/1101169599 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0004bb0 con 0x557b0e47b1c0 2024-05-10T04:05:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.474+0000 7fbbcd7fa700 1 -- 172.21.15.150:0/1101169599 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbe000f040 con 0x557b0e47b1c0 2024-05-10T04:05:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.474+0000 7fbbcd7fa700 1 -- 172.21.15.150:0/1101169599 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0007cb0 con 0x557b0e47b1c0 2024-05-10T04:05:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.475+0000 7fbbfd410b80 1 -- 172.21.15.150:0/1101169599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 msgr2=0x557b0e47d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.475+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/1101169599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e47d6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0003db0 tx=0x7fbbe0009920 comp rx=0 tx=0).stop 2024-05-10T04:05:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.475+0000 7fbbe659c700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbbd00a11a0 0x7fbbd00a36b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8199050 tx=0x7fbbdc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.475+0000 7fbbfd410b80 1 -- 172.21.15.150:0/1101169599 shutdown_connections 2024-05-10T04:05:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.475+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/1101169599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e47d6a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.475+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/1101169599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46d0a0 0x557b0e46d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.475+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/1101169599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46da50 0x557b0e47abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.475+0000 7fbbfd410b80 1 -- 172.21.15.150:0/1101169599 >> 172.21.15.150:0/1101169599 conn(0x557b0e3cfa00 msgr2=0x557b0e467a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.475+0000 7fbbfd410b80 1 -- 172.21.15.150:0/1101169599 shutdown_connections 2024-05-10T04:05:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.475+0000 7fbbfd410b80 1 -- 172.21.15.150:0/1101169599 wait complete. 2024-05-10T04:05:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.476+0000 7fbbfd410b80 1 -- start start 2024-05-10T04:05:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.476+0000 7fbbfd410b80 1 Processor -- start 2024-05-10T04:05:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.476+0000 7fbbfd410b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46d0a0 0x557b0e47f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.476+0000 7fbbfd410b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46da50 0x557b0e47feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.476+0000 7fbbfd410b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e486e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.476+0000 7fbbfd410b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e329500 con 0x557b0e46d0a0 2024-05-10T04:05:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.476+0000 7fbbfd410b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e3291a0 con 0x557b0e47b1c0 2024-05-10T04:05:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.476+0000 7fbbfd410b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e210640 con 0x557b0e46da50 2024-05-10T04:05:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.476+0000 7fbbfd410b80 10 client.0 _ll_register_callbacks cb 0x557b0e332ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.476+0000 7fbbe6d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46d0a0 0x557b0e47f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.476+0000 7fbbe759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e486e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.476+0000 7fbbe6d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46d0a0 0x557b0e47f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44302/0 (socket says 172.21.15.150:44302) 2024-05-10T04:05:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.476+0000 7fbbe6d9d700 1 -- 172.21.15.150:0/2384004507 learned_addr learned my addr 172.21.15.150:0/2384004507 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.477+0000 7fbbe759e700 1 -- 172.21.15.150:0/2384004507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46da50 msgr2=0x557b0e47feb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.477+0000 7fbbe759e700 1 --2- 172.21.15.150:0/2384004507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46da50 0x557b0e47feb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.477+0000 7fbbe759e700 1 -- 172.21.15.150:0/2384004507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46d0a0 msgr2=0x557b0e47f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.477+0000 7fbbe759e700 1 --2- 172.21.15.150:0/2384004507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46d0a0 0x557b0e47f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.477+0000 7fbbe759e700 1 -- 172.21.15.150:0/2384004507 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe81a0310 con 0x557b0e47b1c0 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.477+0000 7fbbe6d9d700 1 --2- 172.21.15.150:0/2384004507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46d0a0 0x557b0e47f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.477+0000 7fbbe759e700 1 --2- 172.21.15.150:0/2384004507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e486e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0004600 tx=0x7fbbe0004a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd1a570e442cb6eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.477+0000 7fbbcaff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.477+0000 7fbbcaff5700 1 -- 172.21.15.150:0/2384004507 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0007a30 con 0x557b0e47b1c0 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.477+0000 7fbbfd410b80 1 -- 172.21.15.150:0/2384004507 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b0e2e6210 con 0x557b0e47b1c0 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.477+0000 7fbbcaff5700 1 -- 172.21.15.150:0/2384004507 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbbe0007b90 con 0x557b0e47b1c0 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.478+0000 7fbbcaff5700 1 -- 172.21.15.150:0/2384004507 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbe0007d50 con 0x557b0e47b1c0 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.478+0000 7fbbfd410b80 10 client.25652 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.478+0000 7fbbfd410b80 1 -- 172.21.15.150:0/2384004507 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fbbe0003a60 con 0x557b0e47b1c0 2024-05-10T04:05:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.478+0000 7fbbcaff5700 1 -- 172.21.15.150:0/2384004507 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557b0e2e6210 con 0x557b0e47b1c0 2024-05-10T04:05:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.479+0000 7fbbcaff5700 1 -- 172.21.15.150:0/2384004507 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbbe000e700 con 0x557b0e47b1c0 2024-05-10T04:05:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.479+0000 7fbbfd410b80 10 client.25652 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.479+0000 7fbbfd410b80 10 client.25652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.479+0000 7fbbfd410b80 20 client.25652 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.479+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/2384004507 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557b0e48b650 0x557b0e48daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.479+0000 7fbbfd410b80 4 client.25652 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.479+0000 7fbbfd410b80 1 -- 172.21.15.150:0/2384004507 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b0e43c720 con 0x557b0e48b650 2024-05-10T04:05:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.479+0000 7fbbe6d9d700 1 --2- 172.21.15.150:0/2384004507 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557b0e48b650 0x557b0e48daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.480+0000 7fbbe6d9d700 1 --2- 172.21.15.150:0/2384004507 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557b0e48b650 0x557b0e48daa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ddcc6833ba1951 server_cookie=bac8a3502994338e in_seq=0 out_seq=0 2024-05-10T04:05:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.480+0000 7fbbcaff5700 10 client.25652 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbcaff5700 1 -- 172.21.15.150:0/2384004507 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557b0e43c720 con 0x557b0e48b650 2024-05-10T04:05:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbcaff5700 10 client.25652 handle_command_reply: tid=0 2024-05-10T04:05:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbfd410b80 1 client.25652 shutdown 2024-05-10T04:05:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbfd410b80 1 -- 172.21.15.150:0/2384004507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 msgr2=0x557b0e486e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/2384004507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e486e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0004600 tx=0x7fbbe0004a20 comp rx=0 tx=0).stop 2024-05-10T04:05:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbfd410b80 1 -- 172.21.15.150:0/2384004507 shutdown_connections 2024-05-10T04:05:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/2384004507 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557b0e48b650 0x557b0e48daa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/2384004507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e486e10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/2384004507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46d0a0 0x557b0e47f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/2384004507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46da50 0x557b0e47feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbfd410b80 1 -- 172.21.15.150:0/2384004507 >> 172.21.15.150:0/2384004507 conn(0x557b0e3cfa00 msgr2=0x557b0e467a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbcaff5700 0 client.25652 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbfd410b80 1 -- 172.21.15.150:0/2384004507 shutdown_connections 2024-05-10T04:05:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbfd410b80 1 -- 172.21.15.150:0/2384004507 wait complete. 2024-05-10T04:05:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.481+0000 7fbbfd410b80 20 client.25652 trim_cache size 0 max 16384 2024-05-10T04:05:31.488 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:31 smithi181 ceph-mon[46602]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-10T04:05:31.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:31 smithi161 ceph-mon[46918]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-10T04:05:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.496+0000 7fbbfd410b80 1 Processor -- start 2024-05-10T04:05:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.496+0000 7fbbfd410b80 1 -- start start 2024-05-10T04:05:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.496+0000 7fbbfd410b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e46d0a0 0x557b0e467880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.496+0000 7fbbfd410b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46da50 0x557b0e486dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.496+0000 7fbbfd410b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e47b1c0 0x557b0e48d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.496+0000 7fbbfd410b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e47d830 con 0x557b0e46da50 2024-05-10T04:05:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.496+0000 7fbbfd410b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e48b370 con 0x557b0e46d0a0 2024-05-10T04:05:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.496+0000 7fbbfd410b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e469200 con 0x557b0e47b1c0 2024-05-10T04:05:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbe659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46da50 0x557b0e486dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbe659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46da50 0x557b0e486dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44304/0 (socket says 172.21.15.150:44304) 2024-05-10T04:05:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbe659c700 1 -- 172.21.15.150:0/198673199 learned_addr learned my addr 172.21.15.150:0/198673199 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbe6d9d700 1 --2- 172.21.15.150:0/198673199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e46d0a0 0x557b0e467880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbe759e700 1 --2- 172.21.15.150:0/198673199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e47b1c0 0x557b0e48d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbe659c700 1 -- 172.21.15.150:0/198673199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e47b1c0 msgr2=0x557b0e48d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbe659c700 1 --2- 172.21.15.150:0/198673199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e47b1c0 0x557b0e48d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbe659c700 1 -- 172.21.15.150:0/198673199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e46d0a0 msgr2=0x557b0e467880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbe659c700 1 --2- 172.21.15.150:0/198673199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e46d0a0 0x557b0e467880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbe659c700 1 -- 172.21.15.150:0/198673199 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b0e1bab00 con 0x557b0e46da50 2024-05-10T04:05:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbe759e700 1 --2- 172.21.15.150:0/198673199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e47b1c0 0x557b0e48d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:05:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbe6d9d700 1 --2- 172.21.15.150:0/198673199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e46d0a0 0x557b0e467880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbe659c700 1 --2- 172.21.15.150:0/198673199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46da50 0x557b0e486dd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00b6c0 tx=0x7fbbdc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70731ce369107fda server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.497+0000 7fbbcdffb700 1 -- 172.21.15.150:0/198673199 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc011de0 con 0x557b0e46da50 2024-05-10T04:05:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.498+0000 7fbbcdffb700 1 -- 172.21.15.150:0/198673199 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbdc015070 con 0x557b0e46da50 2024-05-10T04:05:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.498+0000 7fbbcdffb700 1 -- 172.21.15.150:0/198673199 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc01c610 con 0x557b0e46da50 2024-05-10T04:05:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.498+0000 7fbbfd410b80 1 -- 172.21.15.150:0/198673199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46da50 msgr2=0x557b0e486dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.498+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/198673199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46da50 0x557b0e486dd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00b6c0 tx=0x7fbbdc013000 comp rx=0 tx=0).stop 2024-05-10T04:05:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.498+0000 7fbbfd410b80 1 -- 172.21.15.150:0/198673199 shutdown_connections 2024-05-10T04:05:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.498+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/198673199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e47b1c0 0x557b0e48d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.498+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/198673199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e46d0a0 0x557b0e467880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.498+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/198673199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46da50 0x557b0e486dd0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.498+0000 7fbbfd410b80 1 -- 172.21.15.150:0/198673199 >> 172.21.15.150:0/198673199 conn(0x557b0e3cfa00 msgr2=0x557b0e492370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.499+0000 7fbbfd410b80 1 -- 172.21.15.150:0/198673199 shutdown_connections 2024-05-10T04:05:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.499+0000 7fbbfd410b80 1 -- 172.21.15.150:0/198673199 wait complete. 2024-05-10T04:05:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.499+0000 7fbbfd410b80 1 -- start start 2024-05-10T04:05:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.499+0000 7fbbfd410b80 1 Processor -- start 2024-05-10T04:05:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.499+0000 7fbbfd410b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46d0a0 0x557b0e483b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.499+0000 7fbbfd410b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46da50 0x557b0e4840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.499+0000 7fbbfd410b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e4dfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.499+0000 7fbbfd410b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e487010 con 0x557b0e46d0a0 2024-05-10T04:05:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.499+0000 7fbbfd410b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e487180 con 0x557b0e47b1c0 2024-05-10T04:05:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.499+0000 7fbbfd410b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0e4e0010 con 0x557b0e46da50 2024-05-10T04:05:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbfd410b80 10 client.0 _ll_register_callbacks cb 0x557b0e332ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbe759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e4dfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbe759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e4dfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:49800/0 (socket says 172.21.15.150:49800) 2024-05-10T04:05:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbe759e700 1 -- 172.21.15.150:0/1592284086 learned_addr learned my addr 172.21.15.150:0/1592284086 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbe6d9d700 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46d0a0 0x557b0e483b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbe659c700 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46da50 0x557b0e4840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbe759e700 1 -- 172.21.15.150:0/1592284086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46da50 msgr2=0x557b0e4840d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbe759e700 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46da50 0x557b0e4840d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbe759e700 1 -- 172.21.15.150:0/1592284086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46d0a0 msgr2=0x557b0e483b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbe759e700 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46d0a0 0x557b0e483b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbe759e700 1 -- 172.21.15.150:0/1592284086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe819ffa0 con 0x557b0e47b1c0 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbe659c700 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46da50 0x557b0e4840d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbe759e700 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e4dfad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0002230 tx=0x7fbbe0009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b614f3ac5ac27d53 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.500+0000 7fbbcaff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.501+0000 7fbbcaff5700 1 -- 172.21.15.150:0/1592284086 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0007bc0 con 0x557b0e47b1c0 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.501+0000 7fbbfd410b80 1 -- 172.21.15.150:0/1592284086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbe000f040 con 0x557b0e47b1c0 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.501+0000 7fbbcaff5700 1 -- 172.21.15.150:0/1592284086 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbbe0007d20 con 0x557b0e47b1c0 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.501+0000 7fbbcaff5700 1 -- 172.21.15.150:0/1592284086 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbe003d5f0 con 0x557b0e47b1c0 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.501+0000 7fbbfd410b80 10 client.25658 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.501+0000 7fbbfd410b80 1 -- 172.21.15.150:0/1592284086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fbbdc010ba0 con 0x557b0e47b1c0 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.501+0000 7fbbcaff5700 1 -- 172.21.15.150:0/1592284086 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbe000f040 con 0x557b0e47b1c0 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.502+0000 7fbbcaff5700 1 -- 172.21.15.150:0/1592284086 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbbe001f450 con 0x557b0e47b1c0 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.502+0000 7fbbfd410b80 10 client.25658 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.502+0000 7fbbfd410b80 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.502+0000 7fbbfd410b80 20 client.25658 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.502+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557b0e4e15e0 0x557b0e4e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.502+0000 7fbbfd410b80 4 client.25658 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:05:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.502+0000 7fbbfd410b80 1 -- 172.21.15.150:0/1592284086 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557b0e385e60 con 0x557b0e4e15e0 2024-05-10T04:05:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.502+0000 7fbbe6d9d700 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557b0e4e15e0 0x557b0e4e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.502+0000 7fbbcaff5700 10 client.25658 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.502+0000 7fbbe6d9d700 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557b0e4e15e0 0x557b0e4e3a00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f2ced17008e84 server_cookie=1f53fbc4881cf7bc in_seq=0 out_seq=0 2024-05-10T04:05:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.509+0000 7fbbcaff5700 1 -- 172.21.15.150:0/1592284086 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557b0e385e60 con 0x557b0e4e15e0 2024-05-10T04:05:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.509+0000 7fbbcaff5700 10 client.25658 handle_command_reply: tid=0 2024-05-10T04:05:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.509+0000 7fbbfd410b80 1 client.25658 shutdown 2024-05-10T04:05:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.509+0000 7fbbfd410b80 1 -- 172.21.15.150:0/1592284086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 msgr2=0x557b0e4dfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.509+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e4dfad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0002230 tx=0x7fbbe0009990 comp rx=0 tx=0).stop 2024-05-10T04:05:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbfd410b80 1 -- 172.21.15.150:0/1592284086 shutdown_connections 2024-05-10T04:05:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557b0e4e15e0 0x557b0e4e3a00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b0e47b1c0 0x557b0e4dfad0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557b0e46d0a0 0x557b0e483b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbcaff5700 0 client.25658 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbfd410b80 1 --2- 172.21.15.150:0/1592284086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557b0e46da50 0x557b0e4840d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbfd410b80 1 -- 172.21.15.150:0/1592284086 >> 172.21.15.150:0/1592284086 conn(0x557b0e3cfa00 msgr2=0x557b0e492370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbfd410b80 1 -- 172.21.15.150:0/1592284086 shutdown_connections 2024-05-10T04:05:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbfd410b80 1 -- 172.21.15.150:0/1592284086 wait complete. 2024-05-10T04:05:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbfd410b80 20 client.25658 trim_cache size 0 max 16384 2024-05-10T04:05:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:05:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:05:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:05:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:05:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbed0cc700 1 -- 172.21.15.150:0/2953184182 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbbd00a11a0 msgr2=0x7fbbd00a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbed0cc700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbbd00a11a0 0x7fbbd00a36b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8199050 tx=0x7fbbdc00b410 comp rx=0 tx=0).stop 2024-05-10T04:05:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbed0cc700 1 -- 172.21.15.150:0/2953184182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbe8100ae0 msgr2=0x7fbbe8197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.510+0000 7fbbed0cc700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbe8100ae0 0x7fbbe8197fb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8005740 tx=0x7fbbd8009580 comp rx=0 tx=0).stop 2024-05-10T04:05:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.511+0000 7fbbed0cc700 1 -- 172.21.15.150:0/2953184182 shutdown_connections 2024-05-10T04:05:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.511+0000 7fbbed0cc700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbbd00a11a0 0x7fbbd00a36b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.511+0000 7fbbed0cc700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbe81025a0 0x7fbbe819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.511+0000 7fbbed0cc700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbe8100ae0 0x7fbbe8197fb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.511+0000 7fbbed0cc700 1 --2- 172.21.15.150:0/2953184182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8101c00 0x7fbbe81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.511+0000 7fbbed0cc700 1 -- 172.21.15.150:0/2953184182 >> 172.21.15.150:0/2953184182 conn(0x7fbbe80fc0d0 msgr2=0x7fbbe8104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.511+0000 7fbbed0cc700 1 -- 172.21.15.150:0/2953184182 shutdown_connections 2024-05-10T04:05:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:31.511+0000 7fbbed0cc700 1 -- 172.21.15.150:0/2953184182 wait complete. 2024-05-10T04:05:32.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:32 smithi181 ceph-mon[46602]: pgmap v318: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 596 B/s wr, 8 op/s 2024-05-10T04:05:32.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:32 smithi161 ceph-mon[46918]: pgmap v318: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 596 B/s wr, 8 op/s 2024-05-10T04:05:32.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:32 smithi150 ceph-mon[42477]: pgmap v318: 129 pgs: 129 active+clean; 670 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 596 B/s wr, 8 op/s 2024-05-10T04:05:33.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:33 smithi181 ceph-mon[46602]: pgmap v319: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-10T04:05:33.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:33 smithi161 ceph-mon[46918]: pgmap v319: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-10T04:05:33.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:33 smithi150 ceph-mon[42477]: pgmap v319: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-10T04:05:34.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:34 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:05:34.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:34 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:05:34.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:34 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:05:35.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:35 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:05:35.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:35 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:05:35.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:35 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:05:35.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:35 smithi181 ceph-mon[46602]: pgmap v320: 129 pgs: 129 active+clean; 672 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T04:05:35.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:35 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:05:35.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:35 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:05:35.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:35 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:05:35.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:35 smithi161 ceph-mon[46918]: pgmap v320: 129 pgs: 129 active+clean; 672 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T04:05:35.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:35 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:05:35.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:35 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:05:35.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:35 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:05:35.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:35 smithi150 ceph-mon[42477]: pgmap v320: 129 pgs: 129 active+clean; 672 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T04:05:36.021 INFO:tasks.workunit.client.0.smithi150.stderr:Updating files: 96% (11835/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-10T04:05:36.414 INFO:tasks.workunit.client.0.smithi150.stderr:Note: switching to '50174015a7e79f951b988949abe236791f52d4da'. 2024-05-10T04:05:36.414 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:05:36.414 INFO:tasks.workunit.client.0.smithi150.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-10T04:05:36.414 INFO:tasks.workunit.client.0.smithi150.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-10T04:05:36.414 INFO:tasks.workunit.client.0.smithi150.stderr:state without impacting any branches by switching back to a branch. 2024-05-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi150.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi150.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi150.stderr: git switch -c 2024-05-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi150.stderr:Or undo this operation with: 2024-05-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi150.stderr: git switch - 2024-05-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi150.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi150.stderr:HEAD is now at 50174015a7e Merge PR #57343 into wip-pdonnell-testing-20240509.171335-reef-debug 2024-05-10T04:05:36.420 DEBUG:teuthology.orchestra.run.smithi150:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-10T04:05:36.490 INFO:tasks.workunit.client.0.smithi150.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-10T04:05:36.492 INFO:tasks.workunit.client.0.smithi150.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-10T04:05:36.492 INFO:tasks.workunit.client.0.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-10T04:05:36.968 INFO:tasks.workunit.client.0.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-10T04:05:37.033 INFO:tasks.workunit.client.0.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-10T04:05:37.070 INFO:tasks.workunit.client.0.smithi150.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-10T04:05:37.071 INFO:tasks.workunit.client.0.smithi150.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-10T04:05:37.071 INFO:tasks.workunit.client.0.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-10T04:05:37.108 INFO:tasks.workunit.client.0.smithi150.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-10T04:05:37.111 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:05:37.111 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-10T04:05:37.169 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-05-10T04:05:37.171 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-05-10T04:05:37.171 DEBUG:teuthology.orchestra.run.smithi150:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=50174015a7e79f951b988949abe236791f52d4da TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-05-10T04:05:37.233 INFO:tasks.workunit.client.0.smithi150.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-05-10T04:05:37.271 INFO:tasks.workunit.client.0.smithi150.stdout: Iozone: Performance Test of File I/O 2024-05-10T04:05:37.271 INFO:tasks.workunit.client.0.smithi150.stdout: Version $Revision: 3.489 $ 2024-05-10T04:05:37.271 INFO:tasks.workunit.client.0.smithi150.stdout: Compiled for 64 bit mode. 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: Build: linux-AMD64 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: Alexey Skidanov, Sudhir Kumar. 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: Run began: Fri May 10 04:05:37 2024 2024-05-10T04:05:37.272 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:37.273 INFO:tasks.workunit.client.0.smithi150.stdout: Include close in write timing 2024-05-10T04:05:37.273 INFO:tasks.workunit.client.0.smithi150.stdout: Include fsync in write timing 2024-05-10T04:05:37.273 INFO:tasks.workunit.client.0.smithi150.stdout: File size set to 1048576 kB 2024-05-10T04:05:37.273 INFO:tasks.workunit.client.0.smithi150.stdout: Record Size 16 kB 2024-05-10T04:05:37.273 INFO:tasks.workunit.client.0.smithi150.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-05-10T04:05:37.273 INFO:tasks.workunit.client.0.smithi150.stdout: Output is in kBytes/sec 2024-05-10T04:05:37.273 INFO:tasks.workunit.client.0.smithi150.stdout: Time Resolution = 0.000001 seconds. 2024-05-10T04:05:37.273 INFO:tasks.workunit.client.0.smithi150.stdout: Processor cache size set to 1024 kBytes. 2024-05-10T04:05:37.273 INFO:tasks.workunit.client.0.smithi150.stdout: Processor cache line size set to 32 bytes. 2024-05-10T04:05:37.273 INFO:tasks.workunit.client.0.smithi150.stdout: File stride size set to 17 * record size. 2024-05-10T04:05:37.273 INFO:tasks.workunit.client.0.smithi150.stdout: Throughput test with 1 process 2024-05-10T04:05:37.273 INFO:tasks.workunit.client.0.smithi150.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-05-10T04:05:38.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:38 smithi150 ceph-mon[42477]: pgmap v321: 129 pgs: 129 active+clean; 672 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T04:05:38.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:38 smithi181 ceph-mon[46602]: pgmap v321: 129 pgs: 129 active+clean; 672 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T04:05:38.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:38 smithi161 ceph-mon[46918]: pgmap v321: 129 pgs: 129 active+clean; 672 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T04:05:38.605 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:38.606 INFO:tasks.workunit.client.0.smithi150.stdout: Children see throughput for 1 initial writers = 787235.12 kB/sec 2024-05-10T04:05:38.606 INFO:tasks.workunit.client.0.smithi150.stdout: Parent sees throughput for 1 initial writers = 787089.20 kB/sec 2024-05-10T04:05:38.606 INFO:tasks.workunit.client.0.smithi150.stdout: Min throughput per process = 787235.12 kB/sec 2024-05-10T04:05:38.606 INFO:tasks.workunit.client.0.smithi150.stdout: Max throughput per process = 787235.12 kB/sec 2024-05-10T04:05:38.606 INFO:tasks.workunit.client.0.smithi150.stdout: Avg throughput per process = 787235.12 kB/sec 2024-05-10T04:05:38.606 INFO:tasks.workunit.client.0.smithi150.stdout: Min xfer = 1048576.00 kB 2024-05-10T04:05:40.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:40 smithi150 ceph-mon[42477]: pgmap v322: 129 pgs: 129 active+clean; 429 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 36 MiB/s wr, 15 op/s 2024-05-10T04:05:40.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:40 smithi181 ceph-mon[46602]: pgmap v322: 129 pgs: 129 active+clean; 429 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 36 MiB/s wr, 15 op/s 2024-05-10T04:05:40.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:40 smithi161 ceph-mon[46918]: pgmap v322: 129 pgs: 129 active+clean; 429 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 36 MiB/s wr, 15 op/s 2024-05-10T04:05:42.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:42 smithi150 ceph-mon[42477]: pgmap v323: 129 pgs: 129 active+clean; 617 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 51 MiB/s wr, 22 op/s 2024-05-10T04:05:42.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:42 smithi181 ceph-mon[46602]: pgmap v323: 129 pgs: 129 active+clean; 617 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 51 MiB/s wr, 22 op/s 2024-05-10T04:05:42.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:42 smithi161 ceph-mon[46918]: pgmap v323: 129 pgs: 129 active+clean; 617 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 51 MiB/s wr, 22 op/s 2024-05-10T04:05:44.253 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:44.253 INFO:tasks.workunit.client.0.smithi150.stdout: Children see throughput for 1 rewriters = 846169.69 kB/sec 2024-05-10T04:05:44.253 INFO:tasks.workunit.client.0.smithi150.stdout: Parent sees throughput for 1 rewriters = 846037.22 kB/sec 2024-05-10T04:05:44.253 INFO:tasks.workunit.client.0.smithi150.stdout: Min throughput per process = 846169.69 kB/sec 2024-05-10T04:05:44.254 INFO:tasks.workunit.client.0.smithi150.stdout: Max throughput per process = 846169.69 kB/sec 2024-05-10T04:05:44.254 INFO:tasks.workunit.client.0.smithi150.stdout: Avg throughput per process = 846169.69 kB/sec 2024-05-10T04:05:44.254 INFO:tasks.workunit.client.0.smithi150.stdout: Min xfer = 1048576.00 kB 2024-05-10T04:05:44.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:44 smithi150 ceph-mon[42477]: pgmap v324: 129 pgs: 129 active+clean; 997 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 83 MiB/s wr, 29 op/s 2024-05-10T04:05:44.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:44 smithi181 ceph-mon[46602]: pgmap v324: 129 pgs: 129 active+clean; 997 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 83 MiB/s wr, 29 op/s 2024-05-10T04:05:44.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:44 smithi161 ceph-mon[46918]: pgmap v324: 129 pgs: 129 active+clean; 997 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 83 MiB/s wr, 29 op/s 2024-05-10T04:05:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.636+0000 7f28e6dcf700 1 -- 172.21.15.150:0/1780133937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e0100ad0 msgr2=0x7f28e0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.636+0000 7f28e6dcf700 1 --2- 172.21.15.150:0/1780133937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e0100ad0 0x7f28e0100ee0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009990 tx=0x7f28d4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.636+0000 7f28e6dcf700 1 -- 172.21.15.150:0/1780133937 shutdown_connections 2024-05-10T04:05:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.636+0000 7f28e6dcf700 1 --2- 172.21.15.150:0/1780133937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28e0102590 0x7f28e0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.636+0000 7f28e6dcf700 1 --2- 172.21.15.150:0/1780133937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e0100ad0 0x7f28e0100ee0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.636+0000 7f28e6dcf700 1 --2- 172.21.15.150:0/1780133937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e0101bf0 0x7f28e0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.636+0000 7f28e6dcf700 1 -- 172.21.15.150:0/1780133937 >> 172.21.15.150:0/1780133937 conn(0x7f28e00fc0c0 msgr2=0x7f28e00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.636+0000 7f28e6dcf700 1 -- 172.21.15.150:0/1780133937 shutdown_connections 2024-05-10T04:05:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.636+0000 7f28e6dcf700 1 -- 172.21.15.150:0/1780133937 wait complete. 2024-05-10T04:05:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.637+0000 7f28e6dcf700 1 Processor -- start 2024-05-10T04:05:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.637+0000 7f28e6dcf700 1 -- start start 2024-05-10T04:05:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.637+0000 7f28e6dcf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28e0100ad0 0x7f28e006a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.637+0000 7f28e6dcf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e0101bf0 0x7f28e019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.637+0000 7f28e6dcf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e0102590 0x7f28e019f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.637+0000 7f28e6dcf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e019fa20 con 0x7f28e0101bf0 2024-05-10T04:05:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.637+0000 7f28e6dcf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e019fb90 con 0x7f28e0100ad0 2024-05-10T04:05:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.637+0000 7f28e6dcf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e019fd50 con 0x7f28e0102590 2024-05-10T04:05:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.637+0000 7f28dffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e0101bf0 0x7f28e019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.637+0000 7f28dffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e0101bf0 0x7f28e019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52946/0 (socket says 172.21.15.150:52946) 2024-05-10T04:05:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.637+0000 7f28dffff700 1 -- 172.21.15.150:0/3185933830 learned_addr learned my addr 172.21.15.150:0/3185933830 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.637+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28e0100ad0 0x7f28e006a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.637+0000 7f28e536c700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e0102590 0x7f28e019f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.638+0000 7f28dffff700 1 -- 172.21.15.150:0/3185933830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e0102590 msgr2=0x7f28e019f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.638+0000 7f28dffff700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e0102590 0x7f28e019f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.638+0000 7f28dffff700 1 -- 172.21.15.150:0/3185933830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28e0100ad0 msgr2=0x7f28e006a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.638+0000 7f28dffff700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28e0100ad0 0x7f28e006a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.638+0000 7f28dffff700 1 -- 172.21.15.150:0/3185933830 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d40095f0 con 0x7f28e0101bf0 2024-05-10T04:05:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.638+0000 7f28dffff700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e0101bf0 0x7f28e019cd70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00d950 tx=0x7f28cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.638+0000 7f28ddffb700 1 -- 172.21.15.150:0/3185933830 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc00f890 con 0x7f28e0101bf0 2024-05-10T04:05:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.638+0000 7f28ddffb700 1 -- 172.21.15.150:0/3185933830 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28cc010460 con 0x7f28e0101bf0 2024-05-10T04:05:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.638+0000 7f28e6dcf700 1 -- 172.21.15.150:0/3185933830 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e019ffd0 con 0x7f28e0101bf0 2024-05-10T04:05:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.638+0000 7f28ddffb700 1 -- 172.21.15.150:0/3185933830 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc00e610 con 0x7f28e0101bf0 2024-05-10T04:05:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.638+0000 7f28e6dcf700 1 -- 172.21.15.150:0/3185933830 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e01a0450 con 0x7f28e0101bf0 2024-05-10T04:05:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.640+0000 7f28ddffb700 1 -- 172.21.15.150:0/3185933830 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f28cc0108d0 con 0x7f28e0101bf0 2024-05-10T04:05:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.641+0000 7f28ddffb700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f28d00a1120 0x7f28d00a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.641+0000 7f28ddffb700 1 -- 172.21.15.150:0/3185933830 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f28cc0cca00 con 0x7f28e0101bf0 2024-05-10T04:05:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.641+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f28d00a1120 0x7f28d00a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f28d00a1120 0x7f28d00a3630 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f28d4006010 tx=0x7f28d401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28f7113b80 1 Processor -- start 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28f7113b80 1 -- start start 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28f7113b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f5680 0x5600620f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28f7113b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f6090 0x5600620f4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28f7113b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f4980 0x5600620fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28f7113b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600620170f0 con 0x5600620f5680 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28f7113b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560062014ab0 con 0x5600620f6090 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28f7113b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600620103a0 con 0x5600620f4980 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28e4b6b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f5680 0x5600620f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28e4b6b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f5680 0x5600620f5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52956/0 (socket says 172.21.15.150:52956) 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28e4b6b700 1 -- 172.21.15.150:0/3496596020 learned_addr learned my addr 172.21.15.150:0/3496596020 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28dffff700 1 --2- 172.21.15.150:0/3496596020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f6090 0x5600620f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.642+0000 7f28e536c700 1 --2- 172.21.15.150:0/3496596020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f4980 0x5600620fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.643+0000 7f28e4b6b700 1 -- 172.21.15.150:0/3496596020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f4980 msgr2=0x5600620fc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.643+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/3496596020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f4980 0x5600620fc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.643+0000 7f28e4b6b700 1 -- 172.21.15.150:0/3496596020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f6090 msgr2=0x5600620f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.643+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/3496596020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f6090 0x5600620f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.643+0000 7f28e4b6b700 1 -- 172.21.15.150:0/3496596020 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560061e91350 con 0x5600620f5680 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.643+0000 7f28e536c700 1 --2- 172.21.15.150:0/3496596020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f4980 0x5600620fc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.643+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/3496596020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f5680 0x5600620f5b10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f28d4005f50 tx=0x7f28d401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5ecb49e16e6090 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.643+0000 7f28caffd700 1 -- 172.21.15.150:0/3496596020 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d402b070 con 0x5600620f5680 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.643+0000 7f28caffd700 1 -- 172.21.15.150:0/3496596020 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d401e040 con 0x5600620f5680 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.643+0000 7f28caffd700 1 -- 172.21.15.150:0/3496596020 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d4024890 con 0x5600620f5680 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.644+0000 7f28f7113b80 1 -- 172.21.15.150:0/3496596020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f5680 msgr2=0x5600620f5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.644+0000 7f28f7113b80 1 --2- 172.21.15.150:0/3496596020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f5680 0x5600620f5b10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f28d4005f50 tx=0x7f28d401f000 comp rx=0 tx=0).stop 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.644+0000 7f28f7113b80 1 -- 172.21.15.150:0/3496596020 shutdown_connections 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.644+0000 7f28f7113b80 1 --2- 172.21.15.150:0/3496596020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f4980 0x5600620fc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.644+0000 7f28f7113b80 1 --2- 172.21.15.150:0/3496596020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f5680 0x5600620f5b10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.644+0000 7f28f7113b80 1 --2- 172.21.15.150:0/3496596020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f6090 0x5600620f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.644+0000 7f28f7113b80 1 -- 172.21.15.150:0/3496596020 >> 172.21.15.150:0/3496596020 conn(0x560062052a00 msgr2=0x5600620eaa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.644+0000 7f28f7113b80 1 -- 172.21.15.150:0/3496596020 shutdown_connections 2024-05-10T04:05:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.644+0000 7f28f7113b80 1 -- 172.21.15.150:0/3496596020 wait complete. 2024-05-10T04:05:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.644+0000 7f28f7113b80 1 -- start start 2024-05-10T04:05:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.644+0000 7f28f7113b80 1 Processor -- start 2024-05-10T04:05:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28f7113b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 0x5600620fe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28f7113b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f5680 0x5600620feae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28f7113b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f6090 0x560062105980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28f7113b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560061fac500 con 0x5600620f4980 2024-05-10T04:05:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28f7113b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560061fac1a0 con 0x5600620f5680 2024-05-10T04:05:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28f7113b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560061e93640 con 0x5600620f6090 2024-05-10T04:05:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28f7113b80 10 client.0 _ll_register_callbacks cb 0x560061fb5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28e4b6b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 0x5600620fe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28e4b6b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 0x5600620fe4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52966/0 (socket says 172.21.15.150:52966) 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28e4b6b700 1 -- 172.21.15.150:0/2599973447 learned_addr learned my addr 172.21.15.150:0/2599973447 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28e536c700 1 --2- 172.21.15.150:0/2599973447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f6090 0x560062105980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28dffff700 1 --2- 172.21.15.150:0/2599973447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f5680 0x5600620feae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28e4b6b700 1 -- 172.21.15.150:0/2599973447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f6090 msgr2=0x560062105980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/2599973447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f6090 0x560062105980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28e4b6b700 1 -- 172.21.15.150:0/2599973447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f5680 msgr2=0x5600620feae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/2599973447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f5680 0x5600620feae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28e4b6b700 1 -- 172.21.15.150:0/2599973447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e01098d0 con 0x5600620f4980 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/2599973447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 0x5600620fe4e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f28d4031040 tx=0x7f28d40247f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b68d41c510023e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28bffff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28bffff700 1 -- 172.21.15.150:0/2599973447 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d403b070 con 0x5600620f4980 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28f7113b80 1 -- 172.21.15.150:0/2599973447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560061f69210 con 0x5600620f4980 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.645+0000 7f28bffff700 1 -- 172.21.15.150:0/2599973447 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f28d400f770 con 0x5600620f4980 2024-05-10T04:05:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.646+0000 7f28bffff700 1 -- 172.21.15.150:0/2599973447 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28d402b040 con 0x5600620f4980 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.646+0000 7f28f7113b80 10 client.15984 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.646+0000 7f28f7113b80 1 -- 172.21.15.150:0/2599973447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560061e3db00 con 0x5600620f4980 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.646+0000 7f28bffff700 1 -- 172.21.15.150:0/2599973447 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560061f69210 con 0x5600620f4980 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.646+0000 7f28bffff700 1 -- 172.21.15.150:0/2599973447 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f28d403e020 con 0x5600620f4980 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.646+0000 7f28f7113b80 10 client.15984 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.646+0000 7f28f7113b80 10 client.15984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.646+0000 7f28f7113b80 20 client.15984 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.646+0000 7f28f7113b80 1 --2- 172.21.15.150:0/2599973447 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560062109d50 0x56006210c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.646+0000 7f28f7113b80 4 client.15984 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.646+0000 7f28f7113b80 1 -- 172.21.15.150:0/2599973447 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600620bf720 con 0x560062109d50 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.646+0000 7f28dffff700 1 --2- 172.21.15.150:0/2599973447 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560062109d50 0x56006210c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.647+0000 7f28bffff700 10 client.15984 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.647+0000 7f28dffff700 1 --2- 172.21.15.150:0/2599973447 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560062109d50 0x56006210c1a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b4a90372b5a849a server_cookie=5d338d9ad25ea601 in_seq=0 out_seq=0 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.647+0000 7f28bffff700 1 -- 172.21.15.150:0/2599973447 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5600620bf720 con 0x560062109d50 2024-05-10T04:05:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.647+0000 7f28bffff700 10 client.15984 handle_command_reply: tid=0 2024-05-10T04:05:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.647+0000 7f28f7113b80 1 client.15984 shutdown 2024-05-10T04:05:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.648+0000 7f28f7113b80 1 -- 172.21.15.150:0/2599973447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 msgr2=0x5600620fe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.648+0000 7f28f7113b80 1 --2- 172.21.15.150:0/2599973447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 0x5600620fe4e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f28d4031040 tx=0x7f28d40247f0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.648+0000 7f28f7113b80 1 -- 172.21.15.150:0/2599973447 shutdown_connections 2024-05-10T04:05:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.648+0000 7f28f7113b80 1 --2- 172.21.15.150:0/2599973447 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560062109d50 0x56006210c1a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.648+0000 7f28f7113b80 1 --2- 172.21.15.150:0/2599973447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f6090 0x560062105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.648+0000 7f28f7113b80 1 --2- 172.21.15.150:0/2599973447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 0x5600620fe4e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.648+0000 7f28f7113b80 1 --2- 172.21.15.150:0/2599973447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f5680 0x5600620feae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.648+0000 7f28bffff700 0 client.15984 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.648+0000 7f28f7113b80 1 -- 172.21.15.150:0/2599973447 >> 172.21.15.150:0/2599973447 conn(0x560062052a00 msgr2=0x5600620eaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.648+0000 7f28f7113b80 1 -- 172.21.15.150:0/2599973447 shutdown_connections 2024-05-10T04:05:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.648+0000 7f28f7113b80 1 -- 172.21.15.150:0/2599973447 wait complete. 2024-05-10T04:05:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.648+0000 7f28f7113b80 20 client.15984 trim_cache size 0 max 16384 2024-05-10T04:05:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28f7113b80 1 Processor -- start 2024-05-10T04:05:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28f7113b80 1 -- start start 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28f7113b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 0x5600620fcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28f7113b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f5680 0x5600620fd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28f7113b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f6090 0x5600621055b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28f7113b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600620ebc60 con 0x5600620f4980 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28f7113b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600620fc9d0 con 0x5600620f5680 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28f7113b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600620fc650 con 0x5600620f6090 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28e4b6b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 0x5600620fcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28e4b6b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 0x5600620fcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52982/0 (socket says 172.21.15.150:52982) 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28e4b6b700 1 -- 172.21.15.150:0/3929312883 learned_addr learned my addr 172.21.15.150:0/3929312883 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28dffff700 1 --2- 172.21.15.150:0/3929312883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f5680 0x5600620fd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28e4b6b700 1 -- 172.21.15.150:0/3929312883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f6090 msgr2=0x5600621055b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.663+0000 7f28e536c700 1 --2- 172.21.15.150:0/3929312883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f6090 0x5600621055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.664+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/3929312883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f6090 0x5600621055b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.664+0000 7f28e4b6b700 1 -- 172.21.15.150:0/3929312883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f5680 msgr2=0x5600620fd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.664+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/3929312883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f5680 0x5600620fd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.664+0000 7f28e4b6b700 1 -- 172.21.15.150:0/3929312883 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560061fd6500 con 0x5600620f4980 2024-05-10T04:05:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.664+0000 7f28dffff700 1 --2- 172.21.15.150:0/3929312883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f5680 0x5600620fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.664+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/3929312883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 0x5600620fcc20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009990 tx=0x7f28d40058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c49c0dc6b25928 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.664+0000 7f28cb7fe700 1 -- 172.21.15.150:0/3929312883 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d403b070 con 0x5600620f4980 2024-05-10T04:05:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.664+0000 7f28cb7fe700 1 -- 172.21.15.150:0/3929312883 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d4024410 con 0x5600620f4980 2024-05-10T04:05:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.664+0000 7f28cb7fe700 1 -- 172.21.15.150:0/3929312883 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d402b070 con 0x5600620f4980 2024-05-10T04:05:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 -- 172.21.15.150:0/3929312883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 msgr2=0x5600620fcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 --2- 172.21.15.150:0/3929312883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 0x5600620fcc20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009990 tx=0x7f28d40058e0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 -- 172.21.15.150:0/3929312883 shutdown_connections 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 --2- 172.21.15.150:0/3929312883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f6090 0x5600621055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 --2- 172.21.15.150:0/3929312883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f4980 0x5600620fcc20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 --2- 172.21.15.150:0/3929312883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f5680 0x5600620fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 -- 172.21.15.150:0/3929312883 >> 172.21.15.150:0/3929312883 conn(0x560062052a00 msgr2=0x5600620f9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 -- 172.21.15.150:0/3929312883 shutdown_connections 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 -- 172.21.15.150:0/3929312883 wait complete. 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 -- start start 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 Processor -- start 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f4980 0x5600621031d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f5680 0x560062103710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f6090 0x560062104640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560062104b80 con 0x5600620f6090 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560062104cf0 con 0x5600620f4980 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560062104eb0 con 0x5600620f5680 2024-05-10T04:05:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.665+0000 7f28f7113b80 10 client.0 _ll_register_callbacks cb 0x560061fb5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28e536c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f6090 0x560062104640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28e536c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f6090 0x560062104640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52992/0 (socket says 172.21.15.150:52992) 2024-05-10T04:05:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28e536c700 1 -- 172.21.15.150:0/1926391698 learned_addr learned my addr 172.21.15.150:0/1926391698 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/1926391698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f4980 0x5600621031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28dffff700 1 --2- 172.21.15.150:0/1926391698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f5680 0x560062103710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28e536c700 1 -- 172.21.15.150:0/1926391698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f5680 msgr2=0x560062103710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28e536c700 1 --2- 172.21.15.150:0/1926391698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f5680 0x560062103710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28e536c700 1 -- 172.21.15.150:0/1926391698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f4980 msgr2=0x5600621031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28e536c700 1 --2- 172.21.15.150:0/1926391698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f4980 0x5600621031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28e536c700 1 -- 172.21.15.150:0/1926391698 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560061e3db00 con 0x5600620f6090 2024-05-10T04:05:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28e536c700 1 --2- 172.21.15.150:0/1926391698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f6090 0x560062104640 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f28d8007ba0 tx=0x7f28d8007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5ee807702ebb9f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28bffff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28bffff700 1 -- 172.21.15.150:0/1926391698 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d800e040 con 0x5600620f6090 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28bffff700 1 -- 172.21.15.150:0/1926391698 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f28d8003c00 con 0x5600620f6090 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28f7113b80 1 -- 172.21.15.150:0/1926391698 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560062008e60 con 0x5600620f6090 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28bffff700 1 -- 172.21.15.150:0/1926391698 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28d8003dc0 con 0x5600620f6090 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28f7113b80 10 client.15990 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.666+0000 7f28f7113b80 1 -- 172.21.15.150:0/1926391698 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560061e95c70 con 0x5600620f6090 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.667+0000 7f28bffff700 1 -- 172.21.15.150:0/1926391698 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560062008e60 con 0x5600620f6090 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.667+0000 7f28bffff700 1 -- 172.21.15.150:0/1926391698 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f28d8004520 con 0x5600620f6090 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.667+0000 7f28f7113b80 10 client.15990 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.667+0000 7f28f7113b80 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.667+0000 7f28f7113b80 20 client.15990 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.667+0000 7f28f7113b80 1 --2- 172.21.15.150:0/1926391698 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560062160b30 0x560062162f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.667+0000 7f28f7113b80 4 client.15990 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.667+0000 7f28f7113b80 1 -- 172.21.15.150:0/1926391698 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560061e5ec90 con 0x560062160b30 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.667+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/1926391698 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560062160b30 0x560062162f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.667+0000 7f28bffff700 10 client.15990 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.667+0000 7f28e4b6b700 1 --2- 172.21.15.150:0/1926391698 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560062160b30 0x560062162f50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=903093428967f82b server_cookie=c57fedd48db1d800 in_seq=0 out_seq=0 2024-05-10T04:05:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28bffff700 1 -- 172.21.15.150:0/1926391698 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560061e5ec90 con 0x560062160b30 2024-05-10T04:05:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28bffff700 10 client.15990 handle_command_reply: tid=0 2024-05-10T04:05:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28f7113b80 1 client.15990 shutdown 2024-05-10T04:05:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28f7113b80 1 -- 172.21.15.150:0/1926391698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f6090 msgr2=0x560062104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28f7113b80 1 --2- 172.21.15.150:0/1926391698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f6090 0x560062104640 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f28d8007ba0 tx=0x7f28d8007bd0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28f7113b80 1 -- 172.21.15.150:0/1926391698 shutdown_connections 2024-05-10T04:05:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28f7113b80 1 --2- 172.21.15.150:0/1926391698 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560062160b30 0x560062162f50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28f7113b80 1 --2- 172.21.15.150:0/1926391698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600620f6090 0x560062104640 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28f7113b80 1 --2- 172.21.15.150:0/1926391698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600620f4980 0x5600621031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28bffff700 0 client.15990 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28f7113b80 1 --2- 172.21.15.150:0/1926391698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5600620f5680 0x560062103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28f7113b80 1 -- 172.21.15.150:0/1926391698 >> 172.21.15.150:0/1926391698 conn(0x560062052a00 msgr2=0x5600620eeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28f7113b80 1 -- 172.21.15.150:0/1926391698 shutdown_connections 2024-05-10T04:05:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28f7113b80 1 -- 172.21.15.150:0/1926391698 wait complete. 2024-05-10T04:05:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.675+0000 7f28f7113b80 20 client.15990 trim_cache size 0 max 16384 2024-05-10T04:05:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:05:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:05:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:05:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:05:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.676+0000 7f28e6dcf700 1 -- 172.21.15.150:0/3185933830 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f28d00a1120 msgr2=0x7f28d00a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.676+0000 7f28e6dcf700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f28d00a1120 0x7f28d00a3630 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f28d4006010 tx=0x7f28d401a040 comp rx=0 tx=0).stop 2024-05-10T04:05:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.676+0000 7f28e6dcf700 1 -- 172.21.15.150:0/3185933830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e0101bf0 msgr2=0x7f28e019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.676+0000 7f28e6dcf700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e0101bf0 0x7f28e019cd70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00d950 tx=0x7f28cc00dc60 comp rx=0 tx=0).stop 2024-05-10T04:05:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.676+0000 7f28e6dcf700 1 -- 172.21.15.150:0/3185933830 shutdown_connections 2024-05-10T04:05:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.676+0000 7f28e6dcf700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f28d00a1120 0x7f28d00a3630 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.676+0000 7f28e6dcf700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e0102590 0x7f28e019f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.676+0000 7f28e6dcf700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28e0100ad0 0x7f28e006a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.676+0000 7f28e6dcf700 1 --2- 172.21.15.150:0/3185933830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e0101bf0 0x7f28e019cd70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.676+0000 7f28e6dcf700 1 -- 172.21.15.150:0/3185933830 >> 172.21.15.150:0/3185933830 conn(0x7f28e00fc0c0 msgr2=0x7f28e0104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.676+0000 7f28e6dcf700 1 -- 172.21.15.150:0/3185933830 shutdown_connections 2024-05-10T04:05:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:44.676+0000 7f28e6dcf700 1 -- 172.21.15.150:0/3185933830 wait complete. 2024-05-10T04:05:46.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:46 smithi150 ceph-mon[42477]: pgmap v325: 129 pgs: 129 active+clean; 1.0 GiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 101 MiB/s wr, 34 op/s 2024-05-10T04:05:46.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:46 smithi181 ceph-mon[46602]: pgmap v325: 129 pgs: 129 active+clean; 1.0 GiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 101 MiB/s wr, 34 op/s 2024-05-10T04:05:46.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:46 smithi161 ceph-mon[46918]: pgmap v325: 129 pgs: 129 active+clean; 1.0 GiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 101 MiB/s wr, 34 op/s 2024-05-10T04:05:47.135 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:47.136 INFO:tasks.workunit.client.0.smithi150.stdout: Children see throughput for 1 readers = 6523996.50 kB/sec 2024-05-10T04:05:47.136 INFO:tasks.workunit.client.0.smithi150.stdout: Parent sees throughput for 1 readers = 6514226.98 kB/sec 2024-05-10T04:05:47.136 INFO:tasks.workunit.client.0.smithi150.stdout: Min throughput per process = 6523996.50 kB/sec 2024-05-10T04:05:47.136 INFO:tasks.workunit.client.0.smithi150.stdout: Max throughput per process = 6523996.50 kB/sec 2024-05-10T04:05:47.136 INFO:tasks.workunit.client.0.smithi150.stdout: Avg throughput per process = 6523996.50 kB/sec 2024-05-10T04:05:47.136 INFO:tasks.workunit.client.0.smithi150.stdout: Min xfer = 1048576.00 kB 2024-05-10T04:05:48.137 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:05:48.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.276+0000 7f7bfda51700 1 -- 172.21.15.150:0/19726499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bf8101c10 msgr2=0x7f7bf8102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.276+0000 7f7bfda51700 1 --2- 172.21.15.150:0/19726499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bf8101c10 0x7f7bf8102020 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009990 tx=0x7f7bec009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.276+0000 7f7bfda51700 1 -- 172.21.15.150:0/19726499 shutdown_connections 2024-05-10T04:05:48.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.276+0000 7f7bfda51700 1 --2- 172.21.15.150:0/19726499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bf81025b0 0x7f7bf8108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.276+0000 7f7bfda51700 1 --2- 172.21.15.150:0/19726499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bf8100af0 0x7f7bf8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.276+0000 7f7bfda51700 1 --2- 172.21.15.150:0/19726499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bf8101c10 0x7f7bf8102020 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.276+0000 7f7bfda51700 1 -- 172.21.15.150:0/19726499 >> 172.21.15.150:0/19726499 conn(0x7f7bf80fc0e0 msgr2=0x7f7bf80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.276+0000 7f7bfda51700 1 -- 172.21.15.150:0/19726499 shutdown_connections 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.276+0000 7f7bfda51700 1 -- 172.21.15.150:0/19726499 wait complete. 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bfda51700 1 Processor -- start 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bfda51700 1 -- start start 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bfda51700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bf8100af0 0x7f7bf8197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bfda51700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bf8101c10 0x7f7bf8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bfda51700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bf81025b0 0x7f7bf819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bfda51700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf819f960 con 0x7f7bf8101c10 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bfda51700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf819faa0 con 0x7f7bf81025b0 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bfda51700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf819fc30 con 0x7f7bf8100af0 2024-05-10T04:05:48.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bf67fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bf8101c10 0x7f7bf8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bf67fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bf8101c10 0x7f7bf8198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53004/0 (socket says 172.21.15.150:53004) 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bf67fc700 1 -- 172.21.15.150:0/2814297611 learned_addr learned my addr 172.21.15.150:0/2814297611 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bf77fe700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bf81025b0 0x7f7bf819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bf8100af0 0x7f7bf8197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bf67fc700 1 -- 172.21.15.150:0/2814297611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bf8100af0 msgr2=0x7f7bf8197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bf67fc700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bf8100af0 0x7f7bf8197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bf67fc700 1 -- 172.21.15.150:0/2814297611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bf81025b0 msgr2=0x7f7bf819f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bf67fc700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bf81025b0 0x7f7bf819f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bf67fc700 1 -- 172.21.15.150:0/2814297611 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec0095f0 con 0x7f7bf8101c10 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bf8100af0 0x7f7bf8197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.277+0000 7f7bf77fe700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bf81025b0 0x7f7bf819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.278+0000 7f7bf67fc700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bf8101c10 0x7f7bf8198400 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005770 tx=0x7f7bec009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.278+0000 7f7bfca4f700 1 -- 172.21.15.150:0/2814297611 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec01d070 con 0x7f7bf8101c10 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.278+0000 7f7bfca4f700 1 -- 172.21.15.150:0/2814297611 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bec00fb90 con 0x7f7bf8101c10 2024-05-10T04:05:48.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.278+0000 7f7bfda51700 1 -- 172.21.15.150:0/2814297611 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf819feb0 con 0x7f7bf8101c10 2024-05-10T04:05:48.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.278+0000 7f7bfca4f700 1 -- 172.21.15.150:0/2814297611 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec017650 con 0x7f7bf8101c10 2024-05-10T04:05:48.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.278+0000 7f7bfda51700 1 -- 172.21.15.150:0/2814297611 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf81a0220 con 0x7f7bf8101c10 2024-05-10T04:05:48.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.280+0000 7f7bfca4f700 1 -- 172.21.15.150:0/2814297611 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7bec00fd00 con 0x7f7bf8101c10 2024-05-10T04:05:48.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.281+0000 7f7bfca4f700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7be00a11c0 0x7f7be00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.281+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7be00a11c0 0x7f7be00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.281+0000 7f7bfca4f700 1 -- 172.21.15.150:0/2814297611 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7bec0ce050 con 0x7f7bf8101c10 2024-05-10T04:05:48.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.281+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7be00a11c0 0x7f7be00a36d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8101b10 tx=0x7f7be8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.281+0000 7f7c0dd95b80 1 Processor -- start 2024-05-10T04:05:48.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7c0dd95b80 1 -- start start 2024-05-10T04:05:48.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7c0dd95b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d3916a0 0x558a9d391af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7c0dd95b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d392570 0x558a9d392980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7c0dd95b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3997c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7c0dd95b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a9d2b8150 con 0x558a9d392ec0 2024-05-10T04:05:48.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7c0dd95b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a9d2b5b10 con 0x558a9d3916a0 2024-05-10T04:05:48.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7c0dd95b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a9d2b1400 con 0x558a9d392570 2024-05-10T04:05:48.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7bf77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7bf77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53012/0 (socket says 172.21.15.150:53012) 2024-05-10T04:05:48.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7bf77fe700 1 -- 172.21.15.150:0/3577253071 learned_addr learned my addr 172.21.15.150:0/3577253071 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:48.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/3577253071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d3916a0 0x558a9d391af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7bf67fc700 1 --2- 172.21.15.150:0/3577253071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d392570 0x558a9d392980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7bf77fe700 1 -- 172.21.15.150:0/3577253071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d392570 msgr2=0x558a9d392980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7bf77fe700 1 --2- 172.21.15.150:0/3577253071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d392570 0x558a9d392980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7bf77fe700 1 -- 172.21.15.150:0/3577253071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d3916a0 msgr2=0x558a9d391af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7bf77fe700 1 --2- 172.21.15.150:0/3577253071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d3916a0 0x558a9d391af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7bf77fe700 1 -- 172.21.15.150:0/3577253071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a9d1323b0 con 0x558a9d392ec0 2024-05-10T04:05:48.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/3577253071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d3916a0 0x558a9d391af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:48.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7bf67fc700 1 --2- 172.21.15.150:0/3577253071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d392570 0x558a9d392980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:48.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.282+0000 7f7bf77fe700 1 --2- 172.21.15.150:0/3577253071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3997c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000e1e0 tx=0x7f7bf000e4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a602f7ad853ba3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.283+0000 7f7bddffb700 1 -- 172.21.15.150:0/3577253071 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0018070 con 0x558a9d392ec0 2024-05-10T04:05:48.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.283+0000 7f7bddffb700 1 -- 172.21.15.150:0/3577253071 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf0007c90 con 0x558a9d392ec0 2024-05-10T04:05:48.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.283+0000 7f7bddffb700 1 -- 172.21.15.150:0/3577253071 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0014c20 con 0x558a9d392ec0 2024-05-10T04:05:48.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.283+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/3577253071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 msgr2=0x558a9d3997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.283+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/3577253071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3997c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000e1e0 tx=0x7f7bf000e4f0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/3577253071 shutdown_connections 2024-05-10T04:05:48.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/3577253071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3997c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/3577253071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d3916a0 0x558a9d391af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/3577253071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d392570 0x558a9d392980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/3577253071 >> 172.21.15.150:0/3577253071 conn(0x558a9d2f3a60 msgr2=0x558a9d38bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/3577253071 shutdown_connections 2024-05-10T04:05:48.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/3577253071 wait complete. 2024-05-10T04:05:48.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 -- start start 2024-05-10T04:05:48.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 Processor -- start 2024-05-10T04:05:48.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d3916a0 0x558a9d39b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d39bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a9d24d560 con 0x558a9d392ec0 2024-05-10T04:05:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a9d24d200 con 0x558a9d392570 2024-05-10T04:05:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.284+0000 7f7c0dd95b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a9d1346a0 con 0x558a9d3916a0 2024-05-10T04:05:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7c0dd95b80 10 client.0 _ll_register_callbacks cb 0x558a9d256b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bf77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bf77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53028/0 (socket says 172.21.15.150:53028) 2024-05-10T04:05:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bf77fe700 1 -- 172.21.15.150:0/518154510 learned_addr learned my addr 172.21.15.150:0/518154510 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:48.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bf67fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d39bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/518154510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d3916a0 0x558a9d39b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bf77fe700 1 -- 172.21.15.150:0/518154510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d3916a0 msgr2=0x558a9d39b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bf77fe700 1 --2- 172.21.15.150:0/518154510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d3916a0 0x558a9d39b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bf77fe700 1 -- 172.21.15.150:0/518154510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 msgr2=0x558a9d39bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bf77fe700 1 --2- 172.21.15.150:0/518154510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d39bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bf77fe700 1 -- 172.21.15.150:0/518154510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be8011c30 con 0x558a9d392ec0 2024-05-10T04:05:48.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bf67fc700 1 --2- 172.21.15.150:0/518154510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d39bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:48.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bf77fe700 1 --2- 172.21.15.150:0/518154510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3a2b20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00144a0 tx=0x7f7bf00144d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c522aa67b9448cd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:48.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bd6ffd700 1 -- 172.21.15.150:0/518154510 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0018040 con 0x558a9d392ec0 2024-05-10T04:05:48.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/518154510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a9d360780 con 0x558a9d392ec0 2024-05-10T04:05:48.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bd6ffd700 1 -- 172.21.15.150:0/518154510 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7bf0014b60 con 0x558a9d392ec0 2024-05-10T04:05:48.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7bd6ffd700 1 -- 172.21.15.150:0/518154510 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bf0014d20 con 0x558a9d392ec0 2024-05-10T04:05:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7c0dd95b80 10 client.16002 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.285+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/518154510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f7bf000f040 con 0x558a9d392ec0 2024-05-10T04:05:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.286+0000 7f7bd6ffd700 1 -- 172.21.15.150:0/518154510 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a9d360780 con 0x558a9d392ec0 2024-05-10T04:05:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.286+0000 7f7bd6ffd700 1 -- 172.21.15.150:0/518154510 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7bf00046a0 con 0x558a9d392ec0 2024-05-10T04:05:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.286+0000 7f7c0dd95b80 10 client.16002 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.286+0000 7f7c0dd95b80 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.286+0000 7f7c0dd95b80 20 client.16002 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:48.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.286+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/518154510 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x558a9d3a71f0 0x558a9d3a9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.286+0000 7f7c0dd95b80 4 client.16002 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:48.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.286+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/518154510 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a9d2a9ec0 con 0x558a9d3a71f0 2024-05-10T04:05:48.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.286+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/518154510 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x558a9d3a71f0 0x558a9d3a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.287+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/518154510 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x558a9d3a71f0 0x558a9d3a9640 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2dc221d191a1c29 server_cookie=c8f49954412dace6 in_seq=0 out_seq=0 2024-05-10T04:05:48.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.287+0000 7f7bd6ffd700 10 client.16002 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:48.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.287+0000 7f7bd6ffd700 1 -- 172.21.15.150:0/518154510 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558a9d2a9ec0 con 0x558a9d3a71f0 2024-05-10T04:05:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.287+0000 7f7bd6ffd700 10 client.16002 handle_command_reply: tid=0 2024-05-10T04:05:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.287+0000 7f7c0dd95b80 1 client.16002 shutdown 2024-05-10T04:05:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.288+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/518154510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 msgr2=0x558a9d3a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.288+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/518154510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3a2b20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00144a0 tx=0x7f7bf00144d0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.288+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/518154510 shutdown_connections 2024-05-10T04:05:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.288+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/518154510 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x558a9d3a71f0 0x558a9d3a9640 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.288+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/518154510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3a2b20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.288+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/518154510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d3916a0 0x558a9d39b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.288+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/518154510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d39bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.288+0000 7f7bd6ffd700 0 client.16002 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:48.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.288+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/518154510 >> 172.21.15.150:0/518154510 conn(0x558a9d2f3a60 msgr2=0x558a9d38bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.288+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/518154510 shutdown_connections 2024-05-10T04:05:48.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.288+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/518154510 wait complete. 2024-05-10T04:05:48.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.288+0000 7f7c0dd95b80 20 client.16002 trim_cache size 0 max 16384 2024-05-10T04:05:48.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7c0dd95b80 1 Processor -- start 2024-05-10T04:05:48.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7c0dd95b80 1 -- start start 2024-05-10T04:05:48.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7c0dd95b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d3916a0 0x558a9d396250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7c0dd95b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d396790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7c0dd95b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d392ec0 0x558a9d3a2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7c0dd95b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a9d3932d0 con 0x558a9d3916a0 2024-05-10T04:05:48.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7c0dd95b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a9d396cd0 con 0x558a9d392570 2024-05-10T04:05:48.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7c0dd95b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a9d39c720 con 0x558a9d392ec0 2024-05-10T04:05:48.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7bf6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d3916a0 0x558a9d396250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7bf6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d3916a0 0x558a9d396250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53030/0 (socket says 172.21.15.150:53030) 2024-05-10T04:05:48.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7bf6ffd700 1 -- 172.21.15.150:0/3822138088 learned_addr learned my addr 172.21.15.150:0/3822138088 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:48.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7bf6ffd700 1 -- 172.21.15.150:0/3822138088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d392ec0 msgr2=0x558a9d3a2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/3822138088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d392ec0 0x558a9d3a2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7bf6ffd700 1 -- 172.21.15.150:0/3822138088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 msgr2=0x558a9d396790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.305+0000 7f7bf67fc700 1 --2- 172.21.15.150:0/3822138088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d396790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.306+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/3822138088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d396790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.306+0000 7f7bf6ffd700 1 -- 172.21.15.150:0/3822138088 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a9d0deb60 con 0x558a9d3916a0 2024-05-10T04:05:48.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.306+0000 7f7bf67fc700 1 --2- 172.21.15.150:0/3822138088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d396790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:05:48.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.306+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/3822138088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d3916a0 0x558a9d396250 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009b70 tx=0x7f7be8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40310ba7a76016e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.306+0000 7f7bde7fc700 1 -- 172.21.15.150:0/3822138088 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be8024040 con 0x558a9d3916a0 2024-05-10T04:05:48.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.306+0000 7f7bde7fc700 1 -- 172.21.15.150:0/3822138088 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7be8016070 con 0x558a9d3916a0 2024-05-10T04:05:48.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.306+0000 7f7bde7fc700 1 -- 172.21.15.150:0/3822138088 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be80054e0 con 0x558a9d3916a0 2024-05-10T04:05:48.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/3822138088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d3916a0 msgr2=0x558a9d396250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/3822138088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d3916a0 0x558a9d396250 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009b70 tx=0x7f7be8014000 comp rx=0 tx=0).stop 2024-05-10T04:05:48.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/3822138088 shutdown_connections 2024-05-10T04:05:48.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/3822138088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d392ec0 0x558a9d3a2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/3822138088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d3916a0 0x558a9d396250 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/3822138088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d396790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/3822138088 >> 172.21.15.150:0/3822138088 conn(0x558a9d2f3a60 msgr2=0x558a9d3957a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/3822138088 shutdown_connections 2024-05-10T04:05:48.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/3822138088 wait complete. 2024-05-10T04:05:48.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 -- start start 2024-05-10T04:05:48.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 Processor -- start 2024-05-10T04:05:48.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d3916a0 0x558a9d3a0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d3a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a9d3a1d10 con 0x558a9d392ec0 2024-05-10T04:05:48.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a9d3a1e80 con 0x558a9d392570 2024-05-10T04:05:48.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.307+0000 7f7c0dd95b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a9d3a2040 con 0x558a9d3916a0 2024-05-10T04:05:48.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7c0dd95b80 10 client.0 _ll_register_callbacks cb 0x558a9d256b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:48.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bf77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bf77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53042/0 (socket says 172.21.15.150:53042) 2024-05-10T04:05:48.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bf77fe700 1 -- 172.21.15.150:0/2936604672 learned_addr learned my addr 172.21.15.150:0/2936604672 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:48.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bf67fc700 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d3a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d3916a0 0x558a9d3a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bf77fe700 1 -- 172.21.15.150:0/2936604672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d3916a0 msgr2=0x558a9d3a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bf77fe700 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d3916a0 0x558a9d3a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bf77fe700 1 -- 172.21.15.150:0/2936604672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 msgr2=0x558a9d3a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bf77fe700 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d3a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bf77fe700 1 -- 172.21.15.150:0/2936604672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be80126b0 con 0x558a9d392ec0 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bf67fc700 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d3a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bf77fe700 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3a17d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0007f70 tx=0x7f7bf000e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a62a1d9d991530 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bd6ffd700 1 -- 172.21.15.150:0/2936604672 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0018030 con 0x558a9d392ec0 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bd6ffd700 1 -- 172.21.15.150:0/2936604672 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7bf000e840 con 0x558a9d392ec0 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/2936604672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a9d38dbc0 con 0x558a9d392ec0 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7bd6ffd700 1 -- 172.21.15.150:0/2936604672 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bf001dad0 con 0x558a9d392ec0 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7c0dd95b80 10 client.16008 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.308+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/2936604672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558a9d277560 con 0x558a9d392ec0 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.309+0000 7f7bd6ffd700 1 -- 172.21.15.150:0/2936604672 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a9d38dbc0 con 0x558a9d392ec0 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.309+0000 7f7bd6ffd700 1 -- 172.21.15.150:0/2936604672 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7bf00268b0 con 0x558a9d392ec0 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.309+0000 7f7c0dd95b80 10 client.16008 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.309+0000 7f7c0dd95b80 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.309+0000 7f7c0dd95b80 20 client.16008 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:48.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.309+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x558a9d3fdc60 0x558a9d400080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.309+0000 7f7c0dd95b80 4 client.16008 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:05:48.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.309+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/2936604672 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558a9d20a270 con 0x558a9d3fdc60 2024-05-10T04:05:48.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.309+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x558a9d3fdc60 0x558a9d400080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.309+0000 7f7bd6ffd700 10 client.16008 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:48.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.309+0000 7f7bf6ffd700 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x558a9d3fdc60 0x558a9d400080 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47fa0f63c429adb0 server_cookie=8eed7d61d7d1c27b in_seq=0 out_seq=0 2024-05-10T04:05:48.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7bd6ffd700 1 -- 172.21.15.150:0/2936604672 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558a9d20a270 con 0x558a9d3fdc60 2024-05-10T04:05:48.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7bd6ffd700 10 client.16008 handle_command_reply: tid=0 2024-05-10T04:05:48.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7c0dd95b80 1 client.16008 shutdown 2024-05-10T04:05:48.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/2936604672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 msgr2=0x558a9d3a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3a17d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0007f70 tx=0x7f7bf000e160 comp rx=0 tx=0).stop 2024-05-10T04:05:48.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/2936604672 shutdown_connections 2024-05-10T04:05:48.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x558a9d3fdc60 0x558a9d400080 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a9d392ec0 0x558a9d3a17d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a9d3916a0 0x558a9d3a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7bd6ffd700 0 client.16008 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:48.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7c0dd95b80 1 --2- 172.21.15.150:0/2936604672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a9d392570 0x558a9d3a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/2936604672 >> 172.21.15.150:0/2936604672 conn(0x558a9d2f3a60 msgr2=0x558a9d396250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/2936604672 shutdown_connections 2024-05-10T04:05:48.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7c0dd95b80 1 -- 172.21.15.150:0/2936604672 wait complete. 2024-05-10T04:05:48.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7c0dd95b80 20 client.16008 trim_cache size 0 max 16384 2024-05-10T04:05:48.330 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:05:48.331 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "no active scrubs running", 2024-05-10T04:05:48.331 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": {} 2024-05-10T04:05:48.331 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:05:48.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7bfda51700 1 -- 172.21.15.150:0/2814297611 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7be00a11c0 msgr2=0x7f7be00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7bfda51700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7be00a11c0 0x7f7be00a36d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8101b10 tx=0x7f7be8009260 comp rx=0 tx=0).stop 2024-05-10T04:05:48.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7bfda51700 1 -- 172.21.15.150:0/2814297611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bf8101c10 msgr2=0x7f7bf8198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.310+0000 7f7bfda51700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bf8101c10 0x7f7bf8198400 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005770 tx=0x7f7bec009580 comp rx=0 tx=0).stop 2024-05-10T04:05:48.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.311+0000 7f7bfda51700 1 -- 172.21.15.150:0/2814297611 shutdown_connections 2024-05-10T04:05:48.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.311+0000 7f7bfda51700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7be00a11c0 0x7f7be00a36d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.311+0000 7f7bfda51700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bf81025b0 0x7f7bf819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.311+0000 7f7bfda51700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bf8100af0 0x7f7bf8197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.311+0000 7f7bfda51700 1 --2- 172.21.15.150:0/2814297611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bf8101c10 0x7f7bf8198400 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.311+0000 7f7bfda51700 1 -- 172.21.15.150:0/2814297611 >> 172.21.15.150:0/2814297611 conn(0x7f7bf80fc0e0 msgr2=0x7f7bf80fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.311+0000 7f7bfda51700 1 -- 172.21.15.150:0/2814297611 shutdown_connections 2024-05-10T04:05:48.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.311+0000 7f7bfda51700 1 -- 172.21.15.150:0/2814297611 wait complete. 2024-05-10T04:05:48.333 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T04:05:48.333 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:05:48.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:48 smithi150 ceph-mon[42477]: pgmap v326: 129 pgs: 129 active+clean; 1.0 GiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 133 MiB/s wr, 41 op/s 2024-05-10T04:05:48.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.465+0000 7f43fd682700 1 -- 172.21.15.150:0/2753554251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43f8100af0 msgr2=0x7f43f8100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.465+0000 7f43fd682700 1 --2- 172.21.15.150:0/2753554251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43f8100af0 0x7f43f8100f00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009990 tx=0x7f43e8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.465+0000 7f43fd682700 1 -- 172.21.15.150:0/2753554251 shutdown_connections 2024-05-10T04:05:48.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.465+0000 7f43fd682700 1 --2- 172.21.15.150:0/2753554251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43f81025b0 0x7f43f8108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.465+0000 7f43fd682700 1 --2- 172.21.15.150:0/2753554251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43f8100af0 0x7f43f8100f00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.465+0000 7f43fd682700 1 --2- 172.21.15.150:0/2753554251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f8101c10 0x7f43f8102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.465+0000 7f43fd682700 1 -- 172.21.15.150:0/2753554251 >> 172.21.15.150:0/2753554251 conn(0x7f43f80fc0e0 msgr2=0x7f43f80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.465+0000 7f43fd682700 1 -- 172.21.15.150:0/2753554251 shutdown_connections 2024-05-10T04:05:48.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.465+0000 7f43fd682700 1 -- 172.21.15.150:0/2753554251 wait complete. 2024-05-10T04:05:48.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.466+0000 7f43fd682700 1 Processor -- start 2024-05-10T04:05:48.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.466+0000 7f43fd682700 1 -- start start 2024-05-10T04:05:48.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.466+0000 7f43fd682700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43f8100af0 0x7f43f8073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.466+0000 7f43fd682700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43f8101c10 0x7f43f819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.466+0000 7f43fd682700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f81025b0 0x7f43f819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.466+0000 7f43fd682700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f819fae0 con 0x7f43f8100af0 2024-05-10T04:05:48.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.466+0000 7f43fd682700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f819fc50 con 0x7f43f8101c10 2024-05-10T04:05:48.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.466+0000 7f43fd682700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f819fe10 con 0x7f43f81025b0 2024-05-10T04:05:48.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.466+0000 7f43f6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43f8100af0 0x7f43f8073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.466+0000 7f43f6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43f8100af0 0x7f43f8073ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53060/0 (socket says 172.21.15.150:53060) 2024-05-10T04:05:48.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.466+0000 7f43f6ffd700 1 -- 172.21.15.150:0/1544300896 learned_addr learned my addr 172.21.15.150:0/1544300896 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:48.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43f67fc700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43f8101c10 0x7f43f819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43f77fe700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f81025b0 0x7f43f819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43f6ffd700 1 -- 172.21.15.150:0/1544300896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f81025b0 msgr2=0x7f43f819f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43f6ffd700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f81025b0 0x7f43f819f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43f6ffd700 1 -- 172.21.15.150:0/1544300896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43f8101c10 msgr2=0x7f43f819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43f6ffd700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43f8101c10 0x7f43f819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43f6ffd700 1 -- 172.21.15.150:0/1544300896 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e80095f0 con 0x7f43f8100af0 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43f67fc700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43f8101c10 0x7f43f819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43f77fe700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f81025b0 0x7f43f819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43f6ffd700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43f8100af0 0x7f43f8073ed0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f43e80098e0 tx=0x7f43e8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43effff700 1 -- 172.21.15.150:0/1544300896 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e801d070 con 0x7f43f8100af0 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43fd682700 1 -- 172.21.15.150:0/1544300896 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f81a0030 con 0x7f43f8100af0 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43effff700 1 -- 172.21.15.150:0/1544300896 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e800fbd0 con 0x7f43f8100af0 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43effff700 1 -- 172.21.15.150:0/1544300896 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8017650 con 0x7f43f8100af0 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.467+0000 7f43fd682700 1 -- 172.21.15.150:0/1544300896 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f81a0460 con 0x7f43f8100af0 2024-05-10T04:05:48.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.468+0000 7f43fd682700 1 -- 172.21.15.150:0/1544300896 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f8107d00 con 0x7f43f8100af0 2024-05-10T04:05:48.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.469+0000 7f43effff700 1 -- 172.21.15.150:0/1544300896 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f43e8021410 con 0x7f43f8100af0 2024-05-10T04:05:48.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.470+0000 7f43effff700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f43e40a1120 0x7f43e40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.470+0000 7f43effff700 1 -- 172.21.15.150:0/1544300896 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f43e80ce160 con 0x7f43f8100af0 2024-05-10T04:05:48.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.470+0000 7f43f67fc700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f43e40a1120 0x7f43e40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.470+0000 7f43f67fc700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f43e40a1120 0x7f43e40a3630 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f43f8073870 tx=0x7f43e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.475+0000 7f43effff700 1 -- 172.21.15.150:0/1544300896 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f43e8097670 con 0x7f43f8100af0 2024-05-10T04:05:48.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:48 smithi181 ceph-mon[46602]: pgmap v326: 129 pgs: 129 active+clean; 1.0 GiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 133 MiB/s wr, 41 op/s 2024-05-10T04:05:48.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:48 smithi161 ceph-mon[46918]: pgmap v326: 129 pgs: 129 active+clean; 1.0 GiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 133 MiB/s wr, 41 op/s 2024-05-10T04:05:48.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.721+0000 7f43fd682700 1 -- 172.21.15.150:0/1544300896 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43f8068730 con 0x7f43f8100af0 2024-05-10T04:05:48.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.721+0000 7f43effff700 1 -- 172.21.15.150:0/1544300896 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10886 (secure 0 0 0) 0x7f43e809ac90 con 0x7f43f8100af0 2024-05-10T04:05:48.723 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:05:48.724 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:04:04.157819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:05:48.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.724+0000 7f43fd682700 1 -- 172.21.15.150:0/1544300896 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f43e40a1120 msgr2=0x7f43e40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.724+0000 7f43fd682700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f43e40a1120 0x7f43e40a3630 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f43f8073870 tx=0x7f43e0009260 comp rx=0 tx=0).stop 2024-05-10T04:05:48.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.724+0000 7f43fd682700 1 -- 172.21.15.150:0/1544300896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43f8100af0 msgr2=0x7f43f8073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.724+0000 7f43fd682700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43f8100af0 0x7f43f8073ed0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f43e80098e0 tx=0x7f43e8009580 comp rx=0 tx=0).stop 2024-05-10T04:05:48.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.724+0000 7f43fd682700 1 -- 172.21.15.150:0/1544300896 shutdown_connections 2024-05-10T04:05:48.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.724+0000 7f43fd682700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f43e40a1120 0x7f43e40a3630 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.724+0000 7f43fd682700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f81025b0 0x7f43f819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.724+0000 7f43fd682700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43f8100af0 0x7f43f8073ed0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.724+0000 7f43fd682700 1 --2- 172.21.15.150:0/1544300896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43f8101c10 0x7f43f819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.724+0000 7f43fd682700 1 -- 172.21.15.150:0/1544300896 >> 172.21.15.150:0/1544300896 conn(0x7f43f80fc0e0 msgr2=0x7f43f8104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.724+0000 7f43fd682700 1 -- 172.21.15.150:0/1544300896 shutdown_connections 2024-05-10T04:05:48.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.724+0000 7f43fd682700 1 -- 172.21.15.150:0/1544300896 wait complete. 2024-05-10T04:05:48.728 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 25 2024-05-10T04:05:48.740 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T04:05:48.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.872+0000 7f45378d2700 1 -- 172.21.15.150:0/2128657705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4530100db0 msgr2=0x7f45301011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.872+0000 7f45378d2700 1 --2- 172.21.15.150:0/2128657705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4530100db0 0x7f45301011c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f452c009990 tx=0x7f452c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.872+0000 7f45378d2700 1 -- 172.21.15.150:0/2128657705 shutdown_connections 2024-05-10T04:05:48.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.872+0000 7f45378d2700 1 --2- 172.21.15.150:0/2128657705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4530101700 0x7f4530109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.872+0000 7f45378d2700 1 --2- 172.21.15.150:0/2128657705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45301020a0 0x7f45301024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.872+0000 7f45378d2700 1 --2- 172.21.15.150:0/2128657705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4530100db0 0x7f45301011c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.872+0000 7f45378d2700 1 -- 172.21.15.150:0/2128657705 >> 172.21.15.150:0/2128657705 conn(0x7f45300fb8f0 msgr2=0x7f45300fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.872+0000 7f45378d2700 1 -- 172.21.15.150:0/2128657705 shutdown_connections 2024-05-10T04:05:48.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.872+0000 7f45378d2700 1 -- 172.21.15.150:0/2128657705 wait complete. 2024-05-10T04:05:48.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.873+0000 7f45378d2700 1 Processor -- start 2024-05-10T04:05:48.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.873+0000 7f45378d2700 1 -- start start 2024-05-10T04:05:48.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.873+0000 7f45378d2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4530100db0 0x7f4530100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.873+0000 7f45378d2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4530101700 0x7f453019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f45378d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45301020a0 0x7f453019f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f45378d2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453019fa20 con 0x7f4530101700 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f45378d2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453019fb90 con 0x7f4530100db0 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f45378d2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453019fd50 con 0x7f45301020a0 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f4534e6d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4530101700 0x7f453019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f4534e6d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4530101700 0x7f453019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53078/0 (socket says 172.21.15.150:53078) 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f4534e6d700 1 -- 172.21.15.150:0/3115978340 learned_addr learned my addr 172.21.15.150:0/3115978340 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f453566e700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4530100db0 0x7f4530100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f4535e6f700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45301020a0 0x7f453019f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f4534e6d700 1 -- 172.21.15.150:0/3115978340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45301020a0 msgr2=0x7f453019f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f4534e6d700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45301020a0 0x7f453019f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f4534e6d700 1 -- 172.21.15.150:0/3115978340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4530100db0 msgr2=0x7f4530100730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f4534e6d700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4530100db0 0x7f4530100730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f4534e6d700 1 -- 172.21.15.150:0/3115978340 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c0095f0 con 0x7f4530101700 2024-05-10T04:05:48.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f4535e6f700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45301020a0 0x7f453019f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:05:48.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f4534e6d700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4530101700 0x7f453019cc90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f452c0095c0 tx=0x7f452c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f45267fc700 1 -- 172.21.15.150:0/3115978340 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c01d070 con 0x7f4530101700 2024-05-10T04:05:48.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f45378d2700 1 -- 172.21.15.150:0/3115978340 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453019ff70 con 0x7f4530101700 2024-05-10T04:05:48.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f45267fc700 1 -- 172.21.15.150:0/3115978340 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f452c022470 con 0x7f4530101700 2024-05-10T04:05:48.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f45267fc700 1 -- 172.21.15.150:0/3115978340 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c017650 con 0x7f4530101700 2024-05-10T04:05:48.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.874+0000 7f45378d2700 1 -- 172.21.15.150:0/3115978340 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45301a02e0 con 0x7f4530101700 2024-05-10T04:05:48.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.876+0000 7f45267fc700 1 -- 172.21.15.150:0/3115978340 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f452c0225e0 con 0x7f4530101700 2024-05-10T04:05:48.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.877+0000 7f45267fc700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f451c0a11c0 0x7f451c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.877+0000 7f45267fc700 1 -- 172.21.15.150:0/3115978340 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f452c0cde90 con 0x7f4530101700 2024-05-10T04:05:48.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.877+0000 7f453566e700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f451c0a11c0 0x7f451c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.878+0000 7f453566e700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f451c0a11c0 0x7f451c0a36d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4520005950 tx=0x7f452000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.878+0000 7f4547c16b80 1 Processor -- start 2024-05-10T04:05:48.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.878+0000 7f4547c16b80 1 -- start start 2024-05-10T04:05:48.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.878+0000 7f4547c16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea42af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.878+0000 7f4547c16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 0x5592dea43980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.878+0000 7f4547c16b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43ec0 0x5592dea4a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.878+0000 7f4547c16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592de969150 con 0x5592dea426a0 2024-05-10T04:05:48.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.878+0000 7f4547c16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592de966b10 con 0x5592dea43570 2024-05-10T04:05:48.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.878+0000 7f4547c16b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592de962400 con 0x5592dea43ec0 2024-05-10T04:05:48.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.878+0000 7f453566e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea42af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.878+0000 7f453566e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea42af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53080/0 (socket says 172.21.15.150:53080) 2024-05-10T04:05:48.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.878+0000 7f453566e700 1 -- 172.21.15.150:0/2480811772 learned_addr learned my addr 172.21.15.150:0/2480811772 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:48.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.879+0000 7f4534e6d700 1 --2- 172.21.15.150:0/2480811772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 0x5592dea43980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.879+0000 7f4535e6f700 1 --2- 172.21.15.150:0/2480811772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43ec0 0x5592dea4a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.879+0000 7f453566e700 1 -- 172.21.15.150:0/2480811772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43ec0 msgr2=0x5592dea4a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.879+0000 7f453566e700 1 --2- 172.21.15.150:0/2480811772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43ec0 0x5592dea4a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.879+0000 7f453566e700 1 -- 172.21.15.150:0/2480811772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 msgr2=0x5592dea43980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.879+0000 7f453566e700 1 --2- 172.21.15.150:0/2480811772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 0x5592dea43980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.879+0000 7f453566e700 1 -- 172.21.15.150:0/2480811772 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592de7e33b0 con 0x5592dea426a0 2024-05-10T04:05:48.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.879+0000 7f453566e700 1 --2- 172.21.15.150:0/2480811772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea42af0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4520010de0 tx=0x7f4520013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939d63573556e0c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.879+0000 7f451b7fe700 1 -- 172.21.15.150:0/2480811772 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452001e620 con 0x5592dea426a0 2024-05-10T04:05:48.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.879+0000 7f451b7fe700 1 -- 172.21.15.150:0/2480811772 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4520015070 con 0x5592dea426a0 2024-05-10T04:05:48.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.879+0000 7f451b7fe700 1 -- 172.21.15.150:0/2480811772 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452001c900 con 0x5592dea426a0 2024-05-10T04:05:48.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.880+0000 7f4547c16b80 1 -- 172.21.15.150:0/2480811772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 msgr2=0x5592dea42af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.880+0000 7f4547c16b80 1 --2- 172.21.15.150:0/2480811772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea42af0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4520010de0 tx=0x7f4520013000 comp rx=0 tx=0).stop 2024-05-10T04:05:48.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.880+0000 7f4547c16b80 1 -- 172.21.15.150:0/2480811772 shutdown_connections 2024-05-10T04:05:48.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.880+0000 7f4547c16b80 1 --2- 172.21.15.150:0/2480811772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43ec0 0x5592dea4a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.880+0000 7f4547c16b80 1 --2- 172.21.15.150:0/2480811772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea42af0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.880+0000 7f4547c16b80 1 --2- 172.21.15.150:0/2480811772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 0x5592dea43980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.880+0000 7f4547c16b80 1 -- 172.21.15.150:0/2480811772 >> 172.21.15.150:0/2480811772 conn(0x5592de9a4a60 msgr2=0x5592dea3cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.880+0000 7f4547c16b80 1 -- 172.21.15.150:0/2480811772 shutdown_connections 2024-05-10T04:05:48.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.880+0000 7f4547c16b80 1 -- 172.21.15.150:0/2480811772 wait complete. 2024-05-10T04:05:48.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.880+0000 7f4547c16b80 1 -- start start 2024-05-10T04:05:48.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.880+0000 7f4547c16b80 1 Processor -- start 2024-05-10T04:05:48.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f4547c16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea4c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f4547c16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 0x5592dea4cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f4547c16b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43ec0 0x5592dea53b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f4547c16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592de8fe560 con 0x5592dea426a0 2024-05-10T04:05:48.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f4547c16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592de8fe200 con 0x5592dea43570 2024-05-10T04:05:48.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f4547c16b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592de7e56a0 con 0x5592dea43ec0 2024-05-10T04:05:48.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f4547c16b80 10 client.0 _ll_register_callbacks cb 0x5592de907b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:48.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f453566e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea4c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f453566e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea4c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53092/0 (socket says 172.21.15.150:53092) 2024-05-10T04:05:48.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f453566e700 1 -- 172.21.15.150:0/4212086031 learned_addr learned my addr 172.21.15.150:0/4212086031 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:48.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f4534e6d700 1 --2- 172.21.15.150:0/4212086031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 0x5592dea4cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f4535e6f700 1 --2- 172.21.15.150:0/4212086031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43ec0 0x5592dea53b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f453566e700 1 -- 172.21.15.150:0/4212086031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43ec0 msgr2=0x5592dea53b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f453566e700 1 --2- 172.21.15.150:0/4212086031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43ec0 0x5592dea53b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f453566e700 1 -- 172.21.15.150:0/4212086031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 msgr2=0x5592dea4cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f453566e700 1 --2- 172.21.15.150:0/4212086031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 0x5592dea4cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.881+0000 7f453566e700 1 -- 172.21.15.150:0/4212086031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4520010a90 con 0x5592dea426a0 2024-05-10T04:05:48.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f453566e700 1 --2- 172.21.15.150:0/4212086031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea4c740 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f452000b9e0 tx=0x7f452001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5929c758a7178cc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4518ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:48.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4518ff9700 1 -- 172.21.15.150:0/4212086031 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452002c070 con 0x5592dea426a0 2024-05-10T04:05:48.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4518ff9700 1 -- 172.21.15.150:0/4212086031 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4520017070 con 0x5592dea426a0 2024-05-10T04:05:48.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4547c16b80 1 -- 172.21.15.150:0/4212086031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592de8bb270 con 0x5592dea426a0 2024-05-10T04:05:48.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4518ff9700 1 -- 172.21.15.150:0/4212086031 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f452001cde0 con 0x5592dea426a0 2024-05-10T04:05:48.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4547c16b80 10 client.16026 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:48.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4547c16b80 1 -- 172.21.15.150:0/4212086031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5592de78fb60 con 0x5592dea426a0 2024-05-10T04:05:48.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4518ff9700 1 -- 172.21.15.150:0/4212086031 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592de8bb270 con 0x5592dea426a0 2024-05-10T04:05:48.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4518ff9700 1 -- 172.21.15.150:0/4212086031 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4520026650 con 0x5592dea426a0 2024-05-10T04:05:48.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4547c16b80 10 client.16026 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:48.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4547c16b80 10 client.16026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T04:05:48.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4547c16b80 20 client.16026 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:48.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4547c16b80 1 --2- 172.21.15.150:0/4212086031 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5592dea58220 0x5592dea5a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4547c16b80 4 client.16026 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:48.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.882+0000 7f4547c16b80 1 -- 172.21.15.150:0/4212086031 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592dea11780 con 0x5592dea58220 2024-05-10T04:05:48.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.883+0000 7f4534e6d700 1 --2- 172.21.15.150:0/4212086031 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5592dea58220 0x5592dea5a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.883+0000 7f4518ff9700 10 client.16026 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:05:48.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.883+0000 7f4534e6d700 1 --2- 172.21.15.150:0/4212086031 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5592dea58220 0x5592dea5a670 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb62951e3cd3665 server_cookie=9de05814efeea28e in_seq=0 out_seq=0 2024-05-10T04:05:48.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4518ff9700 1 -- 172.21.15.150:0/4212086031 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592dea11780 con 0x5592dea58220 2024-05-10T04:05:48.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4518ff9700 10 client.16026 handle_command_reply: tid=0 2024-05-10T04:05:48.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4547c16b80 1 client.16026 shutdown 2024-05-10T04:05:48.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4547c16b80 1 -- 172.21.15.150:0/4212086031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 msgr2=0x5592dea4c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4547c16b80 1 --2- 172.21.15.150:0/4212086031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea4c740 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f452000b9e0 tx=0x7f452001c970 comp rx=0 tx=0).stop 2024-05-10T04:05:48.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4547c16b80 1 -- 172.21.15.150:0/4212086031 shutdown_connections 2024-05-10T04:05:48.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4547c16b80 1 --2- 172.21.15.150:0/4212086031 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5592dea58220 0x5592dea5a670 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4547c16b80 1 --2- 172.21.15.150:0/4212086031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43ec0 0x5592dea53b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4547c16b80 1 --2- 172.21.15.150:0/4212086031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea4c740 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4518ff9700 0 client.16026 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:05:48.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4547c16b80 1 --2- 172.21.15.150:0/4212086031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 0x5592dea4cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4547c16b80 1 -- 172.21.15.150:0/4212086031 >> 172.21.15.150:0/4212086031 conn(0x5592de9a4a60 msgr2=0x5592dea3cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4547c16b80 1 -- 172.21.15.150:0/4212086031 shutdown_connections 2024-05-10T04:05:48.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.884+0000 7f4547c16b80 1 -- 172.21.15.150:0/4212086031 wait complete. 2024-05-10T04:05:48.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.885+0000 7f4547c16b80 20 client.16026 trim_cache size 0 max 16384 2024-05-10T04:05:48.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.897+0000 7f4547c16b80 1 Processor -- start 2024-05-10T04:05:48.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4547c16b80 1 -- start start 2024-05-10T04:05:48.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4547c16b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea426a0 0x5592dea47310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4547c16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 0x5592dea53b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4547c16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea43ec0 0x5592dea4ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4547c16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dea3e770 con 0x5592dea43ec0 2024-05-10T04:05:48.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4547c16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dea377c0 con 0x5592dea43570 2024-05-10T04:05:48.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4547c16b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dea42f50 con 0x5592dea426a0 2024-05-10T04:05:48.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4535e6f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea43ec0 0x5592dea4ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4535e6f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea43ec0 0x5592dea4ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53096/0 (socket says 172.21.15.150:53096) 2024-05-10T04:05:48.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4535e6f700 1 -- 172.21.15.150:0/1073075786 learned_addr learned my addr 172.21.15.150:0/1073075786 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:48.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f453566e700 1 --2- 172.21.15.150:0/1073075786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea426a0 0x5592dea47310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4534e6d700 1 --2- 172.21.15.150:0/1073075786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 0x5592dea53b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4535e6f700 1 -- 172.21.15.150:0/1073075786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea426a0 msgr2=0x5592dea47310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4535e6f700 1 --2- 172.21.15.150:0/1073075786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea426a0 0x5592dea47310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4535e6f700 1 -- 172.21.15.150:0/1073075786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 msgr2=0x5592dea53b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4535e6f700 1 --2- 172.21.15.150:0/1073075786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 0x5592dea53b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4535e6f700 1 -- 172.21.15.150:0/1073075786 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592de928560 con 0x5592dea43ec0 2024-05-10T04:05:48.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f4535e6f700 1 --2- 172.21.15.150:0/1073075786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea43ec0 0x5592dea4ad00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f45280083f0 tx=0x7f4528008700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34aad69b25341320 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.898+0000 7f451bfff700 1 -- 172.21.15.150:0/1073075786 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45280143f0 con 0x5592dea43ec0 2024-05-10T04:05:48.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.899+0000 7f451bfff700 1 -- 172.21.15.150:0/1073075786 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f452800e030 con 0x5592dea43ec0 2024-05-10T04:05:48.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.899+0000 7f451bfff700 1 -- 172.21.15.150:0/1073075786 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4528013820 con 0x5592dea43ec0 2024-05-10T04:05:48.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.899+0000 7f4547c16b80 1 -- 172.21.15.150:0/1073075786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea43ec0 msgr2=0x5592dea4ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.899+0000 7f4547c16b80 1 --2- 172.21.15.150:0/1073075786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea43ec0 0x5592dea4ad00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f45280083f0 tx=0x7f4528008700 comp rx=0 tx=0).stop 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.899+0000 7f4547c16b80 1 -- 172.21.15.150:0/1073075786 shutdown_connections 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.899+0000 7f4547c16b80 1 --2- 172.21.15.150:0/1073075786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea43ec0 0x5592dea4ad00 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.899+0000 7f4547c16b80 1 --2- 172.21.15.150:0/1073075786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea426a0 0x5592dea47310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.899+0000 7f4547c16b80 1 --2- 172.21.15.150:0/1073075786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43570 0x5592dea53b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.899+0000 7f4547c16b80 1 -- 172.21.15.150:0/1073075786 >> 172.21.15.150:0/1073075786 conn(0x5592de9a4a60 msgr2=0x5592dea467a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.899+0000 7f4547c16b80 1 -- 172.21.15.150:0/1073075786 shutdown_connections 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.899+0000 7f4547c16b80 1 -- 172.21.15.150:0/1073075786 wait complete. 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.900+0000 7f4547c16b80 1 -- start start 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.900+0000 7f4547c16b80 1 Processor -- start 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.900+0000 7f4547c16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea53760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.900+0000 7f4547c16b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43570 0x5592dea51dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.900+0000 7f4547c16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43ec0 0x5592dea52310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.900+0000 7f4547c16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dea53f30 con 0x5592dea426a0 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.900+0000 7f4547c16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dea52880 con 0x5592dea43ec0 2024-05-10T04:05:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.900+0000 7f4547c16b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dea52a40 con 0x5592dea43570 2024-05-10T04:05:48.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.900+0000 7f4547c16b80 10 client.0 _ll_register_callbacks cb 0x5592de907b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:48.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.900+0000 7f453566e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea53760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.900+0000 7f453566e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea53760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53100/0 (socket says 172.21.15.150:53100) 2024-05-10T04:05:48.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.900+0000 7f453566e700 1 -- 172.21.15.150:0/3144271061 learned_addr learned my addr 172.21.15.150:0/3144271061 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:48.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f4535e6f700 1 --2- 172.21.15.150:0/3144271061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43ec0 0x5592dea52310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f4534e6d700 1 --2- 172.21.15.150:0/3144271061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43570 0x5592dea51dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f453566e700 1 -- 172.21.15.150:0/3144271061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43570 msgr2=0x5592dea51dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f453566e700 1 --2- 172.21.15.150:0/3144271061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43570 0x5592dea51dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f453566e700 1 -- 172.21.15.150:0/3144271061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43ec0 msgr2=0x5592dea52310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:48.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f453566e700 1 --2- 172.21.15.150:0/3144271061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43ec0 0x5592dea52310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f453566e700 1 -- 172.21.15.150:0/3144271061 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4528007c10 con 0x5592dea426a0 2024-05-10T04:05:48.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f453566e700 1 --2- 172.21.15.150:0/3144271061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea53760 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f45200269e0 tx=0x7f4520026b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7b59fc70b58c83 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:48.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f4518ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:48.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f4518ff9700 1 -- 172.21.15.150:0/3144271061 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452002c070 con 0x5592dea426a0 2024-05-10T04:05:48.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f4518ff9700 1 -- 172.21.15.150:0/3144271061 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4520017070 con 0x5592dea426a0 2024-05-10T04:05:48.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f4547c16b80 1 -- 172.21.15.150:0/3144271061 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4520015070 con 0x5592dea426a0 2024-05-10T04:05:48.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f4518ff9700 1 -- 172.21.15.150:0/3144271061 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4520011ae0 con 0x5592dea426a0 2024-05-10T04:05:48.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f4547c16b80 10 client.16032 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:48.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f4547c16b80 1 -- 172.21.15.150:0/3144271061 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5592de7e7cd0 con 0x5592dea426a0 2024-05-10T04:05:48.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.901+0000 7f4518ff9700 1 -- 172.21.15.150:0/3144271061 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4520015070 con 0x5592dea426a0 2024-05-10T04:05:48.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.902+0000 7f4518ff9700 1 -- 172.21.15.150:0/3144271061 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f45200117d0 con 0x5592dea426a0 2024-05-10T04:05:48.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.902+0000 7f4547c16b80 10 client.16032 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:48.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.902+0000 7f4547c16b80 10 client.16032 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T04:05:48.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.902+0000 7f4547c16b80 20 client.16032 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.902+0000 7f4547c16b80 1 --2- 172.21.15.150:0/3144271061 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5592deaaf800 0x5592deab1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.902+0000 7f4547c16b80 4 client.16032 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.902+0000 7f4547c16b80 1 -- 172.21.15.150:0/3144271061 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5592de95aec0 con 0x5592deaaf800 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.902+0000 7f4534e6d700 1 --2- 172.21.15.150:0/3144271061 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5592deaaf800 0x5592deab1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.902+0000 7f4518ff9700 10 client.16032 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.902+0000 7f4534e6d700 1 --2- 172.21.15.150:0/3144271061 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5592deaaf800 0x5592deab1c20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3810c14cc82cac server_cookie=3db340f5900f6bb3 in_seq=0 out_seq=0 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.902+0000 7f4518ff9700 1 -- 172.21.15.150:0/3144271061 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5592de95aec0 con 0x5592deaaf800 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.902+0000 7f4518ff9700 10 client.16032 handle_command_reply: tid=0 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f4547c16b80 1 client.16032 shutdown 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f4547c16b80 1 -- 172.21.15.150:0/3144271061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 msgr2=0x5592dea53760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f4547c16b80 1 --2- 172.21.15.150:0/3144271061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea53760 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f45200269e0 tx=0x7f4520026b80 comp rx=0 tx=0).stop 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f4547c16b80 1 -- 172.21.15.150:0/3144271061 shutdown_connections 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f4547c16b80 1 --2- 172.21.15.150:0/3144271061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592dea43ec0 0x5592dea52310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f4547c16b80 1 --2- 172.21.15.150:0/3144271061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592dea426a0 0x5592dea53760 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f4547c16b80 1 --2- 172.21.15.150:0/3144271061 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5592deaaf800 0x5592deab1c20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f4547c16b80 1 --2- 172.21.15.150:0/3144271061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5592dea43570 0x5592dea51dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f4518ff9700 0 client.16032 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:05:48.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f4547c16b80 1 -- 172.21.15.150:0/3144271061 >> 172.21.15.150:0/3144271061 conn(0x5592de9a4a60 msgr2=0x5592dea47310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f4547c16b80 1 -- 172.21.15.150:0/3144271061 shutdown_connections 2024-05-10T04:05:48.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f4547c16b80 1 -- 172.21.15.150:0/3144271061 wait complete. 2024-05-10T04:05:48.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f4547c16b80 20 client.16032 trim_cache size 0 max 16384 2024-05-10T04:05:48.917 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:05:48.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f45378d2700 1 -- 172.21.15.150:0/3115978340 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f451c0a11c0 msgr2=0x7f451c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f45378d2700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f451c0a11c0 0x7f451c0a36d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4520005950 tx=0x7f452000b410 comp rx=0 tx=0).stop 2024-05-10T04:05:48.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f45378d2700 1 -- 172.21.15.150:0/3115978340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4530101700 msgr2=0x7f453019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:48.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f45378d2700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4530101700 0x7f453019cc90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f452c0095c0 tx=0x7f452c00f690 comp rx=0 tx=0).stop 2024-05-10T04:05:48.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f45378d2700 1 -- 172.21.15.150:0/3115978340 shutdown_connections 2024-05-10T04:05:48.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f45378d2700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f451c0a11c0 0x7f451c0a36d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f45378d2700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45301020a0 0x7f453019f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f45378d2700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4530100db0 0x7f4530100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f45378d2700 1 --2- 172.21.15.150:0/3115978340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4530101700 0x7f453019cc90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:48.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f45378d2700 1 -- 172.21.15.150:0/3115978340 >> 172.21.15.150:0/3115978340 conn(0x7f45300fb8f0 msgr2=0x7f4530104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:48.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.903+0000 7f45378d2700 1 -- 172.21.15.150:0/3115978340 shutdown_connections 2024-05-10T04:05:48.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:48.904+0000 7f45378d2700 1 -- 172.21.15.150:0/3115978340 wait complete. 2024-05-10T04:05:48.920 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T04:05:49.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.053+0000 7f12972cf700 1 -- 172.21.15.150:0/3153283982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1290101ab0 msgr2=0x7f1290101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.053+0000 7f12972cf700 1 --2- 172.21.15.150:0/3153283982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1290101ab0 0x7f1290101f40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1280009990 tx=0x7f1280009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.053+0000 7f12972cf700 1 -- 172.21.15.150:0/3153283982 shutdown_connections 2024-05-10T04:05:49.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.053+0000 7f12972cf700 1 --2- 172.21.15.150:0/3153283982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12901025d0 0x7f1290109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.053+0000 7f12972cf700 1 --2- 172.21.15.150:0/3153283982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12901010d0 0x7f12901014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.053+0000 7f12972cf700 1 --2- 172.21.15.150:0/3153283982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1290101ab0 0x7f1290101f40 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.053+0000 7f12972cf700 1 -- 172.21.15.150:0/3153283982 >> 172.21.15.150:0/3153283982 conn(0x7f12900fb8f0 msgr2=0x7f12900fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:49.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.053+0000 7f12972cf700 1 -- 172.21.15.150:0/3153283982 shutdown_connections 2024-05-10T04:05:49.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.053+0000 7f12972cf700 1 -- 172.21.15.150:0/3153283982 wait complete. 2024-05-10T04:05:49.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f12972cf700 1 Processor -- start 2024-05-10T04:05:49.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f12972cf700 1 -- start start 2024-05-10T04:05:49.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f12972cf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12901010d0 0x7f1290104ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f12972cf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1290101ab0 0x7f1290103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f12972cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12901025d0 0x7f12901036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f12972cf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1290103d20 con 0x7f1290101ab0 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f12972cf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1290103e90 con 0x7f12901010d0 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f12972cf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1290104050 con 0x7f12901025d0 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f129486a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1290101ab0 0x7f1290103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f129486a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1290101ab0 0x7f1290103180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53112/0 (socket says 172.21.15.150:53112) 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f129486a700 1 -- 172.21.15.150:0/874977217 learned_addr learned my addr 172.21.15.150:0/874977217 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f129506b700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12901010d0 0x7f1290104ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f129586c700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12901025d0 0x7f12901036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f129486a700 1 -- 172.21.15.150:0/874977217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12901025d0 msgr2=0x7f12901036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f129486a700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12901025d0 0x7f12901036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f129486a700 1 -- 172.21.15.150:0/874977217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12901010d0 msgr2=0x7f1290104ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f129486a700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12901010d0 0x7f1290104ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f129486a700 1 -- 172.21.15.150:0/874977217 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12800095f0 con 0x7f1290101ab0 2024-05-10T04:05:49.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.054+0000 7f129486a700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1290101ab0 0x7f1290103180 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1280006010 tx=0x7f128000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:49.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.055+0000 7f12867fc700 1 -- 172.21.15.150:0/874977217 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128001d070 con 0x7f1290101ab0 2024-05-10T04:05:49.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.055+0000 7f12972cf700 1 -- 172.21.15.150:0/874977217 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12901a20d0 con 0x7f1290101ab0 2024-05-10T04:05:49.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.055+0000 7f12867fc700 1 -- 172.21.15.150:0/874977217 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1280005510 con 0x7f1290101ab0 2024-05-10T04:05:49.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.055+0000 7f12867fc700 1 -- 172.21.15.150:0/874977217 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280004b40 con 0x7f1290101ab0 2024-05-10T04:05:49.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.055+0000 7f12972cf700 1 -- 172.21.15.150:0/874977217 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12901a24f0 con 0x7f1290101ab0 2024-05-10T04:05:49.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.057+0000 7f12867fc700 1 -- 172.21.15.150:0/874977217 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1280004cc0 con 0x7f1290101ab0 2024-05-10T04:05:49.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.057+0000 7f12867fc700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f127c0a11c0 0x7f127c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.057+0000 7f12867fc700 1 -- 172.21.15.150:0/874977217 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f12800cdc30 con 0x7f1290101ab0 2024-05-10T04:05:49.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.057+0000 7f129506b700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f127c0a11c0 0x7f127c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.058+0000 7f129506b700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f127c0a11c0 0x7f127c0a36d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f12900680e0 tx=0x7f128c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:49.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.058+0000 7f12a7613b80 1 Processor -- start 2024-05-10T04:05:49.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.058+0000 7f12a7613b80 1 -- start start 2024-05-10T04:05:49.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f12a7613b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 0x560fbac95af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f12a7613b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 0x560fbac96980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f12a7613b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbac9d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f12a7613b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbabbc150 con 0x560fbac96ec0 2024-05-10T04:05:49.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f12a7613b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbabb9b10 con 0x560fbac956a0 2024-05-10T04:05:49.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f12a7613b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbabb5400 con 0x560fbac96570 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f129586c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbac9d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f129586c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbac9d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53122/0 (socket says 172.21.15.150:53122) 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f129586c700 1 -- 172.21.15.150:0/1162230827 learned_addr learned my addr 172.21.15.150:0/1162230827 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f129486a700 1 --2- 172.21.15.150:0/1162230827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 0x560fbac96980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f129506b700 1 --2- 172.21.15.150:0/1162230827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 0x560fbac95af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f129586c700 1 -- 172.21.15.150:0/1162230827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 msgr2=0x560fbac96980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f129586c700 1 --2- 172.21.15.150:0/1162230827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 0x560fbac96980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f129586c700 1 -- 172.21.15.150:0/1162230827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 msgr2=0x560fbac95af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f129586c700 1 --2- 172.21.15.150:0/1162230827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 0x560fbac95af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f129586c700 1 -- 172.21.15.150:0/1162230827 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fbaa363b0 con 0x560fbac96ec0 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f129586c700 1 --2- 172.21.15.150:0/1162230827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbac9d7c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1288009d50 tx=0x7f12880077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12bf8bc1701d373 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f127b7fe700 1 -- 172.21.15.150:0/1162230827 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288018070 con 0x560fbac96ec0 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.059+0000 7f127b7fe700 1 -- 172.21.15.150:0/1162230827 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f128800f050 con 0x560fbac96ec0 2024-05-10T04:05:49.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.060+0000 7f127b7fe700 1 -- 172.21.15.150:0/1162230827 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12880048f0 con 0x560fbac96ec0 2024-05-10T04:05:49.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.060+0000 7f12a7613b80 1 -- 172.21.15.150:0/1162230827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 msgr2=0x560fbac9d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.060+0000 7f12a7613b80 1 --2- 172.21.15.150:0/1162230827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbac9d7c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1288009d50 tx=0x7f12880077e0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.060+0000 7f12a7613b80 1 -- 172.21.15.150:0/1162230827 shutdown_connections 2024-05-10T04:05:49.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.060+0000 7f12a7613b80 1 --2- 172.21.15.150:0/1162230827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbac9d7c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.060+0000 7f12a7613b80 1 --2- 172.21.15.150:0/1162230827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 0x560fbac95af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.060+0000 7f12a7613b80 1 --2- 172.21.15.150:0/1162230827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 0x560fbac96980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.060+0000 7f12a7613b80 1 -- 172.21.15.150:0/1162230827 >> 172.21.15.150:0/1162230827 conn(0x560fbabf7a60 msgr2=0x560fbac8ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:49.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.060+0000 7f12a7613b80 1 -- 172.21.15.150:0/1162230827 shutdown_connections 2024-05-10T04:05:49.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.060+0000 7f12a7613b80 1 -- 172.21.15.150:0/1162230827 wait complete. 2024-05-10T04:05:49.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.061+0000 7f12a7613b80 1 -- start start 2024-05-10T04:05:49.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.061+0000 7f12a7613b80 1 Processor -- start 2024-05-10T04:05:49.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.061+0000 7f12a7613b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 0x560fbac9f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.061+0000 7f12a7613b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 0x560fbac9fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.061+0000 7f12a7613b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbaca6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.061+0000 7f12a7613b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbab51560 con 0x560fbac96ec0 2024-05-10T04:05:49.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.061+0000 7f12a7613b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbab51200 con 0x560fbac956a0 2024-05-10T04:05:49.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.061+0000 7f12a7613b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbaa386a0 con 0x560fbac96570 2024-05-10T04:05:49.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.061+0000 7f12a7613b80 10 client.0 _ll_register_callbacks cb 0x560fbab5ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:49.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.061+0000 7f129586c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbaca6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.061+0000 7f129586c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbaca6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53134/0 (socket says 172.21.15.150:53134) 2024-05-10T04:05:49.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.061+0000 7f129586c700 1 -- 172.21.15.150:0/4127163823 learned_addr learned my addr 172.21.15.150:0/4127163823 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.061+0000 7f129506b700 1 --2- 172.21.15.150:0/4127163823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 0x560fbac9f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f129486a700 1 --2- 172.21.15.150:0/4127163823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 0x560fbac9fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f129586c700 1 -- 172.21.15.150:0/4127163823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 msgr2=0x560fbac9fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f129586c700 1 --2- 172.21.15.150:0/4127163823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 0x560fbac9fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f129586c700 1 -- 172.21.15.150:0/4127163823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 msgr2=0x560fbac9f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f129586c700 1 --2- 172.21.15.150:0/4127163823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 0x560fbac9f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f129586c700 1 -- 172.21.15.150:0/4127163823 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1288009980 con 0x560fbac96ec0 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f129586c700 1 --2- 172.21.15.150:0/4127163823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbaca6b40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1288003ce0 tx=0x7f1288003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05cb8470d1efc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f1278ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f1278ff9700 1 -- 172.21.15.150:0/4127163823 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288018030 con 0x560fbac96ec0 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f1278ff9700 1 -- 172.21.15.150:0/4127163823 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f128801e650 con 0x560fbac96ec0 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f12a7613b80 1 -- 172.21.15.150:0/4127163823 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fbab0e270 con 0x560fbac96ec0 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f1278ff9700 1 -- 172.21.15.150:0/4127163823 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1288009ea0 con 0x560fbac96ec0 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f12a7613b80 10 client.16044 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:49.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f12a7613b80 1 -- 172.21.15.150:0/4127163823 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560fba9e2b60 con 0x560fbac96ec0 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.062+0000 7f1278ff9700 1 -- 172.21.15.150:0/4127163823 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560fbab0e270 con 0x560fbac96ec0 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.063+0000 7f1278ff9700 1 -- 172.21.15.150:0/4127163823 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1288014710 con 0x560fbac96ec0 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.063+0000 7f12a7613b80 10 client.16044 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.063+0000 7f12a7613b80 10 client.16044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.063+0000 7f12a7613b80 20 client.16044 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.063+0000 7f12a7613b80 1 --2- 172.21.15.150:0/4127163823 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560fbacab340 0x560fbacad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.063+0000 7f12a7613b80 4 client.16044 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.063+0000 7f12a7613b80 1 -- 172.21.15.150:0/4127163823 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fbac64780 con 0x560fbacab340 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.063+0000 7f129506b700 1 --2- 172.21.15.150:0/4127163823 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560fbacab340 0x560fbacad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.063+0000 7f1278ff9700 10 client.16044 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.063+0000 7f129506b700 1 --2- 172.21.15.150:0/4127163823 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560fbacab340 0x560fbacad790 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2f49a3cc5a545ec server_cookie=18494a21061eab1a in_seq=0 out_seq=0 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f1278ff9700 1 -- 172.21.15.150:0/4127163823 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560fbac64780 con 0x560fbacab340 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f1278ff9700 10 client.16044 handle_command_reply: tid=0 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f12a7613b80 1 client.16044 shutdown 2024-05-10T04:05:49.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f12a7613b80 1 -- 172.21.15.150:0/4127163823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 msgr2=0x560fbaca6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f12a7613b80 1 --2- 172.21.15.150:0/4127163823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbaca6b40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1288003ce0 tx=0x7f1288003d10 comp rx=0 tx=0).stop 2024-05-10T04:05:49.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f12a7613b80 1 -- 172.21.15.150:0/4127163823 shutdown_connections 2024-05-10T04:05:49.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f12a7613b80 1 --2- 172.21.15.150:0/4127163823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbaca6b40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f12a7613b80 1 --2- 172.21.15.150:0/4127163823 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560fbacab340 0x560fbacad790 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f12a7613b80 1 --2- 172.21.15.150:0/4127163823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 0x560fbac9f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f12a7613b80 1 --2- 172.21.15.150:0/4127163823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 0x560fbac9fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f1278ff9700 0 client.16044 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:05:49.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f12a7613b80 1 -- 172.21.15.150:0/4127163823 >> 172.21.15.150:0/4127163823 conn(0x560fbabf7a60 msgr2=0x560fbac8ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:49.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f12a7613b80 1 -- 172.21.15.150:0/4127163823 shutdown_connections 2024-05-10T04:05:49.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f12a7613b80 1 -- 172.21.15.150:0/4127163823 wait complete. 2024-05-10T04:05:49.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.064+0000 7f12a7613b80 20 client.16044 trim_cache size 0 max 16384 2024-05-10T04:05:49.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.077+0000 7f12a7613b80 1 Processor -- start 2024-05-10T04:05:49.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f12a7613b80 1 -- start start 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f12a7613b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 0x560fbac997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f12a7613b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 0x560fbac9ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f12a7613b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbac9dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f12a7613b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbac99ce0 con 0x560fbac96ec0 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f12a7613b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbac99e50 con 0x560fbac956a0 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f12a7613b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbac96c50 con 0x560fbac96570 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f129586c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbac9dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f129586c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbac9dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53142/0 (socket says 172.21.15.150:53142) 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f129586c700 1 -- 172.21.15.150:0/2447948679 learned_addr learned my addr 172.21.15.150:0/2447948679 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f129506b700 1 --2- 172.21.15.150:0/2447948679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 0x560fbac997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f129486a700 1 --2- 172.21.15.150:0/2447948679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 0x560fbac9ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f129586c700 1 -- 172.21.15.150:0/2447948679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 msgr2=0x560fbac9ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f129586c700 1 --2- 172.21.15.150:0/2447948679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 0x560fbac9ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f129586c700 1 -- 172.21.15.150:0/2447948679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 msgr2=0x560fbac997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f129586c700 1 --2- 172.21.15.150:0/2447948679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 0x560fbac997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f129586c700 1 -- 172.21.15.150:0/2447948679 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fbab7b560 con 0x560fbac96ec0 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f129486a700 1 --2- 172.21.15.150:0/2447948679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 0x560fbac9ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.078+0000 7f129586c700 1 --2- 172.21.15.150:0/2447948679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbac9dd00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1288009c70 tx=0x7f1288007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67962b3c4989e5d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.079+0000 7f127bfff700 1 -- 172.21.15.150:0/2447948679 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288018070 con 0x560fbac96ec0 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.079+0000 7f127bfff700 1 -- 172.21.15.150:0/2447948679 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1288019040 con 0x560fbac96ec0 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.079+0000 7f127bfff700 1 -- 172.21.15.150:0/2447948679 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288009ea0 con 0x560fbac96ec0 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.079+0000 7f12a7613b80 1 -- 172.21.15.150:0/2447948679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 msgr2=0x560fbac9dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.079+0000 7f12a7613b80 1 --2- 172.21.15.150:0/2447948679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbac9dd00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1288009c70 tx=0x7f1288007be0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.079+0000 7f12a7613b80 1 -- 172.21.15.150:0/2447948679 shutdown_connections 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.079+0000 7f12a7613b80 1 --2- 172.21.15.150:0/2447948679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96ec0 0x560fbac9dd00 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.079+0000 7f12a7613b80 1 --2- 172.21.15.150:0/2447948679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac956a0 0x560fbac997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.079+0000 7f12a7613b80 1 --2- 172.21.15.150:0/2447948679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac96570 0x560fbac9ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.079+0000 7f12a7613b80 1 -- 172.21.15.150:0/2447948679 >> 172.21.15.150:0/2447948679 conn(0x560fbabf7a60 msgr2=0x560fbac8ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.079+0000 7f12a7613b80 1 -- 172.21.15.150:0/2447948679 shutdown_connections 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.079+0000 7f12a7613b80 1 -- 172.21.15.150:0/2447948679 wait complete. 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.080+0000 7f12a7613b80 1 -- start start 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.080+0000 7f12a7613b80 1 Processor -- start 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.080+0000 7f12a7613b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac956a0 0x560fbaca4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.080+0000 7f12a7613b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96570 0x560fbaca4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.080+0000 7f12a7613b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac96ec0 0x560fbaca5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.080+0000 7f12a7613b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbaca60e0 con 0x560fbac96570 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.080+0000 7f12a7613b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbaca6250 con 0x560fbac96ec0 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.080+0000 7f12a7613b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbaca6410 con 0x560fbac956a0 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.080+0000 7f12a7613b80 10 client.0 _ll_register_callbacks cb 0x560fbab5ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.080+0000 7f129486a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96570 0x560fbaca4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.080+0000 7f129486a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96570 0x560fbaca4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53154/0 (socket says 172.21.15.150:53154) 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.080+0000 7f129486a700 1 -- 172.21.15.150:0/50387951 learned_addr learned my addr 172.21.15.150:0/50387951 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:49.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.080+0000 7f129586c700 1 --2- 172.21.15.150:0/50387951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac96ec0 0x560fbaca5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f129506b700 1 --2- 172.21.15.150:0/50387951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac956a0 0x560fbaca4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f129486a700 1 -- 172.21.15.150:0/50387951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac956a0 msgr2=0x560fbaca4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f129486a700 1 --2- 172.21.15.150:0/50387951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac956a0 0x560fbaca4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f129486a700 1 -- 172.21.15.150:0/50387951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac96ec0 msgr2=0x560fbaca5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f129486a700 1 --2- 172.21.15.150:0/50387951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac96ec0 0x560fbaca5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f129486a700 1 -- 172.21.15.150:0/50387951 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fba9e2b60 con 0x560fbac96570 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f129486a700 1 --2- 172.21.15.150:0/50387951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96570 0x560fbaca4c70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1280097300 tx=0x7f128000fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5184f044979f258 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f1278ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f1278ff9700 1 -- 172.21.15.150:0/50387951 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128009a4f0 con 0x560fbac96570 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f1278ff9700 1 -- 172.21.15.150:0/50387951 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f128009ec30 con 0x560fbac96570 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f12a7613b80 1 -- 172.21.15.150:0/50387951 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f128800f050 con 0x560fbac96570 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f1278ff9700 1 -- 172.21.15.150:0/50387951 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12800b9170 con 0x560fbac96570 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f12a7613b80 10 client.16050 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f12a7613b80 1 -- 172.21.15.150:0/50387951 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560fbaa3acd0 con 0x560fbac96570 2024-05-10T04:05:49.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f1278ff9700 1 -- 172.21.15.150:0/50387951 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1280064080 con 0x560fbac96570 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.081+0000 7f1278ff9700 1 -- 172.21.15.150:0/50387951 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f128009fb50 con 0x560fbac96570 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f12a7613b80 10 client.16050 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f12a7613b80 10 client.16050 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f12a7613b80 20 client.16050 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f12a7613b80 1 --2- 172.21.15.150:0/50387951 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560fbad01970 0x560fbad03d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f12a7613b80 4 client.16050 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f12a7613b80 1 -- 172.21.15.150:0/50387951 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560fbabadec0 con 0x560fbad01970 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f129506b700 1 --2- 172.21.15.150:0/50387951 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560fbad01970 0x560fbad03d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f1278ff9700 10 client.16050 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f129506b700 1 --2- 172.21.15.150:0/50387951 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560fbad01970 0x560fbad03d90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1b260ac050aa8c5 server_cookie=343aa9911dbaefc9 in_seq=0 out_seq=0 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f1278ff9700 1 -- 172.21.15.150:0/50387951 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560fbabadec0 con 0x560fbad01970 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f1278ff9700 10 client.16050 handle_command_reply: tid=0 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f12a7613b80 1 client.16050 shutdown 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f12a7613b80 1 -- 172.21.15.150:0/50387951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96570 msgr2=0x560fbaca4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.082+0000 7f12a7613b80 1 --2- 172.21.15.150:0/50387951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96570 0x560fbaca4c70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1280097300 tx=0x7f128000fe40 comp rx=0 tx=0).stop 2024-05-10T04:05:49.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12a7613b80 1 -- 172.21.15.150:0/50387951 shutdown_connections 2024-05-10T04:05:49.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12a7613b80 1 --2- 172.21.15.150:0/50387951 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x560fbad01970 0x560fbad03d90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12a7613b80 1 --2- 172.21.15.150:0/50387951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560fbac96ec0 0x560fbaca5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f1278ff9700 0 client.16050 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:05:49.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12a7613b80 1 --2- 172.21.15.150:0/50387951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fbac956a0 0x560fbaca4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12a7613b80 1 --2- 172.21.15.150:0/50387951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fbac96570 0x560fbaca4c70 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12a7613b80 1 -- 172.21.15.150:0/50387951 >> 172.21.15.150:0/50387951 conn(0x560fbabf7a60 msgr2=0x560fbac9a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:49.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12a7613b80 1 -- 172.21.15.150:0/50387951 shutdown_connections 2024-05-10T04:05:49.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12a7613b80 1 -- 172.21.15.150:0/50387951 wait complete. 2024-05-10T04:05:49.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12a7613b80 20 client.16050 trim_cache size 0 max 16384 2024-05-10T04:05:49.091 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:05:49.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12972cf700 1 -- 172.21.15.150:0/874977217 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f127c0a11c0 msgr2=0x7f127c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12972cf700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f127c0a11c0 0x7f127c0a36d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f12900680e0 tx=0x7f128c00a3b0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12972cf700 1 -- 172.21.15.150:0/874977217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1290101ab0 msgr2=0x7f1290103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12972cf700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1290101ab0 0x7f1290103180 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1280006010 tx=0x7f128000f670 comp rx=0 tx=0).stop 2024-05-10T04:05:49.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12972cf700 1 -- 172.21.15.150:0/874977217 shutdown_connections 2024-05-10T04:05:49.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12972cf700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f127c0a11c0 0x7f127c0a36d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12972cf700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12901025d0 0x7f12901036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12972cf700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12901010d0 0x7f1290104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12972cf700 1 --2- 172.21.15.150:0/874977217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1290101ab0 0x7f1290103180 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12972cf700 1 -- 172.21.15.150:0/874977217 >> 172.21.15.150:0/874977217 conn(0x7f12900fb8f0 msgr2=0x7f12900fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:49.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12972cf700 1 -- 172.21.15.150:0/874977217 shutdown_connections 2024-05-10T04:05:49.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.083+0000 7f12972cf700 1 -- 172.21.15.150:0/874977217 wait complete. 2024-05-10T04:05:49.098 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T04:05:49.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.234+0000 7f315ff65700 1 -- 172.21.15.150:0/1560570240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158107980 msgr2=0x7f3158109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.234+0000 7f315ff65700 1 --2- 172.21.15.150:0/1560570240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158107980 0x7f3158109da0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f315000b0a0 tx=0x7f315000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.234+0000 7f315ff65700 1 -- 172.21.15.150:0/1560570240 shutdown_connections 2024-05-10T04:05:49.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.234+0000 7f315ff65700 1 --2- 172.21.15.150:0/1560570240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158107980 0x7f3158109da0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.234+0000 7f315ff65700 1 --2- 172.21.15.150:0/1560570240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31581026c0 0x7f3158104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.234+0000 7f315ff65700 1 --2- 172.21.15.150:0/1560570240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3158105020 0x7f3158107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.234+0000 7f315ff65700 1 -- 172.21.15.150:0/1560570240 >> 172.21.15.150:0/1560570240 conn(0x7f31580fc0c0 msgr2=0x7f31580fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.234+0000 7f315ff65700 1 -- 172.21.15.150:0/1560570240 shutdown_connections 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.235+0000 7f315ff65700 1 -- 172.21.15.150:0/1560570240 wait complete. 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.235+0000 7f315ff65700 1 Processor -- start 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.235+0000 7f315ff65700 1 -- start start 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.235+0000 7f315ff65700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31581026c0 0x7f3158101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.235+0000 7f315ff65700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3158105020 0x7f31581005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.235+0000 7f315ff65700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158107980 0x7f3158100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.235+0000 7f315ff65700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31581011f0 con 0x7f3158107980 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.235+0000 7f315ff65700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3158101360 con 0x7f3158105020 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.235+0000 7f315ff65700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31581a20e0 con 0x7f31581026c0 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315e502700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158107980 0x7f3158100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315e502700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158107980 0x7f3158100b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53166/0 (socket says 172.21.15.150:53166) 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315e502700 1 -- 172.21.15.150:0/3584366034 learned_addr learned my addr 172.21.15.150:0/3584366034 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315d500700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3158105020 0x7f31581005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315dd01700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31581026c0 0x7f3158101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315e502700 1 -- 172.21.15.150:0/3584366034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31581026c0 msgr2=0x7f3158101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315e502700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31581026c0 0x7f3158101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315e502700 1 -- 172.21.15.150:0/3584366034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3158105020 msgr2=0x7f31581005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315e502700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3158105020 0x7f31581005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315e502700 1 -- 172.21.15.150:0/3584366034 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3150009d00 con 0x7f3158107980 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315dd01700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31581026c0 0x7f3158101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315d500700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3158105020 0x7f31581005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315e502700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158107980 0x7f3158100b10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3150007930 tx=0x7f3150009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f314effd700 1 -- 172.21.15.150:0/3584366034 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315000e040 con 0x7f3158107980 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f314effd700 1 -- 172.21.15.150:0/3584366034 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3150004c50 con 0x7f3158107980 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f314effd700 1 -- 172.21.15.150:0/3584366034 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31500052a0 con 0x7f3158107980 2024-05-10T04:05:49.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315ff65700 1 -- 172.21.15.150:0/3584366034 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31581a22e0 con 0x7f3158107980 2024-05-10T04:05:49.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.236+0000 7f315ff65700 1 -- 172.21.15.150:0/3584366034 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31581a2710 con 0x7f3158107980 2024-05-10T04:05:49.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.239+0000 7f314effd700 1 -- 172.21.15.150:0/3584366034 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3150005400 con 0x7f3158107980 2024-05-10T04:05:49.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.239+0000 7f314effd700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f31440a11c0 0x7f31440a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.240+0000 7f314effd700 1 -- 172.21.15.150:0/3584366034 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f31500ce140 con 0x7f3158107980 2024-05-10T04:05:49.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.240+0000 7f315dd01700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f31440a11c0 0x7f31440a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.240+0000 7f315dd01700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f31440a11c0 0x7f31440a36d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3154005950 tx=0x7f31540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:49.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.240+0000 7f31702a9b80 1 Processor -- start 2024-05-10T04:05:49.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f31702a9b80 1 -- start start 2024-05-10T04:05:49.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f31702a9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9b6a0 0x561622f9baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f31702a9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622f9c570 0x561622f9c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f31702a9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9cec0 0x561622fa37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f31702a9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561622ec2150 con 0x561622f9c570 2024-05-10T04:05:49.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f31702a9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561622ebfb10 con 0x561622f9b6a0 2024-05-10T04:05:49.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f31702a9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561622ebb400 con 0x561622f9cec0 2024-05-10T04:05:49.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f315d500700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622f9c570 0x561622f9c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f315d500700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622f9c570 0x561622f9c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53180/0 (socket says 172.21.15.150:53180) 2024-05-10T04:05:49.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f315d500700 1 -- 172.21.15.150:0/97484081 learned_addr learned my addr 172.21.15.150:0/97484081 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:49.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f315e502700 1 --2- 172.21.15.150:0/97484081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9cec0 0x561622fa37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f315dd01700 1 --2- 172.21.15.150:0/97484081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9b6a0 0x561622f9baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f315d500700 1 -- 172.21.15.150:0/97484081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9cec0 msgr2=0x561622fa37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f315d500700 1 --2- 172.21.15.150:0/97484081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9cec0 0x561622fa37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f315d500700 1 -- 172.21.15.150:0/97484081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9b6a0 msgr2=0x561622f9baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f315d500700 1 --2- 172.21.15.150:0/97484081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9b6a0 0x561622f9baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.241+0000 7f315d500700 1 -- 172.21.15.150:0/97484081 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561622d3c3b0 con 0x561622f9c570 2024-05-10T04:05:49.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.242+0000 7f315e502700 1 --2- 172.21.15.150:0/97484081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9cec0 0x561622fa37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:49.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.242+0000 7f315d500700 1 --2- 172.21.15.150:0/97484081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622f9c570 0x561622f9c980 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f31480115f0 tx=0x7f3148011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b900df7cecb669e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:49.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.242+0000 7f3143fff700 1 -- 172.21.15.150:0/97484081 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148004230 con 0x561622f9c570 2024-05-10T04:05:49.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.242+0000 7f3143fff700 1 -- 172.21.15.150:0/97484081 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3148004390 con 0x561622f9c570 2024-05-10T04:05:49.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.242+0000 7f3143fff700 1 -- 172.21.15.150:0/97484081 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148019870 con 0x561622f9c570 2024-05-10T04:05:49.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.243+0000 7f31702a9b80 1 -- 172.21.15.150:0/97484081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622f9c570 msgr2=0x561622f9c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.243+0000 7f31702a9b80 1 --2- 172.21.15.150:0/97484081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622f9c570 0x561622f9c980 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f31480115f0 tx=0x7f3148011900 comp rx=0 tx=0).stop 2024-05-10T04:05:49.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.243+0000 7f31702a9b80 1 -- 172.21.15.150:0/97484081 shutdown_connections 2024-05-10T04:05:49.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.243+0000 7f31702a9b80 1 --2- 172.21.15.150:0/97484081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9cec0 0x561622fa37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.243+0000 7f31702a9b80 1 --2- 172.21.15.150:0/97484081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9b6a0 0x561622f9baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.243+0000 7f31702a9b80 1 --2- 172.21.15.150:0/97484081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622f9c570 0x561622f9c980 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.243+0000 7f31702a9b80 1 -- 172.21.15.150:0/97484081 >> 172.21.15.150:0/97484081 conn(0x561622efda60 msgr2=0x561622f95fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:49.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.243+0000 7f31702a9b80 1 -- 172.21.15.150:0/97484081 shutdown_connections 2024-05-10T04:05:49.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.243+0000 7f31702a9b80 1 -- 172.21.15.150:0/97484081 wait complete. 2024-05-10T04:05:49.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.243+0000 7f31702a9b80 1 -- start start 2024-05-10T04:05:49.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.244+0000 7f31702a9b80 1 Processor -- start 2024-05-10T04:05:49.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.244+0000 7f31702a9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9b6a0 0x561622fa5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.244+0000 7f31702a9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9c570 0x561622fa5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.244+0000 7f31702a9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622f9cec0 0x561622facb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.244+0000 7f31702a9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561622e57560 con 0x561622f9cec0 2024-05-10T04:05:49.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.244+0000 7f31702a9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561622e57200 con 0x561622f9c570 2024-05-10T04:05:49.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.244+0000 7f31702a9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561622d3e6a0 con 0x561622f9b6a0 2024-05-10T04:05:49.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.244+0000 7f31702a9b80 10 client.0 _ll_register_callbacks cb 0x561622e60b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.244+0000 7f315d500700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9c570 0x561622fa5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.244+0000 7f315d500700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9c570 0x561622fa5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:50662/0 (socket says 172.21.15.150:50662) 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.244+0000 7f315d500700 1 -- 172.21.15.150:0/1454461462 learned_addr learned my addr 172.21.15.150:0/1454461462 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.244+0000 7f315dd01700 1 --2- 172.21.15.150:0/1454461462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9b6a0 0x561622fa5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.245+0000 7f315e502700 1 --2- 172.21.15.150:0/1454461462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622f9cec0 0x561622facb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.245+0000 7f315d500700 1 -- 172.21.15.150:0/1454461462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9b6a0 msgr2=0x561622fa5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.245+0000 7f315d500700 1 --2- 172.21.15.150:0/1454461462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9b6a0 0x561622fa5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.245+0000 7f315d500700 1 -- 172.21.15.150:0/1454461462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622f9cec0 msgr2=0x561622facb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.245+0000 7f315d500700 1 --2- 172.21.15.150:0/1454461462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622f9cec0 0x561622facb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.245+0000 7f315d500700 1 -- 172.21.15.150:0/1454461462 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31581a2710 con 0x561622f9c570 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.245+0000 7f315d500700 1 --2- 172.21.15.150:0/1454461462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9c570 0x561622fa5bc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f31480115c0 tx=0x7f3148005360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7db14f1410f72af server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.245+0000 7f31417fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.245+0000 7f31417fa700 1 -- 172.21.15.150:0/1454461462 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148004950 con 0x561622f9c570 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.245+0000 7f31702a9b80 1 -- 172.21.15.150:0/1454461462 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561622e14270 con 0x561622f9c570 2024-05-10T04:05:49.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.245+0000 7f31417fa700 1 -- 172.21.15.150:0/1454461462 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3148004ab0 con 0x561622f9c570 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.245+0000 7f31417fa700 1 -- 172.21.15.150:0/1454461462 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3148004c70 con 0x561622f9c570 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.245+0000 7f31702a9b80 10 client.25742 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.246+0000 7f31702a9b80 1 -- 172.21.15.150:0/1454461462 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f3148009d50 con 0x561622f9c570 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.246+0000 7f31417fa700 1 -- 172.21.15.150:0/1454461462 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561622e14270 con 0x561622f9c570 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.246+0000 7f31417fa700 1 -- 172.21.15.150:0/1454461462 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3148021040 con 0x561622f9c570 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.246+0000 7f31702a9b80 10 client.25742 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.246+0000 7f31702a9b80 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.246+0000 7f31702a9b80 20 client.25742 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.246+0000 7f31702a9b80 1 --2- 172.21.15.150:0/1454461462 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561622fb11c0 0x561622fb3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.246+0000 7f31702a9b80 4 client.25742 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.246+0000 7f31702a9b80 1 -- 172.21.15.150:0/1454461462 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561622f6a780 con 0x561622fb11c0 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.246+0000 7f315dd01700 1 --2- 172.21.15.150:0/1454461462 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561622fb11c0 0x561622fb3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.247+0000 7f31417fa700 10 client.25742 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:49.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.247+0000 7f315dd01700 1 --2- 172.21.15.150:0/1454461462 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561622fb11c0 0x561622fb3610 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58422fb2618b823a server_cookie=971d339c9fdc6749 in_seq=0 out_seq=0 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.247+0000 7f31417fa700 1 -- 172.21.15.150:0/1454461462 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561622f6a780 con 0x561622fb11c0 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.247+0000 7f31417fa700 10 client.25742 handle_command_reply: tid=0 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.247+0000 7f31702a9b80 1 client.25742 shutdown 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.247+0000 7f31702a9b80 1 -- 172.21.15.150:0/1454461462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9c570 msgr2=0x561622fa5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.247+0000 7f31702a9b80 1 --2- 172.21.15.150:0/1454461462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9c570 0x561622fa5bc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f31480115c0 tx=0x7f3148005360 comp rx=0 tx=0).stop 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.248+0000 7f31702a9b80 1 -- 172.21.15.150:0/1454461462 shutdown_connections 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.248+0000 7f31702a9b80 1 --2- 172.21.15.150:0/1454461462 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561622fb11c0 0x561622fb3610 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.248+0000 7f31702a9b80 1 --2- 172.21.15.150:0/1454461462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622f9cec0 0x561622facb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.248+0000 7f31702a9b80 1 --2- 172.21.15.150:0/1454461462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9b6a0 0x561622fa5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.248+0000 7f31702a9b80 1 --2- 172.21.15.150:0/1454461462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9c570 0x561622fa5bc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.248+0000 7f31417fa700 0 client.25742 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.248+0000 7f31702a9b80 1 -- 172.21.15.150:0/1454461462 >> 172.21.15.150:0/1454461462 conn(0x561622efda60 msgr2=0x561622f95fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.248+0000 7f31702a9b80 1 -- 172.21.15.150:0/1454461462 shutdown_connections 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.248+0000 7f31702a9b80 1 -- 172.21.15.150:0/1454461462 wait complete. 2024-05-10T04:05:49.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.248+0000 7f31702a9b80 20 client.25742 trim_cache size 0 max 16384 2024-05-10T04:05:49.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.261+0000 7f31702a9b80 1 Processor -- start 2024-05-10T04:05:49.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.261+0000 7f31702a9b80 1 -- start start 2024-05-10T04:05:49.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f31702a9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 0x561622f9bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f31702a9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9b6a0 0x561622fa4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f31702a9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9c570 0x561622faa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f31702a9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561622fa6790 con 0x561622efda60 2024-05-10T04:05:49.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f31702a9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561622fa3610 con 0x561622f9b6a0 2024-05-10T04:05:49.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f31702a9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561622fb0f10 con 0x561622f9c570 2024-05-10T04:05:49.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f315dd01700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 0x561622f9bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f315dd01700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 0x561622f9bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55756/0 (socket says 172.21.15.150:55756) 2024-05-10T04:05:49.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f315dd01700 1 -- 172.21.15.150:0/3944374424 learned_addr learned my addr 172.21.15.150:0/3944374424 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:49.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f315d500700 1 --2- 172.21.15.150:0/3944374424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9b6a0 0x561622fa4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f315e502700 1 --2- 172.21.15.150:0/3944374424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9c570 0x561622faa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f315dd01700 1 -- 172.21.15.150:0/3944374424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9c570 msgr2=0x561622faa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f315dd01700 1 --2- 172.21.15.150:0/3944374424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9c570 0x561622faa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f315dd01700 1 -- 172.21.15.150:0/3944374424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9b6a0 msgr2=0x561622fa4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f315dd01700 1 --2- 172.21.15.150:0/3944374424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9b6a0 0x561622fa4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.262+0000 7f315dd01700 1 -- 172.21.15.150:0/3944374424 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561622ce8b60 con 0x561622efda60 2024-05-10T04:05:49.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.263+0000 7f315dd01700 1 --2- 172.21.15.150:0/3944374424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 0x561622f9bd00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f315400b5d0 tx=0x7f3154015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe32e62b86ae3a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:49.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.263+0000 7f314cff9700 1 -- 172.21.15.150:0/3944374424 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154026070 con 0x561622efda60 2024-05-10T04:05:49.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.263+0000 7f314cff9700 1 -- 172.21.15.150:0/3944374424 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3154027040 con 0x561622efda60 2024-05-10T04:05:49.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.263+0000 7f314cff9700 1 -- 172.21.15.150:0/3944374424 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154017040 con 0x561622efda60 2024-05-10T04:05:49.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.264+0000 7f31702a9b80 1 -- 172.21.15.150:0/3944374424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 msgr2=0x561622f9bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.264+0000 7f31702a9b80 1 --2- 172.21.15.150:0/3944374424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 0x561622f9bd00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f315400b5d0 tx=0x7f3154015000 comp rx=0 tx=0).stop 2024-05-10T04:05:49.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.264+0000 7f31702a9b80 1 -- 172.21.15.150:0/3944374424 shutdown_connections 2024-05-10T04:05:49.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.264+0000 7f31702a9b80 1 --2- 172.21.15.150:0/3944374424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9c570 0x561622faa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.264+0000 7f31702a9b80 1 --2- 172.21.15.150:0/3944374424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 0x561622f9bd00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.264+0000 7f31702a9b80 1 --2- 172.21.15.150:0/3944374424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9b6a0 0x561622fa4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.264+0000 7f31702a9b80 1 -- 172.21.15.150:0/3944374424 >> 172.21.15.150:0/3944374424 conn(0x561622f92450 msgr2=0x561622f95ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:49.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.264+0000 7f31702a9b80 1 -- 172.21.15.150:0/3944374424 shutdown_connections 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.264+0000 7f31702a9b80 1 -- 172.21.15.150:0/3944374424 wait complete. 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.264+0000 7f31702a9b80 1 -- start start 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.264+0000 7f31702a9b80 1 Processor -- start 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f31702a9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 0x561622faae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f31702a9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9b6a0 0x561623003670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f31702a9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9c570 0x561623005c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f31702a9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616230062c0 con 0x561622efda60 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f31702a9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561623006430 con 0x561622f9c570 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f31702a9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616230065f0 con 0x561622f9b6a0 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f31702a9b80 10 client.0 _ll_register_callbacks cb 0x561622e60b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f315dd01700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 0x561622faae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f315dd01700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 0x561622faae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55770/0 (socket says 172.21.15.150:55770) 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f315dd01700 1 -- 172.21.15.150:0/2581678348 learned_addr learned my addr 172.21.15.150:0/2581678348 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:49.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f315e502700 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9c570 0x561623005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f315d500700 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9b6a0 0x561623003670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f315e502700 1 -- 172.21.15.150:0/2581678348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9b6a0 msgr2=0x561623003670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f315e502700 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9b6a0 0x561623003670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f315e502700 1 -- 172.21.15.150:0/2581678348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 msgr2=0x561622faae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:49.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f315e502700 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 0x561622faae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f315e502700 1 -- 172.21.15.150:0/2581678348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561622e81560 con 0x561622f9c570 2024-05-10T04:05:49.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.265+0000 7f315dd01700 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 0x561622faae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:05:49.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f315e502700 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9c570 0x561623005c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f31500cb8d0 tx=0x7f3150007960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61e1e882e549a9de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:49.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31417fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:05:49.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31417fa700 1 -- 172.21.15.150:0/2581678348 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315009b2b0 con 0x561622f9c570 2024-05-10T04:05:49.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31417fa700 1 -- 172.21.15.150:0/2581678348 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f31500b1030 con 0x561622f9c570 2024-05-10T04:05:49.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31702a9b80 1 -- 172.21.15.150:0/2581678348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3148004390 con 0x561622f9c570 2024-05-10T04:05:49.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31417fa700 1 -- 172.21.15.150:0/2581678348 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f315009f0e0 con 0x561622f9c570 2024-05-10T04:05:49.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31702a9b80 10 client.25748 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:49.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31702a9b80 1 -- 172.21.15.150:0/2581678348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561622d40cd0 con 0x561622f9c570 2024-05-10T04:05:49.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31417fa700 1 -- 172.21.15.150:0/2581678348 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3148004390 con 0x561622f9c570 2024-05-10T04:05:49.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31417fa700 1 -- 172.21.15.150:0/2581678348 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f31500a3210 con 0x561622f9c570 2024-05-10T04:05:49.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31702a9b80 10 client.25748 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:49.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31702a9b80 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:49.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31702a9b80 20 client.25748 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:49.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31702a9b80 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561622f9cec0 0x561623009dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:49.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31702a9b80 4 client.25748 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:05:49.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.266+0000 7f31702a9b80 1 -- 172.21.15.150:0/2581678348 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561622eb3ec0 con 0x561622f9cec0 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.267+0000 7f315dd01700 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561622f9cec0 0x561623009dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.267+0000 7f31417fa700 10 client.25748 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.267+0000 7f315dd01700 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561622f9cec0 0x561623009dc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e70a7ca326de362 server_cookie=8fff01239d4dde02 in_seq=0 out_seq=0 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.267+0000 7f31417fa700 1 -- 172.21.15.150:0/2581678348 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561622eb3ec0 con 0x561622f9cec0 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.267+0000 7f31417fa700 10 client.25748 handle_command_reply: tid=0 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.267+0000 7f31702a9b80 1 client.25748 shutdown 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.267+0000 7f31702a9b80 1 -- 172.21.15.150:0/2581678348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9c570 msgr2=0x561623005c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.267+0000 7f31702a9b80 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9c570 0x561623005c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f31500cb8d0 tx=0x7f3150007960 comp rx=0 tx=0).stop 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f31702a9b80 1 -- 172.21.15.150:0/2581678348 shutdown_connections 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f31702a9b80 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x561622f9cec0 0x561623009dc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f31702a9b80 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561622f9c570 0x561623005c80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f31702a9b80 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561622efda60 0x561622faae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f31702a9b80 1 --2- 172.21.15.150:0/2581678348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561622f9b6a0 0x561623003670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f31417fa700 0 client.25748 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:49.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f31702a9b80 1 -- 172.21.15.150:0/2581678348 >> 172.21.15.150:0/2581678348 conn(0x561622f92450 msgr2=0x561622f9a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:49.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f31702a9b80 1 -- 172.21.15.150:0/2581678348 shutdown_connections 2024-05-10T04:05:49.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f31702a9b80 1 -- 172.21.15.150:0/2581678348 wait complete. 2024-05-10T04:05:49.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f31702a9b80 20 client.25748 trim_cache size 0 max 16384 2024-05-10T04:05:49.276 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:05:49.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f315ff65700 1 -- 172.21.15.150:0/3584366034 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f31440a11c0 msgr2=0x7f31440a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f315ff65700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f31440a11c0 0x7f31440a36d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3154005950 tx=0x7f31540058e0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f315ff65700 1 -- 172.21.15.150:0/3584366034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158107980 msgr2=0x7f3158100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:49.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f315ff65700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158107980 0x7f3158100b10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3150007930 tx=0x7f3150009800 comp rx=0 tx=0).stop 2024-05-10T04:05:49.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f315ff65700 1 -- 172.21.15.150:0/3584366034 shutdown_connections 2024-05-10T04:05:49.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f315ff65700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f31440a11c0 0x7f31440a36d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f315ff65700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3158107980 0x7f3158100b10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f315ff65700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31581026c0 0x7f3158101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f315ff65700 1 --2- 172.21.15.150:0/3584366034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3158105020 0x7f31581005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:49.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f315ff65700 1 -- 172.21.15.150:0/3584366034 >> 172.21.15.150:0/3584366034 conn(0x7f31580fc0c0 msgr2=0x7f3158103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:49.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f315ff65700 1 -- 172.21.15.150:0/3584366034 shutdown_connections 2024-05-10T04:05:49.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:49.268+0000 7f315ff65700 1 -- 172.21.15.150:0/3584366034 wait complete. 2024-05-10T04:05:49.405 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:49.405 INFO:tasks.workunit.client.0.smithi150.stdout: Children see throughput for 1 re-readers = 5343331.00 kB/sec 2024-05-10T04:05:49.405 INFO:tasks.workunit.client.0.smithi150.stdout: Parent sees throughput for 1 re-readers = 5337209.64 kB/sec 2024-05-10T04:05:49.405 INFO:tasks.workunit.client.0.smithi150.stdout: Min throughput per process = 5343331.00 kB/sec 2024-05-10T04:05:49.405 INFO:tasks.workunit.client.0.smithi150.stdout: Max throughput per process = 5343331.00 kB/sec 2024-05-10T04:05:49.405 INFO:tasks.workunit.client.0.smithi150.stdout: Avg throughput per process = 5343331.00 kB/sec 2024-05-10T04:05:49.405 INFO:tasks.workunit.client.0.smithi150.stdout: Min xfer = 1048576.00 kB 2024-05-10T04:05:49.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:49 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1544300896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:05:49.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:49 smithi150 ceph-mon[42477]: pgmap v327: 129 pgs: 129 active+clean; 1.0 GiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 171 MiB/s wr, 52 op/s 2024-05-10T04:05:49.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:49 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1544300896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:05:49.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:49 smithi181 ceph-mon[46602]: pgmap v327: 129 pgs: 129 active+clean; 1.0 GiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 171 MiB/s wr, 52 op/s 2024-05-10T04:05:49.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:49 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1544300896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:05:49.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:49 smithi161 ceph-mon[46918]: pgmap v327: 129 pgs: 129 active+clean; 1.0 GiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 171 MiB/s wr, 52 op/s 2024-05-10T04:05:50.286 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T04:05:50.286 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T04:05:50.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.435+0000 7fc5a7836700 1 -- 172.21.15.150:0/3712688832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a0107350 msgr2=0x7fc5a0109770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.435+0000 7fc5a7836700 1 --2- 172.21.15.150:0/3712688832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a0107350 0x7fc5a0109770 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc590009990 tx=0x7fc590009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.435+0000 7fc5a7836700 1 -- 172.21.15.150:0/3712688832 shutdown_connections 2024-05-10T04:05:50.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.435+0000 7fc5a7836700 1 --2- 172.21.15.150:0/3712688832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5a0109cb0 0x7fc5a010c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.435+0000 7fc5a7836700 1 --2- 172.21.15.150:0/3712688832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5a0102110 0x7fc5a0102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.435+0000 7fc5a7836700 1 --2- 172.21.15.150:0/3712688832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a0107350 0x7fc5a0109770 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.435+0000 7fc5a7836700 1 -- 172.21.15.150:0/3712688832 >> 172.21.15.150:0/3712688832 conn(0x7fc5a00fb8f0 msgr2=0x7fc5a00fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:50.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.435+0000 7fc5a7836700 1 -- 172.21.15.150:0/3712688832 shutdown_connections 2024-05-10T04:05:50.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.435+0000 7fc5a7836700 1 -- 172.21.15.150:0/3712688832 wait complete. 2024-05-10T04:05:50.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a7836700 1 Processor -- start 2024-05-10T04:05:50.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a7836700 1 -- start start 2024-05-10T04:05:50.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a7836700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5a0102110 0x7fc5a019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a7836700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5a0107350 0x7fc5a019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a7836700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a0109cb0 0x7fc5a01a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a7836700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a01a1bc0 con 0x7fc5a0109cb0 2024-05-10T04:05:50.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a7836700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a01a1d30 con 0x7fc5a0107350 2024-05-10T04:05:50.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a7836700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a01a1ef0 con 0x7fc5a0102110 2024-05-10T04:05:50.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a5dd3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a0109cb0 0x7fc5a01a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a5dd3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a0109cb0 0x7fc5a01a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55784/0 (socket says 172.21.15.150:55784) 2024-05-10T04:05:50.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/98779366 learned_addr learned my addr 172.21.15.150:0/98779366 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:50.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a4dd1700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5a0107350 0x7fc5a019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a55d2700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5a0102110 0x7fc5a019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/98779366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5a0102110 msgr2=0x7fc5a019a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a5dd3700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5a0102110 0x7fc5a019a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/98779366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5a0107350 msgr2=0x7fc5a019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a5dd3700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5a0107350 0x7fc5a019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/98779366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5900095f0 con 0x7fc5a0109cb0 2024-05-10T04:05:50.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.436+0000 7fc5a5dd3700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a0109cb0 0x7fc5a01a15b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc598007ee0 tx=0x7fc59800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:50.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.437+0000 7fc5967fc700 1 -- 172.21.15.150:0/98779366 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59800ddd0 con 0x7fc5a0109cb0 2024-05-10T04:05:50.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.437+0000 7fc5a7836700 1 -- 172.21.15.150:0/98779366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a01a21d0 con 0x7fc5a0109cb0 2024-05-10T04:05:50.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.437+0000 7fc5967fc700 1 -- 172.21.15.150:0/98779366 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc59800f040 con 0x7fc5a0109cb0 2024-05-10T04:05:50.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.437+0000 7fc5967fc700 1 -- 172.21.15.150:0/98779366 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5980048e0 con 0x7fc5a0109cb0 2024-05-10T04:05:50.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.437+0000 7fc5a7836700 1 -- 172.21.15.150:0/98779366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a01a26e0 con 0x7fc5a0109cb0 2024-05-10T04:05:50.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.439+0000 7fc5967fc700 1 -- 172.21.15.150:0/98779366 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc598003a00 con 0x7fc5a0109cb0 2024-05-10T04:05:50.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.439+0000 7fc5967fc700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc58c0a11c0 0x7fc58c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.440+0000 7fc5967fc700 1 -- 172.21.15.150:0/98779366 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc5980cc490 con 0x7fc5a0109cb0 2024-05-10T04:05:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.440+0000 7fc5a55d2700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc58c0a11c0 0x7fc58c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.440+0000 7fc5a55d2700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc58c0a11c0 0x7fc58c0a36d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005950 tx=0x7fc59c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.440+0000 7fc5b7b7ab80 1 Processor -- start 2024-05-10T04:05:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.440+0000 7fc5b7b7ab80 1 -- start start 2024-05-10T04:05:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5b7b7ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be79720 0x557e2be79b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5b7b7ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be7a5f0 0x557e2be7aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5b7b7ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be81840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5b7b7ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2bda01d0 con 0x557e2be7af40 2024-05-10T04:05:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5b7b7ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2bd9db90 con 0x557e2be79720 2024-05-10T04:05:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5b7b7ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2bd99480 con 0x557e2be7a5f0 2024-05-10T04:05:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5a5dd3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be81840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5a5dd3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be81840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55796/0 (socket says 172.21.15.150:55796) 2024-05-10T04:05:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/3810999277 learned_addr learned my addr 172.21.15.150:0/3810999277 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5a4dd1700 1 --2- 172.21.15.150:0/3810999277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be7a5f0 0x557e2be7aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5a55d2700 1 --2- 172.21.15.150:0/3810999277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be79720 0x557e2be79b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/3810999277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be7a5f0 msgr2=0x557e2be7aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5a5dd3700 1 --2- 172.21.15.150:0/3810999277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be7a5f0 0x557e2be7aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/3810999277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be79720 msgr2=0x557e2be79b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5a5dd3700 1 --2- 172.21.15.150:0/3810999277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be79720 0x557e2be79b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/3810999277 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e2bc1a430 con 0x557e2be7af40 2024-05-10T04:05:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc5a5dd3700 1 --2- 172.21.15.150:0/3810999277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be81840 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc5980992d0 tx=0x7fc5980959e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b009290d12aa4684 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.441+0000 7fc58b7fe700 1 -- 172.21.15.150:0/3810999277 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598098f30 con 0x557e2be7af40 2024-05-10T04:05:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.442+0000 7fc58b7fe700 1 -- 172.21.15.150:0/3810999277 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5980cb030 con 0x557e2be7af40 2024-05-10T04:05:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.442+0000 7fc58b7fe700 1 -- 172.21.15.150:0/3810999277 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598004070 con 0x557e2be7af40 2024-05-10T04:05:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.442+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/3810999277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 msgr2=0x557e2be81840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.442+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/3810999277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be81840 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc5980992d0 tx=0x7fc5980959e0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.442+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/3810999277 shutdown_connections 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.442+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/3810999277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be81840 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.442+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/3810999277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be79720 0x557e2be79b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.442+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/3810999277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be7a5f0 0x557e2be7aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.442+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/3810999277 >> 172.21.15.150:0/3810999277 conn(0x557e2bddbae0 msgr2=0x557e2be74060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.442+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/3810999277 shutdown_connections 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.442+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/3810999277 wait complete. 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.443+0000 7fc5b7b7ab80 1 -- start start 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.443+0000 7fc5b7b7ab80 1 Processor -- start 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.443+0000 7fc5b7b7ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be79720 0x557e2be837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.443+0000 7fc5b7b7ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be7a5f0 0x557e2be83d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.443+0000 7fc5b7b7ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be8ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.443+0000 7fc5b7b7ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2bd355e0 con 0x557e2be7af40 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.443+0000 7fc5b7b7ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2bd35280 con 0x557e2be7a5f0 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.443+0000 7fc5b7b7ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2bc1c720 con 0x557e2be79720 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.443+0000 7fc5b7b7ab80 10 client.0 _ll_register_callbacks cb 0x557e2bd3ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.443+0000 7fc5a5dd3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be8ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5a5dd3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be8ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55812/0 (socket says 172.21.15.150:55812) 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/275571392 learned_addr learned my addr 172.21.15.150:0/275571392 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5a4dd1700 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be7a5f0 0x557e2be83d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5a55d2700 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be79720 0x557e2be837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/275571392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be79720 msgr2=0x557e2be837f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5a5dd3700 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be79720 0x557e2be837f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/275571392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be7a5f0 msgr2=0x557e2be83d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5a5dd3700 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be7a5f0 0x557e2be83d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/275571392 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a01a26e0 con 0x557e2be7af40 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5a55d2700 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be79720 0x557e2be837f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5a5dd3700 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be8ac90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc5980c9ff0 tx=0x7fc5980b9170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd11fd8dfbf657e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc588ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc588ff9700 1 -- 172.21.15.150:0/275571392 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59809d400 con 0x557e2be7af40 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc588ff9700 1 -- 172.21.15.150:0/275571392 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc598004070 con 0x557e2be7af40 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/275571392 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e2bcf22f0 con 0x557e2be7af40 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc588ff9700 1 -- 172.21.15.150:0/275571392 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc598099cd0 con 0x557e2be7af40 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5b7b7ab80 10 client.16080 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.444+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/275571392 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557e2bbc6be0 con 0x557e2be7af40 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.445+0000 7fc588ff9700 1 -- 172.21.15.150:0/275571392 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e2bcf22f0 con 0x557e2be7af40 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.445+0000 7fc588ff9700 1 -- 172.21.15.150:0/275571392 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc598098ac0 con 0x557e2be7af40 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.445+0000 7fc5b7b7ab80 10 client.16080 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.445+0000 7fc5b7b7ab80 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.445+0000 7fc5b7b7ab80 20 client.16080 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.445+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557e2be8efc0 0x557e2be913e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.445+0000 7fc5b7b7ab80 4 client.16080 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.445+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/275571392 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e2be48800 con 0x557e2be8efc0 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.445+0000 7fc5a55d2700 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557e2be8efc0 0x557e2be913e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.445+0000 7fc588ff9700 10 client.16080 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.445+0000 7fc5a55d2700 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557e2be8efc0 0x557e2be913e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62fae59f8008ac61 server_cookie=439d0ff16a903a9d in_seq=0 out_seq=0 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc588ff9700 1 -- 172.21.15.150:0/275571392 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557e2be48800 con 0x557e2be8efc0 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc588ff9700 10 client.16080 handle_command_reply: tid=0 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc5b7b7ab80 1 client.16080 shutdown 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/275571392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 msgr2=0x557e2be8ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be8ac90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc5980c9ff0 tx=0x7fc5980b9170 comp rx=0 tx=0).stop 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/275571392 shutdown_connections 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557e2be8efc0 0x557e2be913e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be8ac90 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be79720 0x557e2be837f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/275571392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be7a5f0 0x557e2be83d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc588ff9700 0 client.16080 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:50.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/275571392 >> 172.21.15.150:0/275571392 conn(0x557e2bddbae0 msgr2=0x557e2be74030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:50.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/275571392 shutdown_connections 2024-05-10T04:05:50.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.446+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/275571392 wait complete. 2024-05-10T04:05:50.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.447+0000 7fc5b7b7ab80 20 client.16080 trim_cache size 0 max 16384 2024-05-10T04:05:50.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5b7b7ab80 1 Processor -- start 2024-05-10T04:05:50.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5b7b7ab80 1 -- start start 2024-05-10T04:05:50.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5b7b7ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be79720 0x557e2be837d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5b7b7ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be7a5f0 0x557e2be8ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5b7b7ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be83d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5b7b7ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2be8bd90 con 0x557e2be7af40 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5b7b7ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2be7a440 con 0x557e2be7a5f0 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5b7b7ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2be6e860 con 0x557e2be79720 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5a5dd3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be83d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5a5dd3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be83d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55822/0 (socket says 172.21.15.150:55822) 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/1206520235 learned_addr learned my addr 172.21.15.150:0/1206520235 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5a55d2700 1 --2- 172.21.15.150:0/1206520235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be79720 0x557e2be837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5a4dd1700 1 --2- 172.21.15.150:0/1206520235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be7a5f0 0x557e2be8ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/1206520235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be79720 msgr2=0x557e2be837d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5a5dd3700 1 --2- 172.21.15.150:0/1206520235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be79720 0x557e2be837d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/1206520235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be7a5f0 msgr2=0x557e2be8ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5a5dd3700 1 --2- 172.21.15.150:0/1206520235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be7a5f0 0x557e2be8ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5a5dd3700 1 -- 172.21.15.150:0/1206520235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e2bd5f5e0 con 0x557e2be7af40 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5a4dd1700 1 --2- 172.21.15.150:0/1206520235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be7a5f0 0x557e2be8ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.464+0000 7fc5a5dd3700 1 --2- 172.21.15.150:0/1206520235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be83d10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc59800d7b0 tx=0x7fc598099690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5633a9f95bac47 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.465+0000 7fc58bfff700 1 -- 172.21.15.150:0/1206520235 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5980993a0 con 0x557e2be7af40 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.465+0000 7fc58bfff700 1 -- 172.21.15.150:0/1206520235 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc598098f30 con 0x557e2be7af40 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.465+0000 7fc58bfff700 1 -- 172.21.15.150:0/1206520235 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598097970 con 0x557e2be7af40 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.465+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/1206520235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 msgr2=0x557e2be83d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.465+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/1206520235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be83d10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc59800d7b0 tx=0x7fc598099690 comp rx=0 tx=0).stop 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.465+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/1206520235 shutdown_connections 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.465+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/1206520235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7af40 0x557e2be83d10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.465+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/1206520235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be79720 0x557e2be837d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.465+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/1206520235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be7a5f0 0x557e2be8ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.465+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/1206520235 >> 172.21.15.150:0/1206520235 conn(0x557e2bddbae0 msgr2=0x557e2be7e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:50.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.465+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/1206520235 shutdown_connections 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.465+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/1206520235 wait complete. 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5b7b7ab80 1 -- start start 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5b7b7ab80 1 Processor -- start 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5b7b7ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be79720 0x557e2be88370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5b7b7ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7a5f0 0x557e2be888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5b7b7ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be7af40 0x557e2be897e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5b7b7ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2be89d20 con 0x557e2be7a5f0 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5b7b7ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2be89e90 con 0x557e2be79720 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5b7b7ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2be8a050 con 0x557e2be7af40 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5b7b7ab80 10 client.0 _ll_register_callbacks cb 0x557e2bd3ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5a4dd1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7a5f0 0x557e2be888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5a4dd1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7a5f0 0x557e2be888b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55838/0 (socket says 172.21.15.150:55838) 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5a4dd1700 1 -- 172.21.15.150:0/3463386371 learned_addr learned my addr 172.21.15.150:0/3463386371 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5a55d2700 1 --2- 172.21.15.150:0/3463386371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be79720 0x557e2be88370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.466+0000 7fc5a5dd3700 1 --2- 172.21.15.150:0/3463386371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be7af40 0x557e2be897e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc5a4dd1700 1 -- 172.21.15.150:0/3463386371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be7af40 msgr2=0x557e2be897e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc5a4dd1700 1 --2- 172.21.15.150:0/3463386371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be7af40 0x557e2be897e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc5a4dd1700 1 -- 172.21.15.150:0/3463386371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be79720 msgr2=0x557e2be88370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc5a4dd1700 1 --2- 172.21.15.150:0/3463386371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be79720 0x557e2be88370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc5a4dd1700 1 -- 172.21.15.150:0/3463386371 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e2bbc6be0 con 0x557e2be7a5f0 2024-05-10T04:05:50.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc5a4dd1700 1 --2- 172.21.15.150:0/3463386371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7a5f0 0x557e2be888b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc590005f50 tx=0x7fc59000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bd194f209f2506 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:50.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc588ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:50.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc588ff9700 1 -- 172.21.15.150:0/3463386371 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59000fcc0 con 0x557e2be7a5f0 2024-05-10T04:05:50.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc588ff9700 1 -- 172.21.15.150:0/3463386371 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc59000fe20 con 0x557e2be7a5f0 2024-05-10T04:05:50.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/3463386371 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5980cb030 con 0x557e2be7a5f0 2024-05-10T04:05:50.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc588ff9700 1 -- 172.21.15.150:0/3463386371 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc590003890 con 0x557e2be7a5f0 2024-05-10T04:05:50.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc5b7b7ab80 10 client.16086 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:50.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/3463386371 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557e2bc1ed50 con 0x557e2be7a5f0 2024-05-10T04:05:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc588ff9700 1 -- 172.21.15.150:0/3463386371 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5980cb030 con 0x557e2be7a5f0 2024-05-10T04:05:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.467+0000 7fc588ff9700 1 -- 172.21.15.150:0/3463386371 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc590025400 con 0x557e2be7a5f0 2024-05-10T04:05:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.468+0000 7fc5b7b7ab80 10 client.16086 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.468+0000 7fc5b7b7ab80 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.468+0000 7fc5b7b7ab80 20 client.16086 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.468+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/3463386371 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557e2bee5c90 0x557e2bee80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.468+0000 7fc5b7b7ab80 4 client.16086 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T04:05:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.468+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/3463386371 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557e2bd91f40 con 0x557e2bee5c90 2024-05-10T04:05:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.468+0000 7fc5a55d2700 1 --2- 172.21.15.150:0/3463386371 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557e2bee5c90 0x557e2bee80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.468+0000 7fc588ff9700 10 client.16086 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:50.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.468+0000 7fc5a55d2700 1 --2- 172.21.15.150:0/3463386371 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557e2bee5c90 0x557e2bee80b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fd576866f5d85df server_cookie=10a59acb8e15340d in_seq=0 out_seq=0 2024-05-10T04:05:50.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.469+0000 7fc588ff9700 1 -- 172.21.15.150:0/3463386371 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557e2bd91f40 con 0x557e2bee5c90 2024-05-10T04:05:50.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.469+0000 7fc588ff9700 10 client.16086 handle_command_reply: tid=0 2024-05-10T04:05:50.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.469+0000 7fc5b7b7ab80 1 client.16086 shutdown 2024-05-10T04:05:50.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.469+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/3463386371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7a5f0 msgr2=0x557e2be888b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.469+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/3463386371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7a5f0 0x557e2be888b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc590005f50 tx=0x7fc59000f690 comp rx=0 tx=0).stop 2024-05-10T04:05:50.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.469+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/3463386371 shutdown_connections 2024-05-10T04:05:50.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.469+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/3463386371 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557e2bee5c90 0x557e2bee80b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.469+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/3463386371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2be7af40 0x557e2be897e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.469+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/3463386371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557e2be79720 0x557e2be88370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.469+0000 7fc5b7b7ab80 1 --2- 172.21.15.150:0/3463386371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e2be7a5f0 0x557e2be888b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.469+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/3463386371 >> 172.21.15.150:0/3463386371 conn(0x557e2bddbae0 msgr2=0x557e2be833e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:50.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.469+0000 7fc588ff9700 0 client.16086 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:50.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/3463386371 shutdown_connections 2024-05-10T04:05:50.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5b7b7ab80 1 -- 172.21.15.150:0/3463386371 wait complete. 2024-05-10T04:05:50.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5b7b7ab80 20 client.16086 trim_cache size 0 max 16384 2024-05-10T04:05:50.478 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:05:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: "return_code": 0, 2024-05-10T04:05:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_tag": "96b85304-204e-4c3a-a5ea-e3a815e52b2b", 2024-05-10T04:05:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": "asynchronous" 2024-05-10T04:05:50.478 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:05:50.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5a7836700 1 -- 172.21.15.150:0/98779366 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc58c0a11c0 msgr2=0x7fc58c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5a7836700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc58c0a11c0 0x7fc58c0a36d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005950 tx=0x7fc59c0058e0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5a7836700 1 -- 172.21.15.150:0/98779366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a0109cb0 msgr2=0x7fc5a01a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5a7836700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a0109cb0 0x7fc5a01a15b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc598007ee0 tx=0x7fc59800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5a7836700 1 -- 172.21.15.150:0/98779366 shutdown_connections 2024-05-10T04:05:50.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5a7836700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc58c0a11c0 0x7fc58c0a36d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5a7836700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a0109cb0 0x7fc5a01a15b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5a7836700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5a0102110 0x7fc5a019a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5a7836700 1 --2- 172.21.15.150:0/98779366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5a0107350 0x7fc5a019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5a7836700 1 -- 172.21.15.150:0/98779366 >> 172.21.15.150:0/98779366 conn(0x7fc5a00fb8f0 msgr2=0x7fc5a0108190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:50.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5a7836700 1 -- 172.21.15.150:0/98779366 shutdown_connections 2024-05-10T04:05:50.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.470+0000 7fc5a7836700 1 -- 172.21.15.150:0/98779366 wait complete. 2024-05-10T04:05:50.485 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:05:50.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.619+0000 7efc56685700 1 -- 172.21.15.150:0/1801624724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc501031e0 msgr2=0x7efc501035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.619+0000 7efc56685700 1 --2- 172.21.15.150:0/1801624724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc501031e0 0x7efc501035f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efc44009990 tx=0x7efc44009ca0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.619+0000 7efc56685700 1 -- 172.21.15.150:0/1801624724 shutdown_connections 2024-05-10T04:05:50.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.619+0000 7efc56685700 1 --2- 172.21.15.150:0/1801624724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc50100570 0x7efc5010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.619+0000 7efc56685700 1 --2- 172.21.15.150:0/1801624724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc501031e0 0x7efc501035f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.619+0000 7efc56685700 1 --2- 172.21.15.150:0/1801624724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc500ffc20 0x7efc50100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.619+0000 7efc56685700 1 -- 172.21.15.150:0/1801624724 >> 172.21.15.150:0/1801624724 conn(0x7efc500fb8f0 msgr2=0x7efc500fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:50.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.619+0000 7efc56685700 1 -- 172.21.15.150:0/1801624724 shutdown_connections 2024-05-10T04:05:50.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.619+0000 7efc56685700 1 -- 172.21.15.150:0/1801624724 wait complete. 2024-05-10T04:05:50.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.620+0000 7efc56685700 1 Processor -- start 2024-05-10T04:05:50.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.620+0000 7efc56685700 1 -- start start 2024-05-10T04:05:50.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.620+0000 7efc56685700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc500ffc20 0x7efc501027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.620+0000 7efc56685700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc50100570 0x7efc50100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.620+0000 7efc56685700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc501031e0 0x7efc50101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.620+0000 7efc56685700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc50101a70 con 0x7efc501031e0 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.620+0000 7efc56685700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc50101be0 con 0x7efc50100570 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.620+0000 7efc56685700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc501a1f70 con 0x7efc500ffc20 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.620+0000 7efc54c22700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc501031e0 0x7efc50101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.620+0000 7efc54c22700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc501031e0 0x7efc50101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55848/0 (socket says 172.21.15.150:55848) 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.620+0000 7efc54c22700 1 -- 172.21.15.150:0/1162534201 learned_addr learned my addr 172.21.15.150:0/1162534201 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.620+0000 7efc4ffff700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc500ffc20 0x7efc501027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.620+0000 7efc4f7fe700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc50100570 0x7efc50100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.621+0000 7efc54c22700 1 -- 172.21.15.150:0/1162534201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc500ffc20 msgr2=0x7efc501027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.621+0000 7efc54c22700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc500ffc20 0x7efc501027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.621+0000 7efc54c22700 1 -- 172.21.15.150:0/1162534201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc50100570 msgr2=0x7efc50100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.621+0000 7efc54c22700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc50100570 0x7efc50100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.621+0000 7efc54c22700 1 -- 172.21.15.150:0/1162534201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc440095f0 con 0x7efc501031e0 2024-05-10T04:05:50.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.621+0000 7efc54c22700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc501031e0 0x7efc50101390 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efc48007ee0 tx=0x7efc4800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:50.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.621+0000 7efc4d7fa700 1 -- 172.21.15.150:0/1162534201 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc48017070 con 0x7efc501031e0 2024-05-10T04:05:50.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.621+0000 7efc56685700 1 -- 172.21.15.150:0/1162534201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc501a21d0 con 0x7efc501031e0 2024-05-10T04:05:50.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.621+0000 7efc4d7fa700 1 -- 172.21.15.150:0/1162534201 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc4800f040 con 0x7efc501031e0 2024-05-10T04:05:50.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.621+0000 7efc4d7fa700 1 -- 172.21.15.150:0/1162534201 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc48005740 con 0x7efc501031e0 2024-05-10T04:05:50.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.621+0000 7efc56685700 1 -- 172.21.15.150:0/1162534201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc501a2660 con 0x7efc501031e0 2024-05-10T04:05:50.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.623+0000 7efc4d7fa700 1 -- 172.21.15.150:0/1162534201 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7efc480058a0 con 0x7efc501031e0 2024-05-10T04:05:50.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.624+0000 7efc4d7fa700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7efc380a11c0 0x7efc380a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.624+0000 7efc4d7fa700 1 -- 172.21.15.150:0/1162534201 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efc480cc4b0 con 0x7efc501031e0 2024-05-10T04:05:50.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.624+0000 7efc4ffff700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7efc380a11c0 0x7efc380a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.624+0000 7efc669c9b80 1 Processor -- start 2024-05-10T04:05:50.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc669c9b80 1 -- start start 2024-05-10T04:05:50.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc4ffff700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7efc380a11c0 0x7efc380a36d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efc4400b5c0 tx=0x7efc44011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:50.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc669c9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4a6a0 0x5587dcd4aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc669c9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4b570 0x5587dcd4b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc669c9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 0x5587dcd527c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc669c9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587dcc71150 con 0x5587dcd4a6a0 2024-05-10T04:05:50.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc669c9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587dcc6eb10 con 0x5587dcd4b570 2024-05-10T04:05:50.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc669c9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587dcc6a400 con 0x5587dcd4bec0 2024-05-10T04:05:50.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc4ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4a6a0 0x5587dcd4aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc4ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4a6a0 0x5587dcd4aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55856/0 (socket says 172.21.15.150:55856) 2024-05-10T04:05:50.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc4ffff700 1 -- 172.21.15.150:0/2661437416 learned_addr learned my addr 172.21.15.150:0/2661437416 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:50.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc4f7fe700 1 --2- 172.21.15.150:0/2661437416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4b570 0x5587dcd4b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc54c22700 1 --2- 172.21.15.150:0/2661437416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 0x5587dcd527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc4ffff700 1 -- 172.21.15.150:0/2661437416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 msgr2=0x5587dcd527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc4ffff700 1 --2- 172.21.15.150:0/2661437416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 0x5587dcd527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc4ffff700 1 -- 172.21.15.150:0/2661437416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4b570 msgr2=0x5587dcd4b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc4ffff700 1 --2- 172.21.15.150:0/2661437416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4b570 0x5587dcd4b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc4ffff700 1 -- 172.21.15.150:0/2661437416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587dcaeb3b0 con 0x5587dcd4a6a0 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc4f7fe700 1 --2- 172.21.15.150:0/2661437416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4b570 0x5587dcd4b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.625+0000 7efc4ffff700 1 --2- 172.21.15.150:0/2661437416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4a6a0 0x5587dcd4aaf0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efc500682f0 tx=0x7efc4401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889eec28d0cd622 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.626+0000 7efc367fc700 1 -- 172.21.15.150:0/2661437416 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44029070 con 0x5587dcd4a6a0 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.626+0000 7efc367fc700 1 -- 172.21.15.150:0/2661437416 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc4401e040 con 0x5587dcd4a6a0 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.626+0000 7efc367fc700 1 -- 172.21.15.150:0/2661437416 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4400f660 con 0x5587dcd4a6a0 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.626+0000 7efc669c9b80 1 -- 172.21.15.150:0/2661437416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4a6a0 msgr2=0x5587dcd4aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.626+0000 7efc669c9b80 1 --2- 172.21.15.150:0/2661437416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4a6a0 0x5587dcd4aaf0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efc500682f0 tx=0x7efc4401f000 comp rx=0 tx=0).stop 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.626+0000 7efc669c9b80 1 -- 172.21.15.150:0/2661437416 shutdown_connections 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.626+0000 7efc669c9b80 1 --2- 172.21.15.150:0/2661437416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 0x5587dcd527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.626+0000 7efc669c9b80 1 --2- 172.21.15.150:0/2661437416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4a6a0 0x5587dcd4aaf0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.626+0000 7efc669c9b80 1 --2- 172.21.15.150:0/2661437416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4b570 0x5587dcd4b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.626+0000 7efc669c9b80 1 -- 172.21.15.150:0/2661437416 >> 172.21.15.150:0/2661437416 conn(0x5587dccaca60 msgr2=0x5587dcd44fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.627+0000 7efc669c9b80 1 -- 172.21.15.150:0/2661437416 shutdown_connections 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.627+0000 7efc669c9b80 1 -- 172.21.15.150:0/2661437416 wait complete. 2024-05-10T04:05:50.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.627+0000 7efc669c9b80 1 -- start start 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.627+0000 7efc669c9b80 1 Processor -- start 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.627+0000 7efc669c9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4a6a0 0x5587dcd545f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.627+0000 7efc669c9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4b570 0x5587dcd54b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.627+0000 7efc669c9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4bec0 0x5587dcd5ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.627+0000 7efc669c9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587dcc06560 con 0x5587dcd4bec0 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.627+0000 7efc669c9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587dcc06200 con 0x5587dcd4b570 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.627+0000 7efc669c9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587dcaed6a0 con 0x5587dcd4a6a0 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.627+0000 7efc669c9b80 10 client.0 _ll_register_callbacks cb 0x5587dcc0fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.627+0000 7efc54c22700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4bec0 0x5587dcd5ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc54c22700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4bec0 0x5587dcd5ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55866/0 (socket says 172.21.15.150:55866) 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc54c22700 1 -- 172.21.15.150:0/3618434248 learned_addr learned my addr 172.21.15.150:0/3618434248 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc4f7fe700 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4b570 0x5587dcd54b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc4ffff700 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4a6a0 0x5587dcd545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc54c22700 1 -- 172.21.15.150:0/3618434248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4a6a0 msgr2=0x5587dcd545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc54c22700 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4a6a0 0x5587dcd545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc54c22700 1 -- 172.21.15.150:0/3618434248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4b570 msgr2=0x5587dcd54b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc54c22700 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4b570 0x5587dcd54b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc54c22700 1 -- 172.21.15.150:0/3618434248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5010aa50 con 0x5587dcd4bec0 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc4ffff700 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4a6a0 0x5587dcd545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc4f7fe700 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4b570 0x5587dcd54b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc54c22700 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4bec0 0x5587dcd5ba90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efc480a14e0 tx=0x7efc48004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d428912c1622139 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc2f7fe700 1 -- 172.21.15.150:0/3618434248 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc48099c90 con 0x5587dcd4bec0 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc2f7fe700 1 -- 172.21.15.150:0/3618434248 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efc48099590 con 0x5587dcd4bec0 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc669c9b80 1 -- 172.21.15.150:0/3618434248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587dcbc3270 con 0x5587dcd4bec0 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc2f7fe700 1 -- 172.21.15.150:0/3618434248 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc4809d3c0 con 0x5587dcd4bec0 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc669c9b80 10 client.16098 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc669c9b80 1 -- 172.21.15.150:0/3618434248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5587dca97b60 con 0x5587dcd4bec0 2024-05-10T04:05:50.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.628+0000 7efc2f7fe700 1 -- 172.21.15.150:0/3618434248 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc480b3020 con 0x5587dcd4bec0 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.629+0000 7efc2f7fe700 1 -- 172.21.15.150:0/3618434248 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efc48084150 con 0x5587dcd4bec0 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.629+0000 7efc669c9b80 10 client.16098 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.629+0000 7efc669c9b80 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.629+0000 7efc669c9b80 20 client.16098 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.629+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5587dcd60180 0x5587dcd625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.629+0000 7efc669c9b80 4 client.16098 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.629+0000 7efc669c9b80 1 -- 172.21.15.150:0/3618434248 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587dcd19780 con 0x5587dcd60180 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.629+0000 7efc4ffff700 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5587dcd60180 0x5587dcd625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.629+0000 7efc2f7fe700 10 client.16098 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.629+0000 7efc4ffff700 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5587dcd60180 0x5587dcd625d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d2f1b14ca7b5bb9 server_cookie=94c95e3e8d719dba in_seq=0 out_seq=0 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc2f7fe700 1 -- 172.21.15.150:0/3618434248 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587dcd19780 con 0x5587dcd60180 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc2f7fe700 10 client.16098 handle_command_reply: tid=0 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc669c9b80 1 client.16098 shutdown 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc669c9b80 1 -- 172.21.15.150:0/3618434248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4bec0 msgr2=0x5587dcd5ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4bec0 0x5587dcd5ba90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efc480a14e0 tx=0x7efc48004c40 comp rx=0 tx=0).stop 2024-05-10T04:05:50.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc669c9b80 1 -- 172.21.15.150:0/3618434248 shutdown_connections 2024-05-10T04:05:50.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5587dcd60180 0x5587dcd625d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4bec0 0x5587dcd5ba90 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4a6a0 0x5587dcd545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3618434248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4b570 0x5587dcd54b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc2f7fe700 0 client.16098 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:50.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc669c9b80 1 -- 172.21.15.150:0/3618434248 >> 172.21.15.150:0/3618434248 conn(0x5587dccaca60 msgr2=0x5587dcd44fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:50.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc669c9b80 1 -- 172.21.15.150:0/3618434248 shutdown_connections 2024-05-10T04:05:50.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc669c9b80 1 -- 172.21.15.150:0/3618434248 wait complete. 2024-05-10T04:05:50.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.630+0000 7efc669c9b80 20 client.16098 trim_cache size 0 max 16384 2024-05-10T04:05:50.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.647+0000 7efc669c9b80 1 Processor -- start 2024-05-10T04:05:50.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.647+0000 7efc669c9b80 1 -- start start 2024-05-10T04:05:50.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.647+0000 7efc669c9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4a6a0 0x5587dcd625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.647+0000 7efc669c9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 0x5587dcd5ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.647+0000 7efc669c9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 0x5587dcd4e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.647+0000 7efc669c9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587dcd4b3f0 con 0x5587dcd4b570 2024-05-10T04:05:50.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc669c9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587dcd5ff80 con 0x5587dcd4a6a0 2024-05-10T04:05:50.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc669c9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587dcd46920 con 0x5587dcd4bec0 2024-05-10T04:05:50.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc4f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 0x5587dcd5ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc4f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 0x5587dcd5ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55870/0 (socket says 172.21.15.150:55870) 2024-05-10T04:05:50.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc4f7fe700 1 -- 172.21.15.150:0/3772570831 learned_addr learned my addr 172.21.15.150:0/3772570831 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:50.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc4ffff700 1 --2- 172.21.15.150:0/3772570831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4a6a0 0x5587dcd625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc54c22700 1 --2- 172.21.15.150:0/3772570831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 0x5587dcd4e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc4f7fe700 1 -- 172.21.15.150:0/3772570831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 msgr2=0x5587dcd4e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc4f7fe700 1 --2- 172.21.15.150:0/3772570831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 0x5587dcd4e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc4f7fe700 1 -- 172.21.15.150:0/3772570831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4a6a0 msgr2=0x5587dcd625a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc4f7fe700 1 --2- 172.21.15.150:0/3772570831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4a6a0 0x5587dcd625a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc4f7fe700 1 -- 172.21.15.150:0/3772570831 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587dcc30560 con 0x5587dcd4b570 2024-05-10T04:05:50.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc4f7fe700 1 --2- 172.21.15.150:0/3772570831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 0x5587dcd5ba50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7efc4000bd80 tx=0x7efc400126d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d340d0dd3dc4a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:50.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc36ffd700 1 -- 172.21.15.150:0/3772570831 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40012db0 con 0x5587dcd4b570 2024-05-10T04:05:50.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc36ffd700 1 -- 172.21.15.150:0/3772570831 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc4001a8a0 con 0x5587dcd4b570 2024-05-10T04:05:50.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.648+0000 7efc36ffd700 1 -- 172.21.15.150:0/3772570831 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4000e860 con 0x5587dcd4b570 2024-05-10T04:05:50.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.649+0000 7efc669c9b80 1 -- 172.21.15.150:0/3772570831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 msgr2=0x5587dcd5ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.649+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3772570831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 0x5587dcd5ba50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7efc4000bd80 tx=0x7efc400126d0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.649+0000 7efc669c9b80 1 -- 172.21.15.150:0/3772570831 shutdown_connections 2024-05-10T04:05:50.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.649+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3772570831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 0x5587dcd4e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.649+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3772570831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4a6a0 0x5587dcd625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.649+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3772570831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 0x5587dcd5ba50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.649+0000 7efc669c9b80 1 -- 172.21.15.150:0/3772570831 >> 172.21.15.150:0/3772570831 conn(0x5587dccaca60 msgr2=0x5587dcd41180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:50.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.649+0000 7efc669c9b80 1 -- 172.21.15.150:0/3772570831 shutdown_connections 2024-05-10T04:05:50.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.649+0000 7efc669c9b80 1 -- 172.21.15.150:0/3772570831 wait complete. 2024-05-10T04:05:50.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.649+0000 7efc669c9b80 1 -- start start 2024-05-10T04:05:50.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc669c9b80 1 Processor -- start 2024-05-10T04:05:50.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc669c9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4a6a0 0x5587dcd59740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc669c9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 0x5587dcd59c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc669c9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 0x5587dcd5ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc669c9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587dcd5b0a0 con 0x5587dcd4b570 2024-05-10T04:05:50.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc669c9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587dcd5b210 con 0x5587dcd4a6a0 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc669c9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587dcd5b3d0 con 0x5587dcd4bec0 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc669c9b80 10 client.0 _ll_register_callbacks cb 0x5587dcc0fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc4f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 0x5587dcd59c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc4f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 0x5587dcd59c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55874/0 (socket says 172.21.15.150:55874) 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc4f7fe700 1 -- 172.21.15.150:0/3960455630 learned_addr learned my addr 172.21.15.150:0/3960455630 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc4ffff700 1 --2- 172.21.15.150:0/3960455630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4a6a0 0x5587dcd59740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc54c22700 1 --2- 172.21.15.150:0/3960455630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 0x5587dcd5ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc4f7fe700 1 -- 172.21.15.150:0/3960455630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 msgr2=0x5587dcd5ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc4f7fe700 1 --2- 172.21.15.150:0/3960455630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 0x5587dcd5ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc4f7fe700 1 -- 172.21.15.150:0/3960455630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4a6a0 msgr2=0x5587dcd59740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc4f7fe700 1 --2- 172.21.15.150:0/3960455630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4a6a0 0x5587dcd59740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc4f7fe700 1 -- 172.21.15.150:0/3960455630 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4000b9e0 con 0x5587dcd4b570 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc4f7fe700 1 --2- 172.21.15.150:0/3960455630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 0x5587dcd59c80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7efc40012e60 tx=0x7efc40012e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bac0735a76beb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:50.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.650+0000 7efc2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc2f7fe700 1 -- 172.21.15.150:0/3960455630 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4001a6f0 con 0x5587dcd4b570 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc2f7fe700 1 -- 172.21.15.150:0/3960455630 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efc40004030 con 0x5587dcd4b570 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc669c9b80 1 -- 172.21.15.150:0/3960455630 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587dcd46bc0 con 0x5587dcd4b570 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc2f7fe700 1 -- 172.21.15.150:0/3960455630 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc400041f0 con 0x5587dcd4b570 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc669c9b80 10 client.16104 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc669c9b80 1 -- 172.21.15.150:0/3960455630 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5587dcaefcd0 con 0x5587dcd4b570 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc2f7fe700 1 -- 172.21.15.150:0/3960455630 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5587dcd46bc0 con 0x5587dcd4b570 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc2f7fe700 1 -- 172.21.15.150:0/3960455630 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efc40022070 con 0x5587dcd4b570 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc669c9b80 10 client.16104 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc669c9b80 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc669c9b80 20 client.16104 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3960455630 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5587dcdb73a0 0x5587dcdb97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc669c9b80 4 client.16104 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc669c9b80 1 -- 172.21.15.150:0/3960455630 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7efc4401e040 con 0x5587dcdb73a0 2024-05-10T04:05:50.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.651+0000 7efc4ffff700 1 --2- 172.21.15.150:0/3960455630 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5587dcdb73a0 0x5587dcdb97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc2f7fe700 10 client.16104 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc4ffff700 1 --2- 172.21.15.150:0/3960455630 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5587dcdb73a0 0x5587dcdb97c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f952ba3d675e83b server_cookie=fb9fc56c7e396b1a in_seq=0 out_seq=0 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc2f7fe700 1 -- 172.21.15.150:0/3960455630 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7efc4401e040 con 0x5587dcdb73a0 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc2f7fe700 10 client.16104 handle_command_reply: tid=0 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc669c9b80 1 client.16104 shutdown 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc669c9b80 1 -- 172.21.15.150:0/3960455630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 msgr2=0x5587dcd59c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3960455630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 0x5587dcd59c80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7efc40012e60 tx=0x7efc40012e90 comp rx=0 tx=0).stop 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc669c9b80 1 -- 172.21.15.150:0/3960455630 shutdown_connections 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3960455630 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5587dcdb73a0 0x5587dcdb97c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3960455630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587dcd4bec0 0x5587dcd5ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc2f7fe700 0 client.16104 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3960455630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587dcd4a6a0 0x5587dcd59740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc669c9b80 1 --2- 172.21.15.150:0/3960455630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587dcd4b570 0x5587dcd59c80 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc669c9b80 1 -- 172.21.15.150:0/3960455630 >> 172.21.15.150:0/3960455630 conn(0x5587dccaca60 msgr2=0x5587dcd41180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc669c9b80 1 -- 172.21.15.150:0/3960455630 shutdown_connections 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.652+0000 7efc669c9b80 1 -- 172.21.15.150:0/3960455630 wait complete. 2024-05-10T04:05:50.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.653+0000 7efc669c9b80 20 client.16104 trim_cache size 0 max 16384 2024-05-10T04:05:50.661 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:05:50.661 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T04:05:50.661 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": { 2024-05-10T04:05:50.661 INFO:teuthology.orchestra.run.smithi150.stdout: "96b85304-204e-4c3a-a5ea-e3a815e52b2b": { 2024-05-10T04:05:50.661 INFO:teuthology.orchestra.run.smithi150.stdout: "path": "/", 2024-05-10T04:05:50.661 INFO:teuthology.orchestra.run.smithi150.stdout: "tag": "96b85304-204e-4c3a-a5ea-e3a815e52b2b", 2024-05-10T04:05:50.661 INFO:teuthology.orchestra.run.smithi150.stdout: "options": "recursive,force" 2024-05-10T04:05:50.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:05:50.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:05:50.661 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:05:50.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.653+0000 7efc56685700 1 -- 172.21.15.150:0/1162534201 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7efc380a11c0 msgr2=0x7efc380a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.653+0000 7efc56685700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7efc380a11c0 0x7efc380a36d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efc4400b5c0 tx=0x7efc44011040 comp rx=0 tx=0).stop 2024-05-10T04:05:50.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.653+0000 7efc56685700 1 -- 172.21.15.150:0/1162534201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc501031e0 msgr2=0x7efc50101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:50.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.653+0000 7efc56685700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc501031e0 0x7efc50101390 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efc48007ee0 tx=0x7efc4800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.653+0000 7efc56685700 1 -- 172.21.15.150:0/1162534201 shutdown_connections 2024-05-10T04:05:50.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.653+0000 7efc56685700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7efc380a11c0 0x7efc380a36d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.653+0000 7efc56685700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc501031e0 0x7efc50101390 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.653+0000 7efc56685700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc500ffc20 0x7efc501027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.653+0000 7efc56685700 1 --2- 172.21.15.150:0/1162534201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc50100570 0x7efc50100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:50.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.653+0000 7efc56685700 1 -- 172.21.15.150:0/1162534201 >> 172.21.15.150:0/1162534201 conn(0x7efc500fb8f0 msgr2=0x7efc50106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:50.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.653+0000 7efc56685700 1 -- 172.21.15.150:0/1162534201 shutdown_connections 2024-05-10T04:05:50.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:05:50.653+0000 7efc56685700 1 -- 172.21.15.150:0/1162534201 wait complete. 2024-05-10T04:05:50.668 INFO:tasks.cephfs.filesystem:scrub status for tag:96b85304-204e-4c3a-a5ea-e3a815e52b2b - {'path': '/', 'tag': '96b85304-204e-4c3a-a5ea-e3a815e52b2b', 'options': 'recursive,force'} 2024-05-10T04:05:51.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:51 smithi150 ceph-mon[42477]: scrub queued for path: / 2024-05-10T04:05:51.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:51 smithi150 ceph-mon[42477]: scrub summary: idle+waiting paths [/] 2024-05-10T04:05:51.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:51 smithi150 ceph-mon[42477]: scrub summary: active paths [/] 2024-05-10T04:05:51.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:51 smithi150 ceph-mon[42477]: scrub summary: idle+waiting paths [/] 2024-05-10T04:05:51.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:51 smithi150 ceph-mon[42477]: scrub summary: idle 2024-05-10T04:05:51.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:51 smithi181 ceph-mon[46602]: scrub queued for path: / 2024-05-10T04:05:51.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:51 smithi181 ceph-mon[46602]: scrub summary: idle+waiting paths [/] 2024-05-10T04:05:51.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:51 smithi181 ceph-mon[46602]: scrub summary: active paths [/] 2024-05-10T04:05:51.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:51 smithi181 ceph-mon[46602]: scrub summary: idle+waiting paths [/] 2024-05-10T04:05:51.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:51 smithi181 ceph-mon[46602]: scrub summary: idle 2024-05-10T04:05:51.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:51 smithi161 ceph-mon[46918]: scrub queued for path: / 2024-05-10T04:05:51.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:51 smithi161 ceph-mon[46918]: scrub summary: idle+waiting paths [/] 2024-05-10T04:05:51.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:51 smithi161 ceph-mon[46918]: scrub summary: active paths [/] 2024-05-10T04:05:51.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:51 smithi161 ceph-mon[46918]: scrub summary: idle+waiting paths [/] 2024-05-10T04:05:51.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:51 smithi161 ceph-mon[46918]: scrub summary: idle 2024-05-10T04:05:52.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:52 smithi150 ceph-mon[42477]: pgmap v328: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 135 MiB/s wr, 45 op/s 2024-05-10T04:05:52.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:52 smithi181 ceph-mon[46602]: pgmap v328: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 135 MiB/s wr, 45 op/s 2024-05-10T04:05:52.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:52 smithi161 ceph-mon[46918]: pgmap v328: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 135 MiB/s wr, 45 op/s 2024-05-10T04:05:54.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:54 smithi150 ceph-mon[42477]: pgmap v329: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 119 MiB/s wr, 39 op/s 2024-05-10T04:05:54.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:54 smithi181 ceph-mon[46602]: pgmap v329: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 119 MiB/s wr, 39 op/s 2024-05-10T04:05:54.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:54 smithi161 ceph-mon[46918]: pgmap v329: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 119 MiB/s wr, 39 op/s 2024-05-10T04:05:56.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:56 smithi150 ceph-mon[42477]: pgmap v330: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 88 MiB/s wr, 33 op/s 2024-05-10T04:05:56.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:56 smithi181 ceph-mon[46602]: pgmap v330: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 88 MiB/s wr, 33 op/s 2024-05-10T04:05:56.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:56 smithi161 ceph-mon[46918]: pgmap v330: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 88 MiB/s wr, 33 op/s 2024-05-10T04:05:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.807+0000 7fb95411d700 1 -- 172.21.15.150:0/690388387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c101080 msgr2=0x7fb94c101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.807+0000 7fb95411d700 1 --2- 172.21.15.150:0/690388387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c101080 0x7fb94c101490 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb9480099c0 tx=0x7fb948009cd0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.807+0000 7fb95411d700 1 -- 172.21.15.150:0/690388387 shutdown_connections 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.807+0000 7fb95411d700 1 --2- 172.21.15.150:0/690388387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb94c102560 0x7fb94c109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.807+0000 7fb95411d700 1 --2- 172.21.15.150:0/690388387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c101080 0x7fb94c101490 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.807+0000 7fb95411d700 1 --2- 172.21.15.150:0/690388387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb94c101a60 0x7fb94c101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.807+0000 7fb95411d700 1 -- 172.21.15.150:0/690388387 >> 172.21.15.150:0/690388387 conn(0x7fb94c0fb840 msgr2=0x7fb94c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.807+0000 7fb95411d700 1 -- 172.21.15.150:0/690388387 shutdown_connections 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.807+0000 7fb95411d700 1 -- 172.21.15.150:0/690388387 wait complete. 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb95411d700 1 Processor -- start 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb95411d700 1 -- start start 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb95411d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb94c101080 0x7fb94c196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb95411d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb94c101a60 0x7fb94c1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb95411d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c102560 0x7fb94c19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb95411d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c19e940 con 0x7fb94c102560 2024-05-10T04:05:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb95411d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c19ea80 con 0x7fb94c101a60 2024-05-10T04:05:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb95411d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c19ec10 con 0x7fb94c101080 2024-05-10T04:05:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb9526ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c102560 0x7fb94c19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb9526ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c102560 0x7fb94c19e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55900/0 (socket says 172.21.15.150:55900) 2024-05-10T04:05:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb9526ba700 1 -- 172.21.15.150:0/2239376778 learned_addr learned my addr 172.21.15.150:0/2239376778 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb9516b8700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb94c101a60 0x7fb94c1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb951eb9700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb94c101080 0x7fb94c196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb9526ba700 1 -- 172.21.15.150:0/2239376778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb94c101080 msgr2=0x7fb94c196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb9526ba700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb94c101080 0x7fb94c196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb9526ba700 1 -- 172.21.15.150:0/2239376778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb94c101a60 msgr2=0x7fb94c1973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.808+0000 7fb9526ba700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb94c101a60 0x7fb94c1973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.809+0000 7fb9526ba700 1 -- 172.21.15.150:0/2239376778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9480095f0 con 0x7fb94c102560 2024-05-10T04:05:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.809+0000 7fb9526ba700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c102560 0x7fb94c19e370 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb944007f40 tx=0x7fb94400f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.809+0000 7fb942ffd700 1 -- 172.21.15.150:0/2239376778 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94400fd30 con 0x7fb94c102560 2024-05-10T04:05:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.809+0000 7fb942ffd700 1 -- 172.21.15.150:0/2239376778 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb94400fe90 con 0x7fb94c102560 2024-05-10T04:05:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.809+0000 7fb95411d700 1 -- 172.21.15.150:0/2239376778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c0ffd30 con 0x7fb94c102560 2024-05-10T04:05:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.809+0000 7fb942ffd700 1 -- 172.21.15.150:0/2239376778 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944005020 con 0x7fb94c102560 2024-05-10T04:05:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.809+0000 7fb95411d700 1 -- 172.21.15.150:0/2239376778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c100240 con 0x7fb94c102560 2024-05-10T04:05:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.812+0000 7fb942ffd700 1 -- 172.21.15.150:0/2239376778 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb944012460 con 0x7fb94c102560 2024-05-10T04:05:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.812+0000 7fb942ffd700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb9380a11a0 0x7fb9380a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb964461b80 1 Processor -- start 2024-05-10T04:05:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb951eb9700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb9380a11a0 0x7fb9380a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb942ffd700 1 -- 172.21.15.150:0/2239376778 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb9440cedb0 con 0x7fb94c102560 2024-05-10T04:05:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb964461b80 1 -- start start 2024-05-10T04:05:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb964461b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 0x563dc1eb35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb964461b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb4050 0x563dc1eb4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb964461b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 0x563dc1ebb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb964461b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc1dda0f0 con 0x563dc1eb49a0 2024-05-10T04:05:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb964461b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc1dd7ab0 con 0x563dc1eb4050 2024-05-10T04:05:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb964461b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc1dd33a0 con 0x563dc1eb3180 2024-05-10T04:05:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb951eb9700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb9380a11a0 0x7fb9380a36b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb948009fd0 tx=0x7fb948005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb9526ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 0x563dc1ebb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb9526ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 0x563dc1ebb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55910/0 (socket says 172.21.15.150:55910) 2024-05-10T04:05:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb9526ba700 1 -- 172.21.15.150:0/710165473 learned_addr learned my addr 172.21.15.150:0/710165473 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.813+0000 7fb9516b8700 1 --2- 172.21.15.150:0/710165473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb4050 0x563dc1eb4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.814+0000 7fb951eb9700 1 --2- 172.21.15.150:0/710165473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 0x563dc1eb35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.814+0000 7fb9526ba700 1 -- 172.21.15.150:0/710165473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 msgr2=0x563dc1eb35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.814+0000 7fb9526ba700 1 --2- 172.21.15.150:0/710165473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 0x563dc1eb35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.814+0000 7fb9526ba700 1 -- 172.21.15.150:0/710165473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb4050 msgr2=0x563dc1eb4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.814+0000 7fb9526ba700 1 --2- 172.21.15.150:0/710165473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb4050 0x563dc1eb4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.814+0000 7fb9526ba700 1 -- 172.21.15.150:0/710165473 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563dc1c54350 con 0x563dc1eb49a0 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.814+0000 7fb9526ba700 1 --2- 172.21.15.150:0/710165473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 0x563dc1ebb2a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb94409be80 tx=0x7fb94409c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26404179f04efa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.814+0000 7fb937fff700 1 -- 172.21.15.150:0/710165473 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9440a4370 con 0x563dc1eb49a0 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.814+0000 7fb937fff700 1 -- 172.21.15.150:0/710165473 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb944004ad0 con 0x563dc1eb49a0 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.814+0000 7fb937fff700 1 -- 172.21.15.150:0/710165473 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94409b3e0 con 0x563dc1eb49a0 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.815+0000 7fb964461b80 1 -- 172.21.15.150:0/710165473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 msgr2=0x563dc1ebb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.815+0000 7fb964461b80 1 --2- 172.21.15.150:0/710165473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 0x563dc1ebb2a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb94409be80 tx=0x7fb94409c6c0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.815+0000 7fb964461b80 1 -- 172.21.15.150:0/710165473 shutdown_connections 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.815+0000 7fb964461b80 1 --2- 172.21.15.150:0/710165473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 0x563dc1ebb2a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.815+0000 7fb964461b80 1 --2- 172.21.15.150:0/710165473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 0x563dc1eb35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.815+0000 7fb964461b80 1 --2- 172.21.15.150:0/710165473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb4050 0x563dc1eb4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.815+0000 7fb964461b80 1 -- 172.21.15.150:0/710165473 >> 172.21.15.150:0/710165473 conn(0x563dc1e15a00 msgr2=0x563dc1eada70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.815+0000 7fb964461b80 1 -- 172.21.15.150:0/710165473 shutdown_connections 2024-05-10T04:05:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.815+0000 7fb964461b80 1 -- 172.21.15.150:0/710165473 wait complete. 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.815+0000 7fb964461b80 1 -- start start 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.815+0000 7fb964461b80 1 Processor -- start 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb964461b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb3180 0x563dc1ebd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb964461b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb4050 0x563dc1ebd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb964461b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb49a0 0x563dc1ec45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb964461b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc1d6f500 con 0x563dc1eb3180 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb964461b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc1d6f1a0 con 0x563dc1eb49a0 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb964461b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc1c56640 con 0x563dc1eb4050 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb964461b80 10 client.0 _ll_register_callbacks cb 0x563dc1d78ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb951eb9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb3180 0x563dc1ebd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb951eb9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb3180 0x563dc1ebd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55916/0 (socket says 172.21.15.150:55916) 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb951eb9700 1 -- 172.21.15.150:0/3803488184 learned_addr learned my addr 172.21.15.150:0/3803488184 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb9516b8700 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb4050 0x563dc1ebd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb9526ba700 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb49a0 0x563dc1ec45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb951eb9700 1 -- 172.21.15.150:0/3803488184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb4050 msgr2=0x563dc1ebd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb951eb9700 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb4050 0x563dc1ebd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb951eb9700 1 -- 172.21.15.150:0/3803488184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb49a0 msgr2=0x563dc1ec45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb951eb9700 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb49a0 0x563dc1ec45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb951eb9700 1 -- 172.21.15.150:0/3803488184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c100240 con 0x563dc1eb3180 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.816+0000 7fb9526ba700 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb49a0 0x563dc1ec45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb951eb9700 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb3180 0x563dc1ebd130 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb9480099c0 tx=0x7fb948009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb06537cec9079e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb9357fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb9357fa700 1 -- 172.21.15.150:0/3803488184 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94802a070 con 0x563dc1eb3180 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb9357fa700 1 -- 172.21.15.150:0/3803488184 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb94800f460 con 0x563dc1eb3180 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb964461b80 1 -- 172.21.15.150:0/3803488184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563dc1d2c210 con 0x563dc1eb3180 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb9357fa700 1 -- 172.21.15.150:0/3803488184 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb94801d040 con 0x563dc1eb3180 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb964461b80 10 client.16116 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb964461b80 1 -- 172.21.15.150:0/3803488184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563dc1c00b00 con 0x563dc1eb3180 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb9357fa700 1 -- 172.21.15.150:0/3803488184 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563dc1d2c210 con 0x563dc1eb3180 2024-05-10T04:05:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb9357fa700 1 -- 172.21.15.150:0/3803488184 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb948023640 con 0x563dc1eb3180 2024-05-10T04:05:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb964461b80 10 client.16116 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb964461b80 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb964461b80 20 client.16116 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb964461b80 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563dc1ec8c60 0x563dc1ecb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb964461b80 4 client.16116 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:05:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.817+0000 7fb964461b80 1 -- 172.21.15.150:0/3803488184 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563dc1e82720 con 0x563dc1ec8c60 2024-05-10T04:05:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.818+0000 7fb9516b8700 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563dc1ec8c60 0x563dc1ecb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.818+0000 7fb9357fa700 10 client.16116 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.818+0000 7fb9516b8700 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563dc1ec8c60 0x563dc1ecb0b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc943111980be5c server_cookie=85b5dbea361ca456 in_seq=0 out_seq=0 2024-05-10T04:05:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb9357fa700 1 -- 172.21.15.150:0/3803488184 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563dc1e82720 con 0x563dc1ec8c60 2024-05-10T04:05:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb9357fa700 10 client.16116 handle_command_reply: tid=0 2024-05-10T04:05:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb964461b80 1 client.16116 shutdown 2024-05-10T04:05:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb964461b80 1 -- 172.21.15.150:0/3803488184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb3180 msgr2=0x563dc1ebd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb964461b80 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb3180 0x563dc1ebd130 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb9480099c0 tx=0x7fb948009cd0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb964461b80 1 -- 172.21.15.150:0/3803488184 shutdown_connections 2024-05-10T04:05:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb964461b80 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563dc1ec8c60 0x563dc1ecb0b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb964461b80 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb49a0 0x563dc1ec45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb964461b80 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb3180 0x563dc1ebd130 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb9357fa700 0 client.16116 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb964461b80 1 --2- 172.21.15.150:0/3803488184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb4050 0x563dc1ebd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb964461b80 1 -- 172.21.15.150:0/3803488184 >> 172.21.15.150:0/3803488184 conn(0x563dc1e15a00 msgr2=0x563dc1eada40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb964461b80 1 -- 172.21.15.150:0/3803488184 shutdown_connections 2024-05-10T04:05:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb964461b80 1 -- 172.21.15.150:0/3803488184 wait complete. 2024-05-10T04:05:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.819+0000 7fb964461b80 20 client.16116 trim_cache size 0 max 16384 2024-05-10T04:05:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.834+0000 7fb964461b80 1 Processor -- start 2024-05-10T04:05:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.834+0000 7fb964461b80 1 -- start start 2024-05-10T04:05:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.834+0000 7fb964461b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 0x563dc1ecb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.834+0000 7fb964461b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb4050 0x563dc1ec4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.834+0000 7fb964461b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb49a0 0x563dc1ea9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.834+0000 7fb964461b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc1eaf230 con 0x563dc1eb4050 2024-05-10T04:05:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.834+0000 7fb964461b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc1eb3d90 con 0x563dc1eb49a0 2024-05-10T04:05:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.834+0000 7fb964461b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc1eae560 con 0x563dc1eb3180 2024-05-10T04:05:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.834+0000 7fb9516b8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb4050 0x563dc1ec4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.834+0000 7fb9516b8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb4050 0x563dc1ec4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55930/0 (socket says 172.21.15.150:55930) 2024-05-10T04:05:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.834+0000 7fb9516b8700 1 -- 172.21.15.150:0/4284946763 learned_addr learned my addr 172.21.15.150:0/4284946763 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.835+0000 7fb951eb9700 1 --2- 172.21.15.150:0/4284946763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 0x563dc1ecb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.835+0000 7fb9516b8700 1 -- 172.21.15.150:0/4284946763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 msgr2=0x563dc1ecb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.835+0000 7fb9516b8700 1 --2- 172.21.15.150:0/4284946763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 0x563dc1ecb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.835+0000 7fb9516b8700 1 -- 172.21.15.150:0/4284946763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb49a0 msgr2=0x563dc1ea9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.835+0000 7fb9516b8700 1 --2- 172.21.15.150:0/4284946763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb49a0 0x563dc1ea9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.835+0000 7fb9516b8700 1 -- 172.21.15.150:0/4284946763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563dc1d99500 con 0x563dc1eb4050 2024-05-10T04:05:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.835+0000 7fb951eb9700 1 --2- 172.21.15.150:0/4284946763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 0x563dc1ecb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:05:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.835+0000 7fb9516b8700 1 --2- 172.21.15.150:0/4284946763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb4050 0x563dc1ec4590 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x563dc1ecb920 tx=0x7fb93c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e28c74bb5c6f790 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.835+0000 7fb940ff9700 1 -- 172.21.15.150:0/4284946763 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c017040 con 0x563dc1eb4050 2024-05-10T04:05:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.835+0000 7fb940ff9700 1 -- 172.21.15.150:0/4284946763 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb93c018040 con 0x563dc1eb4050 2024-05-10T04:05:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.835+0000 7fb940ff9700 1 -- 172.21.15.150:0/4284946763 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c003ea0 con 0x563dc1eb4050 2024-05-10T04:05:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.836+0000 7fb964461b80 1 -- 172.21.15.150:0/4284946763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb4050 msgr2=0x563dc1ec4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.836+0000 7fb964461b80 1 --2- 172.21.15.150:0/4284946763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb4050 0x563dc1ec4590 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x563dc1ecb920 tx=0x7fb93c009260 comp rx=0 tx=0).stop 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.836+0000 7fb964461b80 1 -- 172.21.15.150:0/4284946763 shutdown_connections 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.836+0000 7fb964461b80 1 --2- 172.21.15.150:0/4284946763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb49a0 0x563dc1ea9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.836+0000 7fb964461b80 1 --2- 172.21.15.150:0/4284946763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 0x563dc1ecb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.836+0000 7fb964461b80 1 --2- 172.21.15.150:0/4284946763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb4050 0x563dc1ec4590 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.836+0000 7fb964461b80 1 -- 172.21.15.150:0/4284946763 >> 172.21.15.150:0/4284946763 conn(0x563dc1e15a00 msgr2=0x563dc1eada40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.836+0000 7fb964461b80 1 -- 172.21.15.150:0/4284946763 shutdown_connections 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.836+0000 7fb964461b80 1 -- 172.21.15.150:0/4284946763 wait complete. 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.836+0000 7fb964461b80 1 -- start start 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.836+0000 7fb964461b80 1 Processor -- start 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb964461b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 0x563dc1ec17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb964461b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb4050 0x563dc1ec1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb964461b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 0x563dc1ec2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb964461b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc1ec3150 con 0x563dc1eb49a0 2024-05-10T04:05:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb964461b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc1ec32c0 con 0x563dc1eb4050 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb964461b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc1ec3480 con 0x563dc1eb3180 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb964461b80 10 client.0 _ll_register_callbacks cb 0x563dc1d78ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb9526ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 0x563dc1ec2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb9526ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 0x563dc1ec2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55942/0 (socket says 172.21.15.150:55942) 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb9526ba700 1 -- 172.21.15.150:0/2530516815 learned_addr learned my addr 172.21.15.150:0/2530516815 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb951eb9700 1 --2- 172.21.15.150:0/2530516815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 0x563dc1ec17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb9516b8700 1 --2- 172.21.15.150:0/2530516815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb4050 0x563dc1ec1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb9526ba700 1 -- 172.21.15.150:0/2530516815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 msgr2=0x563dc1ec17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb9526ba700 1 --2- 172.21.15.150:0/2530516815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 0x563dc1ec17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb9526ba700 1 -- 172.21.15.150:0/2530516815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb4050 msgr2=0x563dc1ec1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb9526ba700 1 --2- 172.21.15.150:0/2530516815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb4050 0x563dc1ec1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb9526ba700 1 -- 172.21.15.150:0/2530516815 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c011440 con 0x563dc1eb49a0 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb9526ba700 1 --2- 172.21.15.150:0/2530516815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 0x563dc1ec2c10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb94409bbf0 tx=0x7fb944098360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe3e92b183681b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:05:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb9357fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.837+0000 7fb9357fa700 1 -- 172.21.15.150:0/2530516815 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94409a290 con 0x563dc1eb49a0 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb9357fa700 1 -- 172.21.15.150:0/2530516815 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb944004d40 con 0x563dc1eb49a0 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb964461b80 1 -- 172.21.15.150:0/2530516815 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb944004ad0 con 0x563dc1eb49a0 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb9357fa700 1 -- 172.21.15.150:0/2530516815 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb94409b3e0 con 0x563dc1eb49a0 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb964461b80 10 client.16122 fetch_fsmap learned FSMap version 25 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb964461b80 1 -- 172.21.15.150:0/2530516815 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563dc1c58c70 con 0x563dc1eb49a0 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb9357fa700 1 -- 172.21.15.150:0/2530516815 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb944004ad0 con 0x563dc1eb49a0 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb9357fa700 1 -- 172.21.15.150:0/2530516815 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb94409c150 con 0x563dc1eb49a0 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb964461b80 10 client.16122 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb964461b80 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb964461b80 20 client.16122 populate_metadata read hostname 'smithi150' 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb964461b80 1 --2- 172.21.15.150:0/2530516815 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563dc1f1f240 0x563dc1f21660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb964461b80 4 client.16122 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb964461b80 1 -- 172.21.15.150:0/2530516815 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563dc1dcbe60 con 0x563dc1f1f240 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.838+0000 7fb951eb9700 1 --2- 172.21.15.150:0/2530516815 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563dc1f1f240 0x563dc1f21660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:05:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.839+0000 7fb9357fa700 10 client.16122 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.839+0000 7fb951eb9700 1 --2- 172.21.15.150:0/2530516815 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563dc1f1f240 0x563dc1f21660 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=598e3321a746904a server_cookie=72b9c1d037e175b2 in_seq=0 out_seq=0 2024-05-10T04:05:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.848+0000 7fb9357fa700 1 -- 172.21.15.150:0/2530516815 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563dc1dcbe60 con 0x563dc1f1f240 2024-05-10T04:05:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.848+0000 7fb9357fa700 10 client.16122 handle_command_reply: tid=0 2024-05-10T04:05:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.848+0000 7fb964461b80 1 client.16122 shutdown 2024-05-10T04:05:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.848+0000 7fb964461b80 1 -- 172.21.15.150:0/2530516815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 msgr2=0x563dc1ec2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.848+0000 7fb964461b80 1 --2- 172.21.15.150:0/2530516815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 0x563dc1ec2c10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb94409bbf0 tx=0x7fb944098360 comp rx=0 tx=0).stop 2024-05-10T04:05:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.848+0000 7fb964461b80 1 -- 172.21.15.150:0/2530516815 shutdown_connections 2024-05-10T04:05:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.848+0000 7fb964461b80 1 --2- 172.21.15.150:0/2530516815 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x563dc1f1f240 0x563dc1f21660 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.848+0000 7fb964461b80 1 --2- 172.21.15.150:0/2530516815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dc1eb49a0 0x563dc1ec2c10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.848+0000 7fb964461b80 1 --2- 172.21.15.150:0/2530516815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dc1eb3180 0x563dc1ec17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.848+0000 7fb9357fa700 0 client.16122 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:05:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.848+0000 7fb964461b80 1 --2- 172.21.15.150:0/2530516815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563dc1eb4050 0x563dc1ec1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.848+0000 7fb964461b80 1 -- 172.21.15.150:0/2530516815 >> 172.21.15.150:0/2530516815 conn(0x563dc1e15a00 msgr2=0x563dc1eada20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.848+0000 7fb964461b80 1 -- 172.21.15.150:0/2530516815 shutdown_connections 2024-05-10T04:05:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb964461b80 1 -- 172.21.15.150:0/2530516815 wait complete. 2024-05-10T04:05:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb964461b80 20 client.16122 trim_cache size 0 max 16384 2024-05-10T04:05:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:05:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:05:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:05:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:05:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb95411d700 1 -- 172.21.15.150:0/2239376778 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb9380a11a0 msgr2=0x7fb9380a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb95411d700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb9380a11a0 0x7fb9380a36b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb948009fd0 tx=0x7fb948005fb0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb95411d700 1 -- 172.21.15.150:0/2239376778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c102560 msgr2=0x7fb94c19e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:05:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb95411d700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c102560 0x7fb94c19e370 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb944007f40 tx=0x7fb94400f5c0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb95411d700 1 -- 172.21.15.150:0/2239376778 shutdown_connections 2024-05-10T04:05:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb95411d700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb9380a11a0 0x7fb9380a36b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb95411d700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c102560 0x7fb94c19e370 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb95411d700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb94c101080 0x7fb94c196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb95411d700 1 --2- 172.21.15.150:0/2239376778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb94c101a60 0x7fb94c1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:05:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb95411d700 1 -- 172.21.15.150:0/2239376778 >> 172.21.15.150:0/2239376778 conn(0x7fb94c0fb840 msgr2=0x7fb94c105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:05:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb95411d700 1 -- 172.21.15.150:0/2239376778 shutdown_connections 2024-05-10T04:05:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:05:57.849+0000 7fb95411d700 1 -- 172.21.15.150:0/2239376778 wait complete. 2024-05-10T04:05:58.232 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:58.232 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:58.233 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:58.233 INFO:tasks.workunit.client.0.smithi150.stdout:iozone test complete. 2024-05-10T04:05:58.233 INFO:tasks.workunit.client.0.smithi150.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-05-10T04:05:58.265 INFO:tasks.workunit.client.0.smithi150.stdout: Iozone: Performance Test of File I/O 2024-05-10T04:05:58.266 INFO:tasks.workunit.client.0.smithi150.stdout: Version $Revision: 3.489 $ 2024-05-10T04:05:58.266 INFO:tasks.workunit.client.0.smithi150.stdout: Compiled for 64 bit mode. 2024-05-10T04:05:58.266 INFO:tasks.workunit.client.0.smithi150.stdout: Build: linux-AMD64 2024-05-10T04:05:58.266 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:58.266 INFO:tasks.workunit.client.0.smithi150.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-05-10T04:05:58.267 INFO:tasks.workunit.client.0.smithi150.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-05-10T04:05:58.267 INFO:tasks.workunit.client.0.smithi150.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-05-10T04:05:58.267 INFO:tasks.workunit.client.0.smithi150.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-05-10T04:05:58.267 INFO:tasks.workunit.client.0.smithi150.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-05-10T04:05:58.267 INFO:tasks.workunit.client.0.smithi150.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-05-10T04:05:58.268 INFO:tasks.workunit.client.0.smithi150.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-05-10T04:05:58.268 INFO:tasks.workunit.client.0.smithi150.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-05-10T04:05:58.268 INFO:tasks.workunit.client.0.smithi150.stdout: Alexey Skidanov, Sudhir Kumar. 2024-05-10T04:05:58.268 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:58.268 INFO:tasks.workunit.client.0.smithi150.stdout: Run began: Fri May 10 04:05:58 2024 2024-05-10T04:05:58.269 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:58.269 INFO:tasks.workunit.client.0.smithi150.stdout: Include close in write timing 2024-05-10T04:05:58.269 INFO:tasks.workunit.client.0.smithi150.stdout: Include fsync in write timing 2024-05-10T04:05:58.269 INFO:tasks.workunit.client.0.smithi150.stdout: File size set to 1048576 kB 2024-05-10T04:05:58.269 INFO:tasks.workunit.client.0.smithi150.stdout: Record Size 1024 kB 2024-05-10T04:05:58.270 INFO:tasks.workunit.client.0.smithi150.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-05-10T04:05:58.270 INFO:tasks.workunit.client.0.smithi150.stdout: Output is in kBytes/sec 2024-05-10T04:05:58.270 INFO:tasks.workunit.client.0.smithi150.stdout: Time Resolution = 0.000001 seconds. 2024-05-10T04:05:58.270 INFO:tasks.workunit.client.0.smithi150.stdout: Processor cache size set to 1024 kBytes. 2024-05-10T04:05:58.270 INFO:tasks.workunit.client.0.smithi150.stdout: Processor cache line size set to 32 bytes. 2024-05-10T04:05:58.270 INFO:tasks.workunit.client.0.smithi150.stdout: File stride size set to 17 * record size. 2024-05-10T04:05:58.271 INFO:tasks.workunit.client.0.smithi150.stdout: Throughput test with 1 process 2024-05-10T04:05:58.271 INFO:tasks.workunit.client.0.smithi150.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-05-10T04:05:58.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:05:58 smithi150 ceph-mon[42477]: pgmap v331: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 70 MiB/s wr, 27 op/s 2024-05-10T04:05:58.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:05:58 smithi181 ceph-mon[46602]: pgmap v331: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 70 MiB/s wr, 27 op/s 2024-05-10T04:05:58.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:05:58 smithi161 ceph-mon[46918]: pgmap v331: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 70 MiB/s wr, 27 op/s 2024-05-10T04:05:59.602 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:05:59.602 INFO:tasks.workunit.client.0.smithi150.stdout: Children see throughput for 1 initial writers = 804098.62 kB/sec 2024-05-10T04:05:59.603 INFO:tasks.workunit.client.0.smithi150.stdout: Parent sees throughput for 1 initial writers = 803900.70 kB/sec 2024-05-10T04:05:59.603 INFO:tasks.workunit.client.0.smithi150.stdout: Min throughput per process = 804098.62 kB/sec 2024-05-10T04:05:59.603 INFO:tasks.workunit.client.0.smithi150.stdout: Max throughput per process = 804098.62 kB/sec 2024-05-10T04:05:59.603 INFO:tasks.workunit.client.0.smithi150.stdout: Avg throughput per process = 804098.62 kB/sec 2024-05-10T04:05:59.603 INFO:tasks.workunit.client.0.smithi150.stdout: Min xfer = 1048576.00 kB 2024-05-10T04:06:00.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:00 smithi150 ceph-mon[42477]: pgmap v332: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 41 MiB/s wr, 20 op/s 2024-05-10T04:06:00.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:00 smithi181 ceph-mon[46602]: pgmap v332: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 41 MiB/s wr, 20 op/s 2024-05-10T04:06:00.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:00 smithi161 ceph-mon[46918]: pgmap v332: 129 pgs: 129 active+clean; 1.0 GiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 41 MiB/s wr, 20 op/s 2024-05-10T04:06:01.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:01 smithi181 ceph-mon[46602]: pgmap v333: 129 pgs: 129 active+clean; 1.1 GiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 20 op/s 2024-05-10T04:06:01.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:01 smithi161 ceph-mon[46918]: pgmap v333: 129 pgs: 129 active+clean; 1.1 GiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 20 op/s 2024-05-10T04:06:01.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:01 smithi150 ceph-mon[42477]: pgmap v333: 129 pgs: 129 active+clean; 1.1 GiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 20 op/s 2024-05-10T04:06:03.349 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:06:03.350 INFO:tasks.workunit.client.0.smithi150.stdout: Children see throughput for 1 rewriters = 881035.69 kB/sec 2024-05-10T04:06:03.350 INFO:tasks.workunit.client.0.smithi150.stdout: Parent sees throughput for 1 rewriters = 880810.59 kB/sec 2024-05-10T04:06:03.350 INFO:tasks.workunit.client.0.smithi150.stdout: Min throughput per process = 881035.69 kB/sec 2024-05-10T04:06:03.350 INFO:tasks.workunit.client.0.smithi150.stdout: Max throughput per process = 881035.69 kB/sec 2024-05-10T04:06:03.350 INFO:tasks.workunit.client.0.smithi150.stdout: Avg throughput per process = 881035.69 kB/sec 2024-05-10T04:06:03.350 INFO:tasks.workunit.client.0.smithi150.stdout: Min xfer = 1048576.00 kB 2024-05-10T04:06:04.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:04 smithi150 ceph-mon[42477]: pgmap v334: 129 pgs: 129 active+clean; 1.1 GiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 64 MiB/s wr, 39 op/s 2024-05-10T04:06:04.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:04 smithi181 ceph-mon[46602]: pgmap v334: 129 pgs: 129 active+clean; 1.1 GiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 64 MiB/s wr, 39 op/s 2024-05-10T04:06:04.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:04 smithi161 ceph-mon[46918]: pgmap v334: 129 pgs: 129 active+clean; 1.1 GiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 64 MiB/s wr, 39 op/s 2024-05-10T04:06:06.212 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:06:06.212 INFO:tasks.workunit.client.0.smithi150.stdout: Children see throughput for 1 readers = 6817072.00 kB/sec 2024-05-10T04:06:06.213 INFO:tasks.workunit.client.0.smithi150.stdout: Parent sees throughput for 1 readers = 6803331.76 kB/sec 2024-05-10T04:06:06.213 INFO:tasks.workunit.client.0.smithi150.stdout: Min throughput per process = 6817072.00 kB/sec 2024-05-10T04:06:06.213 INFO:tasks.workunit.client.0.smithi150.stdout: Max throughput per process = 6817072.00 kB/sec 2024-05-10T04:06:06.213 INFO:tasks.workunit.client.0.smithi150.stdout: Avg throughput per process = 6817072.00 kB/sec 2024-05-10T04:06:06.213 INFO:tasks.workunit.client.0.smithi150.stdout: Min xfer = 1048576.00 kB 2024-05-10T04:06:06.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:06 smithi150 ceph-mon[42477]: pgmap v335: 129 pgs: 129 active+clean; 1.0 GiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 130 MiB/s wr, 67 op/s 2024-05-10T04:06:06.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:06 smithi181 ceph-mon[46602]: pgmap v335: 129 pgs: 129 active+clean; 1.0 GiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 130 MiB/s wr, 67 op/s 2024-05-10T04:06:06.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:06 smithi161 ceph-mon[46918]: pgmap v335: 129 pgs: 129 active+clean; 1.0 GiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 130 MiB/s wr, 67 op/s 2024-05-10T04:06:08.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:08 smithi150 ceph-mon[42477]: pgmap v336: 129 pgs: 129 active+clean; 1.0 GiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 162 MiB/s wr, 73 op/s 2024-05-10T04:06:08.471 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:06:08.472 INFO:tasks.workunit.client.0.smithi150.stdout: Children see throughput for 1 re-readers = 5594071.00 kB/sec 2024-05-10T04:06:08.472 INFO:tasks.workunit.client.0.smithi150.stdout: Parent sees throughput for 1 re-readers = 5585283.35 kB/sec 2024-05-10T04:06:08.472 INFO:tasks.workunit.client.0.smithi150.stdout: Min throughput per process = 5594071.00 kB/sec 2024-05-10T04:06:08.472 INFO:tasks.workunit.client.0.smithi150.stdout: Max throughput per process = 5594071.00 kB/sec 2024-05-10T04:06:08.472 INFO:tasks.workunit.client.0.smithi150.stdout: Avg throughput per process = 5594071.00 kB/sec 2024-05-10T04:06:08.472 INFO:tasks.workunit.client.0.smithi150.stdout: Min xfer = 1048576.00 kB 2024-05-10T04:06:08.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:08 smithi181 ceph-mon[46602]: pgmap v336: 129 pgs: 129 active+clean; 1.0 GiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 162 MiB/s wr, 73 op/s 2024-05-10T04:06:08.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:08 smithi161 ceph-mon[46918]: pgmap v336: 129 pgs: 129 active+clean; 1.0 GiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 162 MiB/s wr, 73 op/s 2024-05-10T04:06:10.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:10 smithi150 ceph-mon[42477]: pgmap v337: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 171 MiB/s wr, 75 op/s 2024-05-10T04:06:10.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:10 smithi181 ceph-mon[46602]: pgmap v337: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 171 MiB/s wr, 75 op/s 2024-05-10T04:06:10.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:10 smithi161 ceph-mon[46918]: pgmap v337: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 171 MiB/s wr, 75 op/s 2024-05-10T04:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.980+0000 7f9f8775f700 1 -- 172.21.15.150:0/1106545161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f80101be0 msgr2=0x7f9f80101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.980+0000 7f9f8775f700 1 --2- 172.21.15.150:0/1106545161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f80101be0 0x7f9f80101ff0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009990 tx=0x7f9f70009ca0 comp rx=0 tx=0).stop 2024-05-10T04:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.980+0000 7f9f8775f700 1 -- 172.21.15.150:0/1106545161 shutdown_connections 2024-05-10T04:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.980+0000 7f9f8775f700 1 --2- 172.21.15.150:0/1106545161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f80102580 0x7f9f80108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.980+0000 7f9f8775f700 1 --2- 172.21.15.150:0/1106545161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f80100ac0 0x7f9f80100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.980+0000 7f9f8775f700 1 --2- 172.21.15.150:0/1106545161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f80101be0 0x7f9f80101ff0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.980+0000 7f9f8775f700 1 -- 172.21.15.150:0/1106545161 >> 172.21.15.150:0/1106545161 conn(0x7f9f800fc0f0 msgr2=0x7f9f800fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.981+0000 7f9f8775f700 1 -- 172.21.15.150:0/1106545161 shutdown_connections 2024-05-10T04:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.981+0000 7f9f8775f700 1 -- 172.21.15.150:0/1106545161 wait complete. 2024-05-10T04:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.981+0000 7f9f8775f700 1 Processor -- start 2024-05-10T04:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.981+0000 7f9f8775f700 1 -- start start 2024-05-10T04:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.981+0000 7f9f8775f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f80100ac0 0x7f9f800690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.981+0000 7f9f8775f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f80101be0 0x7f9f8019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.981+0000 7f9f8775f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f80102580 0x7f9f8019f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.981+0000 7f9f8775f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8019fa20 con 0x7f9f80101be0 2024-05-10T04:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.981+0000 7f9f8775f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8019fb90 con 0x7f9f80100ac0 2024-05-10T04:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.981+0000 7f9f8775f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8019fd50 con 0x7f9f80102580 2024-05-10T04:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f84cfa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f80101be0 0x7f9f8019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f84cfa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f80101be0 0x7f9f8019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60162/0 (socket says 172.21.15.150:60162) 2024-05-10T04:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f84cfa700 1 -- 172.21.15.150:0/692102643 learned_addr learned my addr 172.21.15.150:0/692102643 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f854fb700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f80100ac0 0x7f9f800690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f85cfc700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f80102580 0x7f9f8019f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f84cfa700 1 -- 172.21.15.150:0/692102643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f80102580 msgr2=0x7f9f8019f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f84cfa700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f80102580 0x7f9f8019f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f84cfa700 1 -- 172.21.15.150:0/692102643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f80100ac0 msgr2=0x7f9f800690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f84cfa700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f80100ac0 0x7f9f800690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f84cfa700 1 -- 172.21.15.150:0/692102643 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f700095f0 con 0x7f9f80101be0 2024-05-10T04:06:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f84cfa700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f80101be0 0x7f9f8019cd70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000f690 tx=0x7f9f7000f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f767fc700 1 -- 172.21.15.150:0/692102643 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7001d070 con 0x7f9f80101be0 2024-05-10T04:06:10.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f767fc700 1 -- 172.21.15.150:0/692102643 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f70003df0 con 0x7f9f80101be0 2024-05-10T04:06:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f8775f700 1 -- 172.21.15.150:0/692102643 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8019ffd0 con 0x7f9f80101be0 2024-05-10T04:06:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f767fc700 1 -- 172.21.15.150:0/692102643 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f700055d0 con 0x7f9f80101be0 2024-05-10T04:06:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.982+0000 7f9f8775f700 1 -- 172.21.15.150:0/692102643 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f801a0390 con 0x7f9f80101be0 2024-05-10T04:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.985+0000 7f9f767fc700 1 -- 172.21.15.150:0/692102643 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f9f7002b440 con 0x7f9f80101be0 2024-05-10T04:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.986+0000 7f9f767fc700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9f6c0a1120 0x7f9f6c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.986+0000 7f9f854fb700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9f6c0a1120 0x7f9f6c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.986+0000 7f9f767fc700 1 -- 172.21.15.150:0/692102643 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9f700cdd60 con 0x7f9f80101be0 2024-05-10T04:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.986+0000 7f9f854fb700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9f6c0a1120 0x7f9f6c0a3630 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9f80101ae0 tx=0x7f9f7c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.986+0000 7f9f97aa3b80 1 Processor -- start 2024-05-10T04:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f97aa3b80 1 -- start start 2024-05-10T04:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f97aa3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb321180 0x5612eb3215b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f97aa3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb322050 0x5612eb322460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f97aa3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb3229a0 0x5612eb3292a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f97aa3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612eb2480f0 con 0x5612eb321180 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f97aa3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612eb245ab0 con 0x5612eb3229a0 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f97aa3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612eb2413a0 con 0x5612eb322050 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f854fb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb321180 0x5612eb3215b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f854fb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb321180 0x5612eb3215b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60174/0 (socket says 172.21.15.150:60174) 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f854fb700 1 -- 172.21.15.150:0/2697475112 learned_addr learned my addr 172.21.15.150:0/2697475112 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f84cfa700 1 --2- 172.21.15.150:0/2697475112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb322050 0x5612eb322460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f85cfc700 1 --2- 172.21.15.150:0/2697475112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb3229a0 0x5612eb3292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f854fb700 1 -- 172.21.15.150:0/2697475112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb322050 msgr2=0x5612eb322460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f854fb700 1 --2- 172.21.15.150:0/2697475112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb322050 0x5612eb322460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f854fb700 1 -- 172.21.15.150:0/2697475112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb3229a0 msgr2=0x5612eb3292a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f854fb700 1 --2- 172.21.15.150:0/2697475112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb3229a0 0x5612eb3292a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f854fb700 1 -- 172.21.15.150:0/2697475112 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612eb0c2350 con 0x5612eb321180 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.987+0000 7f9f854fb700 1 --2- 172.21.15.150:0/2697475112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb321180 0x5612eb3215b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c010f40 tx=0x7f9f7c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0412dd0cd6b7f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.988+0000 7f9f6b7fe700 1 -- 172.21.15.150:0/2697475112 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c011db0 con 0x5612eb321180 2024-05-10T04:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.988+0000 7f9f6b7fe700 1 -- 172.21.15.150:0/2697475112 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f7c015070 con 0x5612eb321180 2024-05-10T04:06:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.988+0000 7f9f6b7fe700 1 -- 172.21.15.150:0/2697475112 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c01c610 con 0x5612eb321180 2024-05-10T04:06:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.988+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2697475112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb321180 msgr2=0x5612eb3215b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.988+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2697475112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb321180 0x5612eb3215b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c010f40 tx=0x7f9f7c013000 comp rx=0 tx=0).stop 2024-05-10T04:06:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.989+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2697475112 shutdown_connections 2024-05-10T04:06:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.989+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2697475112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb3229a0 0x5612eb3292a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.989+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2697475112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb321180 0x5612eb3215b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.989+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2697475112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb322050 0x5612eb322460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.989+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2697475112 >> 172.21.15.150:0/2697475112 conn(0x5612eb283a00 msgr2=0x5612eb31ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.989+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2697475112 shutdown_connections 2024-05-10T04:06:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.989+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2697475112 wait complete. 2024-05-10T04:06:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.989+0000 7f9f97aa3b80 1 -- start start 2024-05-10T04:06:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.989+0000 7f9f97aa3b80 1 Processor -- start 2024-05-10T04:06:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f97aa3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb321180 0x5612eb32b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f97aa3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb322050 0x5612eb32b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f97aa3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 0x5612eb332650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f97aa3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612eb1dd500 con 0x5612eb3229a0 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f97aa3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612eb1dd1a0 con 0x5612eb322050 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f97aa3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612eb0c4640 con 0x5612eb321180 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f97aa3b80 10 client.0 _ll_register_callbacks cb 0x5612eb1e6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f85cfc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 0x5612eb332650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f85cfc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 0x5612eb332650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60180/0 (socket says 172.21.15.150:60180) 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f85cfc700 1 -- 172.21.15.150:0/3970756056 learned_addr learned my addr 172.21.15.150:0/3970756056 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f854fb700 1 --2- 172.21.15.150:0/3970756056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb321180 0x5612eb32b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f84cfa700 1 --2- 172.21.15.150:0/3970756056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb322050 0x5612eb32b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f85cfc700 1 -- 172.21.15.150:0/3970756056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb321180 msgr2=0x5612eb32b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f85cfc700 1 --2- 172.21.15.150:0/3970756056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb321180 0x5612eb32b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f85cfc700 1 -- 172.21.15.150:0/3970756056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb322050 msgr2=0x5612eb32b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f85cfc700 1 --2- 172.21.15.150:0/3970756056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb322050 0x5612eb32b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f85cfc700 1 -- 172.21.15.150:0/3970756056 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f7c010c20 con 0x5612eb3229a0 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f85cfc700 1 --2- 172.21.15.150:0/3970756056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 0x5612eb332650 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9f78007ab0 tx=0x7f9f78007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22af69e149e364ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.990+0000 7f9f68ff9700 1 -- 172.21.15.150:0/3970756056 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78012b40 con 0x5612eb3229a0 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.991+0000 7f9f68ff9700 1 -- 172.21.15.150:0/3970756056 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9f78012ca0 con 0x5612eb3229a0 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.991+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/3970756056 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612eb19a210 con 0x5612eb3229a0 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.991+0000 7f9f68ff9700 1 -- 172.21.15.150:0/3970756056 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f78012e60 con 0x5612eb3229a0 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.991+0000 7f9f97aa3b80 10 client.16134 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.991+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/3970756056 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5612eb06eb00 con 0x5612eb3229a0 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.991+0000 7f9f68ff9700 1 -- 172.21.15.150:0/3970756056 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612eb19a210 con 0x5612eb3229a0 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.991+0000 7f9f68ff9700 1 -- 172.21.15.150:0/3970756056 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f78014040 con 0x5612eb3229a0 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.991+0000 7f9f97aa3b80 10 client.16134 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.991+0000 7f9f97aa3b80 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.991+0000 7f9f97aa3b80 20 client.16134 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.991+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/3970756056 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5612eb336eb0 0x5612eb339300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.991+0000 7f9f97aa3b80 4 client.16134 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.991+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/3970756056 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612eb2f0720 con 0x5612eb336eb0 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.992+0000 7f9f854fb700 1 --2- 172.21.15.150:0/3970756056 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5612eb336eb0 0x5612eb339300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.992+0000 7f9f68ff9700 10 client.16134 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.992+0000 7f9f854fb700 1 --2- 172.21.15.150:0/3970756056 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5612eb336eb0 0x5612eb339300 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c752e7050a3aeec server_cookie=923401b45fea3d30 in_seq=0 out_seq=0 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f68ff9700 1 -- 172.21.15.150:0/3970756056 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5612eb2f0720 con 0x5612eb336eb0 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f68ff9700 10 client.16134 handle_command_reply: tid=0 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f97aa3b80 1 client.16134 shutdown 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/3970756056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 msgr2=0x5612eb332650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/3970756056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 0x5612eb332650 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9f78007ab0 tx=0x7f9f78007dc0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/3970756056 shutdown_connections 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/3970756056 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5612eb336eb0 0x5612eb339300 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/3970756056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 0x5612eb332650 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/3970756056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb321180 0x5612eb32b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f68ff9700 0 client.16134 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/3970756056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb322050 0x5612eb32b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/3970756056 >> 172.21.15.150:0/3970756056 conn(0x5612eb283a00 msgr2=0x5612eb31ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/3970756056 shutdown_connections 2024-05-10T04:06:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/3970756056 wait complete. 2024-05-10T04:06:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:10.993+0000 7f9f97aa3b80 20 client.16134 trim_cache size 0 max 16384 2024-05-10T04:06:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.008+0000 7f9f97aa3b80 1 Processor -- start 2024-05-10T04:06:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.008+0000 7f9f97aa3b80 1 -- start start 2024-05-10T04:06:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.008+0000 7f9f97aa3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb321180 0x5612eb3304f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.008+0000 7f9f97aa3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb322050 0x5612eb330a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.008+0000 7f9f97aa3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 0x5612eb32b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.008+0000 7f9f97aa3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612eb32c250 con 0x5612eb3229a0 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.008+0000 7f9f97aa3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612eb336c30 con 0x5612eb321180 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.008+0000 7f9f97aa3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612eb3264d0 con 0x5612eb322050 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.008+0000 7f9f85cfc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 0x5612eb32b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.009+0000 7f9f85cfc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 0x5612eb32b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60196/0 (socket says 172.21.15.150:60196) 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.009+0000 7f9f85cfc700 1 -- 172.21.15.150:0/2162337416 learned_addr learned my addr 172.21.15.150:0/2162337416 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.009+0000 7f9f854fb700 1 --2- 172.21.15.150:0/2162337416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb321180 0x5612eb3304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.009+0000 7f9f84cfa700 1 --2- 172.21.15.150:0/2162337416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb322050 0x5612eb330a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.009+0000 7f9f85cfc700 1 -- 172.21.15.150:0/2162337416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb322050 msgr2=0x5612eb330a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.009+0000 7f9f85cfc700 1 --2- 172.21.15.150:0/2162337416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb322050 0x5612eb330a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.009+0000 7f9f85cfc700 1 -- 172.21.15.150:0/2162337416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb321180 msgr2=0x5612eb3304f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.009+0000 7f9f85cfc700 1 --2- 172.21.15.150:0/2162337416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb321180 0x5612eb3304f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.009+0000 7f9f85cfc700 1 -- 172.21.15.150:0/2162337416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612eb207500 con 0x5612eb3229a0 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.009+0000 7f9f85cfc700 1 --2- 172.21.15.150:0/2162337416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 0x5612eb32b1b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009aa0 tx=0x7f9f780125a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41967360f2b3d89 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.009+0000 7f9f6bfff700 1 -- 172.21.15.150:0/2162337416 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78015d10 con 0x5612eb3229a0 2024-05-10T04:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.009+0000 7f9f6bfff700 1 -- 172.21.15.150:0/2162337416 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f7800f040 con 0x5612eb3229a0 2024-05-10T04:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.009+0000 7f9f6bfff700 1 -- 172.21.15.150:0/2162337416 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78012ea0 con 0x5612eb3229a0 2024-05-10T04:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.010+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2162337416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 msgr2=0x5612eb32b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.010+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2162337416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 0x5612eb32b1b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009aa0 tx=0x7f9f780125a0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.010+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2162337416 shutdown_connections 2024-05-10T04:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.010+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2162337416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb3229a0 0x5612eb32b1b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.010+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2162337416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb321180 0x5612eb3304f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.010+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2162337416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb322050 0x5612eb330a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.010+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2162337416 >> 172.21.15.150:0/2162337416 conn(0x5612eb283a00 msgr2=0x5612eb32a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.010+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2162337416 shutdown_connections 2024-05-10T04:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.010+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2162337416 wait complete. 2024-05-10T04:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.010+0000 7f9f97aa3b80 1 -- start start 2024-05-10T04:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.010+0000 7f9f97aa3b80 1 Processor -- start 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f97aa3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb321180 0x5612eb387350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f97aa3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb322050 0x5612eb3898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f97aa3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb3229a0 0x5612eb38bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f97aa3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612eb38c330 con 0x5612eb322050 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f97aa3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612eb38c470 con 0x5612eb321180 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f97aa3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612eb38c5b0 con 0x5612eb3229a0 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f97aa3b80 10 client.0 _ll_register_callbacks cb 0x5612eb1e6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f84cfa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb322050 0x5612eb3898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f854fb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb321180 0x5612eb387350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f84cfa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb322050 0x5612eb3898a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60200/0 (socket says 172.21.15.150:60200) 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f84cfa700 1 -- 172.21.15.150:0/2957994804 learned_addr learned my addr 172.21.15.150:0/2957994804 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f85cfc700 1 --2- 172.21.15.150:0/2957994804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb3229a0 0x5612eb38bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f854fb700 1 -- 172.21.15.150:0/2957994804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb3229a0 msgr2=0x5612eb38bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f854fb700 1 --2- 172.21.15.150:0/2957994804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb3229a0 0x5612eb38bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f854fb700 1 -- 172.21.15.150:0/2957994804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb322050 msgr2=0x5612eb3898a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f854fb700 1 --2- 172.21.15.150:0/2957994804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb322050 0x5612eb3898a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f854fb700 1 -- 172.21.15.150:0/2957994804 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612eb06eb00 con 0x5612eb321180 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f84cfa700 1 --2- 172.21.15.150:0/2957994804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb322050 0x5612eb3898a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f854fb700 1 --2- 172.21.15.150:0/2957994804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb321180 0x5612eb387350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c011a00 tx=0x7f9f7c005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459bda6cdfd3f6dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.011+0000 7f9f68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f68ff9700 1 -- 172.21.15.150:0/2957994804 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c02c040 con 0x5612eb321180 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f68ff9700 1 -- 172.21.15.150:0/2957994804 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9f7c017070 con 0x5612eb321180 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2957994804 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f7c015070 con 0x5612eb321180 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f68ff9700 1 -- 172.21.15.150:0/2957994804 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f7c033620 con 0x5612eb321180 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f97aa3b80 10 client.25814 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2957994804 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5612eb0c6c70 con 0x5612eb321180 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f68ff9700 1 -- 172.21.15.150:0/2957994804 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f7c015070 con 0x5612eb321180 2024-05-10T04:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f68ff9700 1 -- 172.21.15.150:0/2957994804 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f7c03c440 con 0x5612eb321180 2024-05-10T04:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f97aa3b80 10 client.25814 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f97aa3b80 10 client.25814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f97aa3b80 20 client.25814 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2957994804 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5612eb38de30 0x5612eb390250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f97aa3b80 4 client.25814 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2957994804 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5612eb239e60 con 0x5612eb38de30 2024-05-10T04:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.012+0000 7f9f84cfa700 1 --2- 172.21.15.150:0/2957994804 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5612eb38de30 0x5612eb390250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.013+0000 7f9f68ff9700 10 client.25814 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.013+0000 7f9f84cfa700 1 --2- 172.21.15.150:0/2957994804 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5612eb38de30 0x5612eb390250 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2797b610155259 server_cookie=c83de2048c8f00b0 in_seq=0 out_seq=0 2024-05-10T04:06:11.024 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f68ff9700 1 -- 172.21.15.150:0/2957994804 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5612eb239e60 con 0x5612eb38de30 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f68ff9700 10 client.25814 handle_command_reply: tid=0 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f97aa3b80 1 client.25814 shutdown 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2957994804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb321180 msgr2=0x5612eb387350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2957994804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb321180 0x5612eb387350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c011a00 tx=0x7f9f7c005dd0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2957994804 shutdown_connections 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2957994804 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5612eb38de30 0x5612eb390250 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2957994804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612eb3229a0 0x5612eb38bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2957994804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612eb321180 0x5612eb387350 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f68ff9700 0 client.25814 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f97aa3b80 1 --2- 172.21.15.150:0/2957994804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612eb322050 0x5612eb3898a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2957994804 >> 172.21.15.150:0/2957994804 conn(0x5612eb283a00 msgr2=0x5612eb3297e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2957994804 shutdown_connections 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f97aa3b80 1 -- 172.21.15.150:0/2957994804 wait complete. 2024-05-10T04:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.023+0000 7f9f97aa3b80 20 client.25814 trim_cache size 0 max 16384 2024-05-10T04:06:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:06:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:06:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:06:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:06:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.024+0000 7f9f8775f700 1 -- 172.21.15.150:0/692102643 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9f6c0a1120 msgr2=0x7f9f6c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.024+0000 7f9f8775f700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9f6c0a1120 0x7f9f6c0a3630 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9f80101ae0 tx=0x7f9f7c00b410 comp rx=0 tx=0).stop 2024-05-10T04:06:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.024+0000 7f9f8775f700 1 -- 172.21.15.150:0/692102643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f80101be0 msgr2=0x7f9f8019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.024+0000 7f9f8775f700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f80101be0 0x7f9f8019cd70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000f690 tx=0x7f9f7000f6c0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.024+0000 7f9f8775f700 1 -- 172.21.15.150:0/692102643 shutdown_connections 2024-05-10T04:06:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.024+0000 7f9f8775f700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f9f6c0a1120 0x7f9f6c0a3630 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.024+0000 7f9f8775f700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f80102580 0x7f9f8019f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.024+0000 7f9f8775f700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f80100ac0 0x7f9f800690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.024+0000 7f9f8775f700 1 --2- 172.21.15.150:0/692102643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f80101be0 0x7f9f8019cd70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.024+0000 7f9f8775f700 1 -- 172.21.15.150:0/692102643 >> 172.21.15.150:0/692102643 conn(0x7f9f800fc0f0 msgr2=0x7f9f80104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.024+0000 7f9f8775f700 1 -- 172.21.15.150:0/692102643 shutdown_connections 2024-05-10T04:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:11.024+0000 7f9f8775f700 1 -- 172.21.15.150:0/692102643 wait complete. 2024-05-10T04:06:11.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:11 smithi150 ceph-mon[42477]: pgmap v338: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 168 MiB/s wr, 74 op/s 2024-05-10T04:06:11.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:11 smithi181 ceph-mon[46602]: pgmap v338: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 168 MiB/s wr, 74 op/s 2024-05-10T04:06:11.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:11 smithi161 ceph-mon[46918]: pgmap v338: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 168 MiB/s wr, 74 op/s 2024-05-10T04:06:14.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:14 smithi150 ceph-mon[42477]: pgmap v339: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 151 MiB/s wr, 64 op/s 2024-05-10T04:06:14.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:14 smithi181 ceph-mon[46602]: pgmap v339: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 151 MiB/s wr, 64 op/s 2024-05-10T04:06:14.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:14 smithi161 ceph-mon[46918]: pgmap v339: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 151 MiB/s wr, 64 op/s 2024-05-10T04:06:16.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:16 smithi150 ceph-mon[42477]: pgmap v340: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 107 MiB/s wr, 47 op/s 2024-05-10T04:06:16.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:16 smithi181 ceph-mon[46602]: pgmap v340: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 107 MiB/s wr, 47 op/s 2024-05-10T04:06:16.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:16 smithi161 ceph-mon[46918]: pgmap v340: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 107 MiB/s wr, 47 op/s 2024-05-10T04:06:18.290 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:06:18.291 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:06:18.291 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:06:18.291 INFO:tasks.workunit.client.0.smithi150.stdout:iozone test complete. 2024-05-10T04:06:18.291 INFO:tasks.workunit.client.0.smithi150.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-05-10T04:06:18.324 INFO:tasks.workunit.client.0.smithi150.stdout: Iozone: Performance Test of File I/O 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: Version $Revision: 3.489 $ 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: Compiled for 64 bit mode. 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: Build: linux-AMD64 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: Alexey Skidanov, Sudhir Kumar. 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:06:18.325 INFO:tasks.workunit.client.0.smithi150.stdout: Run began: Fri May 10 04:06:18 2024 2024-05-10T04:06:18.326 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:06:18.326 INFO:tasks.workunit.client.0.smithi150.stdout: Include close in write timing 2024-05-10T04:06:18.326 INFO:tasks.workunit.client.0.smithi150.stdout: Include fsync in write timing 2024-05-10T04:06:18.326 INFO:tasks.workunit.client.0.smithi150.stdout: File size set to 10485760 kB 2024-05-10T04:06:18.326 INFO:tasks.workunit.client.0.smithi150.stdout: Record Size 1024 kB 2024-05-10T04:06:18.326 INFO:tasks.workunit.client.0.smithi150.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-05-10T04:06:18.326 INFO:tasks.workunit.client.0.smithi150.stdout: Output is in kBytes/sec 2024-05-10T04:06:18.326 INFO:tasks.workunit.client.0.smithi150.stdout: Time Resolution = 0.000001 seconds. 2024-05-10T04:06:18.326 INFO:tasks.workunit.client.0.smithi150.stdout: Processor cache size set to 1024 kBytes. 2024-05-10T04:06:18.326 INFO:tasks.workunit.client.0.smithi150.stdout: Processor cache line size set to 32 bytes. 2024-05-10T04:06:18.326 INFO:tasks.workunit.client.0.smithi150.stdout: File stride size set to 17 * record size. 2024-05-10T04:06:18.326 INFO:tasks.workunit.client.0.smithi150.stdout: Throughput test with 1 process 2024-05-10T04:06:18.327 INFO:tasks.workunit.client.0.smithi150.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-05-10T04:06:18.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:18 smithi150 ceph-mon[42477]: pgmap v341: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 41 MiB/s wr, 19 op/s 2024-05-10T04:06:18.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:18 smithi181 ceph-mon[46602]: pgmap v341: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 41 MiB/s wr, 19 op/s 2024-05-10T04:06:18.491 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:18 smithi161 ceph-mon[46918]: pgmap v341: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 41 MiB/s wr, 19 op/s 2024-05-10T04:06:19.732 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:19 smithi150 ceph-mon[42477]: pgmap v342: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8.3 MiB/s wr, 11 op/s 2024-05-10T04:06:19.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:19 smithi181 ceph-mon[46602]: pgmap v342: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8.3 MiB/s wr, 11 op/s 2024-05-10T04:06:19.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:19 smithi161 ceph-mon[46918]: pgmap v342: 129 pgs: 129 active+clean; 1.0 GiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8.3 MiB/s wr, 11 op/s 2024-05-10T04:06:20.669 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:06:21.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.362+0000 7f2fc8228700 1 -- 172.21.15.150:0/3255013676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fc0109720 msgr2=0x7f2fc0109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:21.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.362+0000 7f2fc8228700 1 --2- 172.21.15.150:0/3255013676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fc0109720 0x7f2fc0109b30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000b9a0 tx=0x7f2fb0009260 comp rx=0 tx=0).stop 2024-05-10T04:06:21.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.364+0000 7f2fc8228700 1 -- 172.21.15.150:0/3255013676 shutdown_connections 2024-05-10T04:06:21.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.364+0000 7f2fc8228700 1 --2- 172.21.15.150:0/3255013676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fc010a0c0 0x7f2fc0110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.364+0000 7f2fc8228700 1 --2- 172.21.15.150:0/3255013676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fc0108630 0x7f2fc0108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.364+0000 7f2fc8228700 1 --2- 172.21.15.150:0/3255013676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fc0109720 0x7f2fc0109b30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.364+0000 7f2fc8228700 1 -- 172.21.15.150:0/3255013676 >> 172.21.15.150:0/3255013676 conn(0x7f2fc006ee40 msgr2=0x7f2fc00712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:21.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.368+0000 7f2fc8228700 1 -- 172.21.15.150:0/3255013676 shutdown_connections 2024-05-10T04:06:21.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.379+0000 7f2fc8228700 1 -- 172.21.15.150:0/3255013676 wait complete. 2024-05-10T04:06:21.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.380+0000 7f2fc8228700 1 Processor -- start 2024-05-10T04:06:21.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.380+0000 7f2fc8228700 1 -- start start 2024-05-10T04:06:21.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.380+0000 7f2fc8228700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fc0108630 0x7f2fc019e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.380+0000 7f2fc8228700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fc0109720 0x7f2fc019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.380+0000 7f2fc8228700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fc010a0c0 0x7f2fc01b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.380+0000 7f2fc8228700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc01b7040 con 0x7f2fc010a0c0 2024-05-10T04:06:21.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.380+0000 7f2fc8228700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc01b7180 con 0x7f2fc0109720 2024-05-10T04:06:21.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.381+0000 7f2fc8228700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc01b7340 con 0x7f2fc0108630 2024-05-10T04:06:21.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.382+0000 7f2fc57c3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fc0109720 0x7f2fc019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.382+0000 7f2fc57c3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fc0109720 0x7f2fc019ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:37924/0 (socket says 172.21.15.150:37924) 2024-05-10T04:06:21.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.382+0000 7f2fc57c3700 1 -- 172.21.15.150:0/2295959953 learned_addr learned my addr 172.21.15.150:0/2295959953 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:21.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.387+0000 7f2fc67c5700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fc010a0c0 0x7f2fc01b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.387+0000 7f2fc5fc4700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fc0108630 0x7f2fc019e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.388+0000 7f2fc57c3700 1 -- 172.21.15.150:0/2295959953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fc0108630 msgr2=0x7f2fc019e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.388+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fc0108630 0x7f2fc019e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.388+0000 7f2fc57c3700 1 -- 172.21.15.150:0/2295959953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fc010a0c0 msgr2=0x7f2fc01b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.388+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fc010a0c0 0x7f2fc01b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.388+0000 7f2fc57c3700 1 -- 172.21.15.150:0/2295959953 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb000b5a0 con 0x7f2fc0109720 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.389+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fc0109720 0x7f2fc019ea80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0013b40 tx=0x7f2fb0009810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.390+0000 7f2fb6ffd700 1 -- 172.21.15.150:0/2295959953 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00052c0 con 0x7f2fc0109720 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.390+0000 7f2fb6ffd700 1 -- 172.21.15.150:0/2295959953 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fb0005420 con 0x7f2fc0109720 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.390+0000 7f2fb6ffd700 1 -- 172.21.15.150:0/2295959953 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb001e680 con 0x7f2fc0109720 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.390+0000 7f2fc8228700 1 -- 172.21.15.150:0/2295959953 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc01b75c0 con 0x7f2fc0109720 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.390+0000 7f2fc8228700 1 -- 172.21.15.150:0/2295959953 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc01b7a20 con 0x7f2fc0109720 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.394+0000 7f2fb6ffd700 1 -- 172.21.15.150:0/2295959953 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2fb0005590 con 0x7f2fc0109720 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.394+0000 7f2fb6ffd700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2fac0a1290 0x7f2fac0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.395+0000 7f2fb6ffd700 1 -- 172.21.15.150:0/2295959953 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2fb00ce860 con 0x7f2fc0109720 2024-05-10T04:06:21.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.413+0000 7f2fd856cb80 1 Processor -- start 2024-05-10T04:06:21.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.425+0000 7f2fd856cb80 1 -- start start 2024-05-10T04:06:21.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.426+0000 7f2fd856cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 0x55e43dcccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.426+0000 7f2fd856cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccd570 0x55e43dccd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.426+0000 7f2fd856cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccdec0 0x55e43dcd47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.426+0000 7f2fd856cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43dbf3150 con 0x55e43dccd570 2024-05-10T04:06:21.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.426+0000 7f2fd856cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43dbf0b10 con 0x55e43dccc6a0 2024-05-10T04:06:21.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.426+0000 7f2fd856cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43dbec400 con 0x55e43dccdec0 2024-05-10T04:06:21.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.427+0000 7f2fc67c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccdec0 0x55e43dcd47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.429+0000 7f2fc57c3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccd570 0x55e43dccd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.429+0000 7f2fc57c3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccd570 0x55e43dccd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59172/0 (socket says 172.21.15.150:59172) 2024-05-10T04:06:21.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.429+0000 7f2fc57c3700 1 -- 172.21.15.150:0/767074004 learned_addr learned my addr 172.21.15.150:0/767074004 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:21.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.433+0000 7f2fc57c3700 1 -- 172.21.15.150:0/767074004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccdec0 msgr2=0x55e43dcd47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:21.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.433+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/767074004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccdec0 0x55e43dcd47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.433+0000 7f2fc57c3700 1 -- 172.21.15.150:0/767074004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 msgr2=0x55e43dcccaf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:06:21.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.433+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/767074004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 0x55e43dcccaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.433+0000 7f2fc57c3700 1 -- 172.21.15.150:0/767074004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e43da6d3b0 con 0x55e43dccd570 2024-05-10T04:06:21.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.434+0000 7f2fc5fc4700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2fac0a1290 0x7f2fac0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.440+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/767074004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccd570 0x55e43dccd980 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2fb009b730 tx=0x7f2fb0097df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8954d35b1aaf01b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:21.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.444+0000 7f2fabfff700 1 -- 172.21.15.150:0/767074004 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb009c130 con 0x55e43dccd570 2024-05-10T04:06:21.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.444+0000 7f2fabfff700 1 -- 172.21.15.150:0/767074004 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fb009ba30 con 0x55e43dccd570 2024-05-10T04:06:21.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.444+0000 7f2fabfff700 1 -- 172.21.15.150:0/767074004 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb009bc90 con 0x55e43dccd570 2024-05-10T04:06:21.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.446+0000 7f2fd856cb80 1 -- 172.21.15.150:0/767074004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccd570 msgr2=0x55e43dccd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:21.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.446+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/767074004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccd570 0x55e43dccd980 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2fb009b730 tx=0x7f2fb0097df0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.446+0000 7f2fd856cb80 1 -- 172.21.15.150:0/767074004 shutdown_connections 2024-05-10T04:06:21.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.446+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/767074004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccdec0 0x55e43dcd47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.446+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/767074004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 0x55e43dcccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.446+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/767074004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccd570 0x55e43dccd980 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.446+0000 7f2fd856cb80 1 -- 172.21.15.150:0/767074004 >> 172.21.15.150:0/767074004 conn(0x55e43dc2ea60 msgr2=0x55e43dcc6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:21.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.456+0000 7f2fc5fc4700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2fac0a1290 0x7f2fac0a37a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55e43dccc380 tx=0x7f2fbc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:21.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.465+0000 7f2fd856cb80 1 -- 172.21.15.150:0/767074004 shutdown_connections 2024-05-10T04:06:21.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.480+0000 7f2fd856cb80 1 -- 172.21.15.150:0/767074004 wait complete. 2024-05-10T04:06:21.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.481+0000 7f2fd856cb80 1 -- start start 2024-05-10T04:06:21.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.481+0000 7f2fd856cb80 1 Processor -- start 2024-05-10T04:06:21.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.483+0000 7f2fd856cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccc6a0 0x55e43dcd6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.483+0000 7f2fd856cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccd570 0x55e43dcd69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.483+0000 7f2fd856cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccdec0 0x55e43dcdd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.483+0000 7f2fd856cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43db88560 con 0x55e43dccd570 2024-05-10T04:06:21.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.483+0000 7f2fd856cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43db88200 con 0x55e43dccdec0 2024-05-10T04:06:21.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.483+0000 7f2fd856cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43da6f6a0 con 0x55e43dccc6a0 2024-05-10T04:06:21.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.483+0000 7f2fd856cb80 10 client.0 _ll_register_callbacks cb 0x55e43db91b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:21.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.485+0000 7f2fc67c5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccdec0 0x55e43dcdd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.485+0000 7f2fc67c5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccdec0 0x55e43dcdd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:37948/0 (socket says 172.21.15.150:37948) 2024-05-10T04:06:21.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.485+0000 7f2fc67c5700 1 -- 172.21.15.150:0/776056993 learned_addr learned my addr 172.21.15.150:0/776056993 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.486+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/776056993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccd570 0x55e43dcd69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.486+0000 7f2fc57c3700 1 -- 172.21.15.150:0/776056993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccc6a0 msgr2=0x55e43dcd6480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:06:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.487+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/776056993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccc6a0 0x55e43dcd6480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.487+0000 7f2fc57c3700 1 -- 172.21.15.150:0/776056993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccdec0 msgr2=0x55e43dcdd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.487+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/776056993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccdec0 0x55e43dcdd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.487+0000 7f2fc57c3700 1 -- 172.21.15.150:0/776056993 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc01b7a20 con 0x55e43dccd570 2024-05-10T04:06:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.487+0000 7f2fc67c5700 1 --2- 172.21.15.150:0/776056993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccdec0 0x55e43dcdd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:06:21.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.487+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/776056993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccd570 0x55e43dcd69c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2fb009a020 tx=0x7f2fb00058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5b44c0509112f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:21.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.487+0000 7f2fa97fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:21.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.492+0000 7f2fa97fa700 1 -- 172.21.15.150:0/776056993 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00b8070 con 0x55e43dccd570 2024-05-10T04:06:21.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.492+0000 7f2fa97fa700 1 -- 172.21.15.150:0/776056993 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2fb00a3e20 con 0x55e43dccd570 2024-05-10T04:06:21.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.492+0000 7f2fd856cb80 1 -- 172.21.15.150:0/776056993 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e43db45270 con 0x55e43dccd570 2024-05-10T04:06:21.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.498+0000 7f2fa97fa700 1 -- 172.21.15.150:0/776056993 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fb009f860 con 0x55e43dccd570 2024-05-10T04:06:21.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.498+0000 7f2fd856cb80 10 client.16152 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:21.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.498+0000 7f2fd856cb80 1 -- 172.21.15.150:0/776056993 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e43da19b60 con 0x55e43dccd570 2024-05-10T04:06:21.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.514+0000 7f2fa97fa700 1 -- 172.21.15.150:0/776056993 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e43db45270 con 0x55e43dccd570 2024-05-10T04:06:21.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.514+0000 7f2fa97fa700 1 -- 172.21.15.150:0/776056993 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2fb001a030 con 0x55e43dccd570 2024-05-10T04:06:21.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.516+0000 7f2fd856cb80 10 client.16152 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:21.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.516+0000 7f2fd856cb80 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:21.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.516+0000 7f2fd856cb80 20 client.16152 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:21.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.516+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/776056993 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55e43dce1b40 0x55e43dce3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.516+0000 7f2fd856cb80 4 client.16152 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:21.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.516+0000 7f2fd856cb80 1 -- 172.21.15.150:0/776056993 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e43dc9b780 con 0x55e43dce1b40 2024-05-10T04:06:21.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.517+0000 7f2fc5fc4700 1 --2- 172.21.15.150:0/776056993 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55e43dce1b40 0x55e43dce3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.529+0000 7f2fc5fc4700 1 --2- 172.21.15.150:0/776056993 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55e43dce1b40 0x55e43dce3f90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2de0d75f5138ab4 server_cookie=cc6b3c6ef6efa742 in_seq=0 out_seq=0 2024-05-10T04:06:21.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.530+0000 7f2fa97fa700 10 client.16152 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:21.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.549+0000 7f2fa97fa700 1 -- 172.21.15.150:0/776056993 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e43dc9b780 con 0x55e43dce1b40 2024-05-10T04:06:21.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.549+0000 7f2fa97fa700 10 client.16152 handle_command_reply: tid=0 2024-05-10T04:06:21.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.555+0000 7f2fd856cb80 1 client.16152 shutdown 2024-05-10T04:06:21.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.587+0000 7f2fd856cb80 1 -- 172.21.15.150:0/776056993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccd570 msgr2=0x55e43dcd69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:21.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.587+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/776056993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccd570 0x55e43dcd69c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2fb009a020 tx=0x7f2fb00058c0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.599+0000 7f2fd856cb80 1 -- 172.21.15.150:0/776056993 shutdown_connections 2024-05-10T04:06:21.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.599+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/776056993 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55e43dce1b40 0x55e43dce3f90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.599+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/776056993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccdec0 0x55e43dcdd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.599+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/776056993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccc6a0 0x55e43dcd6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.599+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/776056993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccd570 0x55e43dcd69c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.599+0000 7f2fd856cb80 1 -- 172.21.15.150:0/776056993 >> 172.21.15.150:0/776056993 conn(0x55e43dc2ea60 msgr2=0x55e43dcc6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:21.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.608+0000 7f2fa97fa700 0 client.16152 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:21.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.618+0000 7f2fd856cb80 1 -- 172.21.15.150:0/776056993 shutdown_connections 2024-05-10T04:06:21.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.618+0000 7f2fd856cb80 1 -- 172.21.15.150:0/776056993 wait complete. 2024-05-10T04:06:21.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.618+0000 7f2fd856cb80 20 client.16152 trim_cache size 0 max 16384 2024-05-10T04:06:21.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.653+0000 7f2fd856cb80 1 Processor -- start 2024-05-10T04:06:21.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.654+0000 7f2fd856cb80 1 -- start start 2024-05-10T04:06:21.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.654+0000 7f2fd856cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 0x55e43dce3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.654+0000 7f2fd856cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccd570 0x55e43dce3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.654+0000 7f2fd856cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccdec0 0x55e43dcdd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.654+0000 7f2fd856cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43db45270 con 0x55e43dccdec0 2024-05-10T04:06:21.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.654+0000 7f2fd856cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43dcc1810 con 0x55e43dccc6a0 2024-05-10T04:06:21.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.654+0000 7f2fd856cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43dcc3d30 con 0x55e43dccd570 2024-05-10T04:06:21.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.660+0000 7f2fc5fc4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 0x55e43dce3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.660+0000 7f2fc5fc4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 0x55e43dce3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:37956/0 (socket says 172.21.15.150:37956) 2024-05-10T04:06:21.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.660+0000 7f2fc5fc4700 1 -- 172.21.15.150:0/3658972222 learned_addr learned my addr 172.21.15.150:0/3658972222 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:21.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.666+0000 7f2fc67c5700 1 --2- 172.21.15.150:0/3658972222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccdec0 0x55e43dcdd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.666+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/3658972222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccd570 0x55e43dce3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.672+0000 7f2fc57c3700 1 -- 172.21.15.150:0/3658972222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 msgr2=0x55e43dce3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:21.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.672+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/3658972222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 0x55e43dce3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.672+0000 7f2fc57c3700 1 -- 172.21.15.150:0/3658972222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccdec0 msgr2=0x55e43dcdd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:21.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.672+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/3658972222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccdec0 0x55e43dcdd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.672+0000 7f2fc57c3700 1 -- 172.21.15.150:0/3658972222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e43dbb2560 con 0x55e43dccd570 2024-05-10T04:06:21.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.693+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/3658972222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccd570 0x55e43dce3f40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2fb009b500 tx=0x7f2fb0097df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16db0081ac9104ad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:21.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.707+0000 7f2fb4ff9700 1 -- 172.21.15.150:0/3658972222 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00b8030 con 0x55e43dccd570 2024-05-10T04:06:21.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.707+0000 7f2fb4ff9700 1 -- 172.21.15.150:0/3658972222 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fb0099dd0 con 0x55e43dccd570 2024-05-10T04:06:21.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.707+0000 7f2fb4ff9700 1 -- 172.21.15.150:0/3658972222 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb009f860 con 0x55e43dccd570 2024-05-10T04:06:21.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.713+0000 7f2fd856cb80 1 -- 172.21.15.150:0/3658972222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccd570 msgr2=0x55e43dce3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:21.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.713+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/3658972222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccd570 0x55e43dce3f40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2fb009b500 tx=0x7f2fb0097df0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.721+0000 7f2fd856cb80 1 -- 172.21.15.150:0/3658972222 shutdown_connections 2024-05-10T04:06:21.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.721+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/3658972222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccdec0 0x55e43dcdd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.721+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/3658972222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 0x55e43dce3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.721+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/3658972222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccd570 0x55e43dce3f40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.721+0000 7f2fd856cb80 1 -- 172.21.15.150:0/3658972222 >> 172.21.15.150:0/3658972222 conn(0x55e43dc2ea60 msgr2=0x55e43dcd4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:21.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.741+0000 7f2fd856cb80 1 -- 172.21.15.150:0/3658972222 shutdown_connections 2024-05-10T04:06:21.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.746+0000 7f2fd856cb80 1 -- 172.21.15.150:0/3658972222 wait complete. 2024-05-10T04:06:21.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.747+0000 7f2fd856cb80 1 -- start start 2024-05-10T04:06:21.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.747+0000 7f2fd856cb80 1 Processor -- start 2024-05-10T04:06:21.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.757+0000 7f2fd856cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 0x55e43dcdab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.757+0000 7f2fd856cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccd570 0x55e43dcdb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.757+0000 7f2fd856cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccdec0 0x55e43dcdbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.757+0000 7f2fd856cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43dcdc510 con 0x55e43dccdec0 2024-05-10T04:06:21.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.757+0000 7f2fd856cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43dcdc680 con 0x55e43dccc6a0 2024-05-10T04:06:21.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.757+0000 7f2fd856cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43dcdc840 con 0x55e43dccd570 2024-05-10T04:06:21.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.757+0000 7f2fd856cb80 10 client.0 _ll_register_callbacks cb 0x55e43db91b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:21.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.765+0000 7f2fc57c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccd570 0x55e43dcdb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.765+0000 7f2fc57c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccd570 0x55e43dcdb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:35456/0 (socket says 172.21.15.150:35456) 2024-05-10T04:06:21.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.765+0000 7f2fc57c3700 1 -- 172.21.15.150:0/3765474454 learned_addr learned my addr 172.21.15.150:0/3765474454 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:21.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.766+0000 7f2fc67c5700 1 --2- 172.21.15.150:0/3765474454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccdec0 0x55e43dcdbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.769+0000 7f2fc57c3700 1 -- 172.21.15.150:0/3765474454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 msgr2=0x55e43dcdab60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:06:21.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.769+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/3765474454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 0x55e43dcdab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.769+0000 7f2fc57c3700 1 -- 172.21.15.150:0/3765474454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccdec0 msgr2=0x55e43dcdbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:21.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.769+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/3765474454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccdec0 0x55e43dcdbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.769+0000 7f2fc57c3700 1 -- 172.21.15.150:0/3765474454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e43da19b60 con 0x55e43dccd570 2024-05-10T04:06:21.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.774+0000 7f2fc57c3700 1 --2- 172.21.15.150:0/3765474454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccd570 0x55e43dcdb0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00a38f0 tx=0x7f2fb009bba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e5815145840fccc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:21.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.775+0000 7f2fa97fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:06:21.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.777+0000 7f2fa97fa700 1 -- 172.21.15.150:0/3765474454 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00b8070 con 0x55e43dccd570 2024-05-10T04:06:21.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.777+0000 7f2fa97fa700 1 -- 172.21.15.150:0/3765474454 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2fb00b9070 con 0x55e43dccd570 2024-05-10T04:06:21.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.778+0000 7f2fd856cb80 1 -- 172.21.15.150:0/3765474454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fb009ba30 con 0x55e43dccd570 2024-05-10T04:06:21.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.779+0000 7f2fa97fa700 1 -- 172.21.15.150:0/3765474454 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fb009f860 con 0x55e43dccd570 2024-05-10T04:06:21.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.779+0000 7f2fd856cb80 10 client.25735 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:21.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.779+0000 7f2fd856cb80 1 -- 172.21.15.150:0/3765474454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e43da71cd0 con 0x55e43dccd570 2024-05-10T04:06:21.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.781+0000 7f2fa97fa700 1 -- 172.21.15.150:0/3765474454 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fb009ba30 con 0x55e43dccd570 2024-05-10T04:06:21.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.781+0000 7f2fa97fa700 1 -- 172.21.15.150:0/3765474454 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2fb001abc0 con 0x55e43dccd570 2024-05-10T04:06:21.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.783+0000 7f2fd856cb80 10 client.25735 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:21.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.783+0000 7f2fd856cb80 10 client.25735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:21.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.783+0000 7f2fd856cb80 20 client.25735 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:21.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.783+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/3765474454 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55e43dcddac0 0x55e43dd39f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:21.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.783+0000 7f2fd856cb80 4 client.25735 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:06:21.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.783+0000 7f2fd856cb80 1 -- 172.21.15.150:0/3765474454 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e43dbe4ec0 con 0x55e43dcddac0 2024-05-10T04:06:21.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.787+0000 7f2fc5fc4700 1 --2- 172.21.15.150:0/3765474454 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55e43dcddac0 0x55e43dd39f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:21.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.792+0000 7f2fc5fc4700 1 --2- 172.21.15.150:0/3765474454 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55e43dcddac0 0x55e43dd39f90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82bd5897af1373f1 server_cookie=a4e5f10254ce66e9 in_seq=0 out_seq=0 2024-05-10T04:06:21.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.792+0000 7f2fa97fa700 10 client.25735 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:21.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.792+0000 7f2fa97fa700 1 -- 172.21.15.150:0/3765474454 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e43dbe4ec0 con 0x55e43dcddac0 2024-05-10T04:06:21.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.792+0000 7f2fa97fa700 10 client.25735 handle_command_reply: tid=0 2024-05-10T04:06:21.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.792+0000 7f2fd856cb80 1 client.25735 shutdown 2024-05-10T04:06:21.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.796+0000 7f2fd856cb80 1 -- 172.21.15.150:0/3765474454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccd570 msgr2=0x55e43dcdb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:21.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.796+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/3765474454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccd570 0x55e43dcdb0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00a38f0 tx=0x7f2fb009bba0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.796+0000 7f2fd856cb80 1 -- 172.21.15.150:0/3765474454 shutdown_connections 2024-05-10T04:06:21.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.796+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/3765474454 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55e43dcddac0 0x55e43dd39f90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.796+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/3765474454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e43dccdec0 0x55e43dcdbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.796+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/3765474454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e43dccc6a0 0x55e43dcdab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.796+0000 7f2fd856cb80 1 --2- 172.21.15.150:0/3765474454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e43dccd570 0x55e43dcdb0a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.796+0000 7f2fd856cb80 1 -- 172.21.15.150:0/3765474454 >> 172.21.15.150:0/3765474454 conn(0x55e43dc2ea60 msgr2=0x55e43dce3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:21.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.796+0000 7f2fa97fa700 0 client.25735 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:21.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.796+0000 7f2fd856cb80 1 -- 172.21.15.150:0/3765474454 shutdown_connections 2024-05-10T04:06:21.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.796+0000 7f2fd856cb80 1 -- 172.21.15.150:0/3765474454 wait complete. 2024-05-10T04:06:21.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.797+0000 7f2fd856cb80 20 client.25735 trim_cache size 0 max 16384 2024-05-10T04:06:21.802 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:06:21.803 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "no active scrubs running", 2024-05-10T04:06:21.803 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": {} 2024-05-10T04:06:21.803 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:06:21.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.802+0000 7f2fc8228700 1 -- 172.21.15.150:0/2295959953 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2fac0a1290 msgr2=0x7f2fac0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:21.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.802+0000 7f2fc8228700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2fac0a1290 0x7f2fac0a37a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55e43dccc380 tx=0x7f2fbc0058e0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.802+0000 7f2fc8228700 1 -- 172.21.15.150:0/2295959953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fc0109720 msgr2=0x7f2fc019ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:21.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.802+0000 7f2fc8228700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fc0109720 0x7f2fc019ea80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0013b40 tx=0x7f2fb0009810 comp rx=0 tx=0).stop 2024-05-10T04:06:21.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.810+0000 7f2fc8228700 1 -- 172.21.15.150:0/2295959953 shutdown_connections 2024-05-10T04:06:21.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.810+0000 7f2fc8228700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2fac0a1290 0x7f2fac0a37a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.810+0000 7f2fc8228700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fc010a0c0 0x7f2fc01b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.810+0000 7f2fc8228700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fc0108630 0x7f2fc019e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.810+0000 7f2fc8228700 1 --2- 172.21.15.150:0/2295959953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fc0109720 0x7f2fc019ea80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:21.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.810+0000 7f2fc8228700 1 -- 172.21.15.150:0/2295959953 >> 172.21.15.150:0/2295959953 conn(0x7f2fc006ee40 msgr2=0x7f2fc010c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:21.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.822+0000 7f2fc8228700 1 -- 172.21.15.150:0/2295959953 shutdown_connections 2024-05-10T04:06:21.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:21.824+0000 7f2fc8228700 1 -- 172.21.15.150:0/2295959953 wait complete. 2024-05-10T04:06:21.902 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T04:06:21.902 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:06:22.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.417+0000 7f341d62a700 1 -- 172.21.15.150:0/58415418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3418109300 msgr2=0x7f3418109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:22.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.417+0000 7f341d62a700 1 --2- 172.21.15.150:0/58415418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3418109300 0x7f3418109710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3408009990 tx=0x7f3408009ca0 comp rx=0 tx=0).stop 2024-05-10T04:06:22.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.426+0000 7f341d62a700 1 -- 172.21.15.150:0/58415418 shutdown_connections 2024-05-10T04:06:22.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.426+0000 7f341d62a700 1 --2- 172.21.15.150:0/58415418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3418109c50 0x7f341810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:22.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.426+0000 7f341d62a700 1 --2- 172.21.15.150:0/58415418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3418108630 0x7f3418108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:22.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.426+0000 7f341d62a700 1 --2- 172.21.15.150:0/58415418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3418109300 0x7f3418109710 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:22.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.426+0000 7f341d62a700 1 -- 172.21.15.150:0/58415418 >> 172.21.15.150:0/58415418 conn(0x7f341806ea00 msgr2=0x7f3418070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:22.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.430+0000 7f341d62a700 1 -- 172.21.15.150:0/58415418 shutdown_connections 2024-05-10T04:06:22.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.432+0000 7f341d62a700 1 -- 172.21.15.150:0/58415418 wait complete. 2024-05-10T04:06:22.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.433+0000 7f341d62a700 1 Processor -- start 2024-05-10T04:06:22.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.435+0000 7f341d62a700 1 -- start start 2024-05-10T04:06:22.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.435+0000 7f341d62a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3418108630 0x7f3418119dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:22.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.435+0000 7f341d62a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3418109300 0x7f341811a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:22.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.435+0000 7f341d62a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3418109c50 0x7f34181184e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:22.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.435+0000 7f341d62a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3418118be0 con 0x7f3418109c50 2024-05-10T04:06:22.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.435+0000 7f341d62a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3418118d50 con 0x7f3418109300 2024-05-10T04:06:22.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.435+0000 7f341d62a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3418118f10 con 0x7f3418108630 2024-05-10T04:06:22.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.436+0000 7f3416ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3418108630 0x7f3418119dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:22.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.436+0000 7f3416ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3418108630 0x7f3418119dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:35468/0 (socket says 172.21.15.150:35468) 2024-05-10T04:06:22.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.436+0000 7f3416ffd700 1 -- 172.21.15.150:0/161559419 learned_addr learned my addr 172.21.15.150:0/161559419 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:22.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.441+0000 7f34167fc700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3418109300 0x7f341811a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:22.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.446+0000 7f34177fe700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3418109c50 0x7f34181184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:22.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.447+0000 7f3416ffd700 1 -- 172.21.15.150:0/161559419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3418109300 msgr2=0x7f341811a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:22.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.447+0000 7f3416ffd700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3418109300 0x7f341811a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:22.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.447+0000 7f3416ffd700 1 -- 172.21.15.150:0/161559419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3418109c50 msgr2=0x7f34181184e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:22.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.447+0000 7f3416ffd700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3418109c50 0x7f34181184e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:22.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.447+0000 7f3416ffd700 1 -- 172.21.15.150:0/161559419 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34080095f0 con 0x7f3418108630 2024-05-10T04:06:22.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.465+0000 7f3416ffd700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3418108630 0x7f3418119dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f341000ada0 tx=0x7f341000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:22.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.471+0000 7f33fffff700 1 -- 172.21.15.150:0/161559419 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341000f070 con 0x7f3418108630 2024-05-10T04:06:22.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.472+0000 7f33fffff700 1 -- 172.21.15.150:0/161559419 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f341000ae80 con 0x7f3418108630 2024-05-10T04:06:22.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.472+0000 7f33fffff700 1 -- 172.21.15.150:0/161559419 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410005520 con 0x7f3418108630 2024-05-10T04:06:22.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.474+0000 7f341d62a700 1 -- 172.21.15.150:0/161559419 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34181191f0 con 0x7f3418108630 2024-05-10T04:06:22.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.474+0000 7f341d62a700 1 -- 172.21.15.150:0/161559419 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34181b7c70 con 0x7f3418108630 2024-05-10T04:06:22.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.481+0000 7f341d62a700 1 -- 172.21.15.150:0/161559419 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3418112330 con 0x7f3418108630 2024-05-10T04:06:22.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.489+0000 7f33fffff700 1 -- 172.21.15.150:0/161559419 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3410004ad0 con 0x7f3418108630 2024-05-10T04:06:22.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.490+0000 7f33fffff700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f34000a1320 0x7f34000a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:22.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.491+0000 7f33fffff700 1 -- 172.21.15.150:0/161559419 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f34100cc210 con 0x7f3418108630 2024-05-10T04:06:22.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.497+0000 7f33fffff700 1 -- 172.21.15.150:0/161559419 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3410095740 con 0x7f3418108630 2024-05-10T04:06:22.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.501+0000 7f34167fc700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f34000a1320 0x7f34000a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:22.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:22.508+0000 7f34167fc700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f34000a1320 0x7f34000a3830 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f34080038c0 tx=0x7f3408005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:22.723 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:22 smithi150 ceph-mon[42477]: pgmap v343: 129 pgs: 129 active+clean; 1.1 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-10T04:06:22.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:22 smithi181 ceph-mon[46602]: pgmap v343: 129 pgs: 129 active+clean; 1.1 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-10T04:06:22.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:22 smithi161 ceph-mon[46918]: pgmap v343: 129 pgs: 129 active+clean; 1.1 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-10T04:06:23.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.057+0000 7f341d62a700 1 -- 172.21.15.150:0/161559419 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3418110870 con 0x7f3418108630 2024-05-10T04:06:23.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.072+0000 7f33fffff700 1 -- 172.21.15.150:0/161559419 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10886 (secure 0 0 0) 0x7f3410098d60 con 0x7f3418108630 2024-05-10T04:06:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:06:23.092 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:04:04.157819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:06:23.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.091+0000 7f341d62a700 1 -- 172.21.15.150:0/161559419 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f34000a1320 msgr2=0x7f34000a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:23.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.091+0000 7f341d62a700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f34000a1320 0x7f34000a3830 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f34080038c0 tx=0x7f3408005040 comp rx=0 tx=0).stop 2024-05-10T04:06:23.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.091+0000 7f341d62a700 1 -- 172.21.15.150:0/161559419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3418108630 msgr2=0x7f3418119dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:23.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.091+0000 7f341d62a700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3418108630 0x7f3418119dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f341000ada0 tx=0x7f341000c330 comp rx=0 tx=0).stop 2024-05-10T04:06:23.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.107+0000 7f341d62a700 1 -- 172.21.15.150:0/161559419 shutdown_connections 2024-05-10T04:06:23.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.107+0000 7f341d62a700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f34000a1320 0x7f34000a3830 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.107+0000 7f341d62a700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3418109c50 0x7f34181184e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.107+0000 7f341d62a700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3418108630 0x7f3418119dc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.107+0000 7f341d62a700 1 --2- 172.21.15.150:0/161559419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3418109300 0x7f341811a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.107+0000 7f341d62a700 1 -- 172.21.15.150:0/161559419 >> 172.21.15.150:0/161559419 conn(0x7f341806ea00 msgr2=0x7f341810c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:23.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.131+0000 7f341d62a700 1 -- 172.21.15.150:0/161559419 shutdown_connections 2024-05-10T04:06:23.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.143+0000 7f341d62a700 1 -- 172.21.15.150:0/161559419 wait complete. 2024-05-10T04:06:23.186 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 25 2024-05-10T04:06:23.210 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T04:06:23.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:23 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/161559419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:06:23.466 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:23 smithi150 ceph-mon[42477]: pgmap v344: 129 pgs: 129 active+clean; 2.2 GiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 107 MiB/s wr, 39 op/s 2024-05-10T04:06:23.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:23 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/161559419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:06:23.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:23 smithi181 ceph-mon[46602]: pgmap v344: 129 pgs: 129 active+clean; 2.2 GiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 107 MiB/s wr, 39 op/s 2024-05-10T04:06:23.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:23 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/161559419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:06:23.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:23 smithi161 ceph-mon[46918]: pgmap v344: 129 pgs: 129 active+clean; 2.2 GiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 107 MiB/s wr, 39 op/s 2024-05-10T04:06:23.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.842+0000 7fc793be7700 1 -- 172.21.15.150:0/24035706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc78c109300 msgr2=0x7fc78c109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:23.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.842+0000 7fc793be7700 1 --2- 172.21.15.150:0/24035706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc78c109300 0x7fc78c109710 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc788009990 tx=0x7fc788009ca0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.853+0000 7fc793be7700 1 -- 172.21.15.150:0/24035706 shutdown_connections 2024-05-10T04:06:23.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.853+0000 7fc793be7700 1 --2- 172.21.15.150:0/24035706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc78c109c50 0x7fc78c10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.853+0000 7fc793be7700 1 --2- 172.21.15.150:0/24035706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc78c108630 0x7fc78c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.853+0000 7fc793be7700 1 --2- 172.21.15.150:0/24035706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc78c109300 0x7fc78c109710 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.853+0000 7fc793be7700 1 -- 172.21.15.150:0/24035706 >> 172.21.15.150:0/24035706 conn(0x7fc78c06ea00 msgr2=0x7fc78c070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:23.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.854+0000 7fc793be7700 1 -- 172.21.15.150:0/24035706 shutdown_connections 2024-05-10T04:06:23.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.854+0000 7fc793be7700 1 -- 172.21.15.150:0/24035706 wait complete. 2024-05-10T04:06:23.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.855+0000 7fc793be7700 1 Processor -- start 2024-05-10T04:06:23.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.855+0000 7fc793be7700 1 -- start start 2024-05-10T04:06:23.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.856+0000 7fc793be7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc78c108630 0x7fc78c119e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:23.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.856+0000 7fc793be7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc78c109300 0x7fc78c11a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:23.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.856+0000 7fc793be7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc78c109c50 0x7fc78c118610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:23.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.856+0000 7fc793be7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c118d10 con 0x7fc78c108630 2024-05-10T04:06:23.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.856+0000 7fc793be7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c118e80 con 0x7fc78c109300 2024-05-10T04:06:23.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.856+0000 7fc793be7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c119040 con 0x7fc78c109c50 2024-05-10T04:06:23.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.858+0000 7fc791182700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc78c109300 0x7fc78c11a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:23.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.859+0000 7fc791182700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc78c109300 0x7fc78c11a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:38004/0 (socket says 172.21.15.150:38004) 2024-05-10T04:06:23.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.859+0000 7fc791182700 1 -- 172.21.15.150:0/2474124208 learned_addr learned my addr 172.21.15.150:0/2474124208 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:23.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.859+0000 7fc792184700 1 --2- 172.21.15.150:0/2474124208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc78c109c50 0x7fc78c118610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:23.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.860+0000 7fc791182700 1 -- 172.21.15.150:0/2474124208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc78c109c50 msgr2=0x7fc78c118610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:23.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.860+0000 7fc791182700 1 --2- 172.21.15.150:0/2474124208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc78c109c50 0x7fc78c118610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.860+0000 7fc791182700 1 -- 172.21.15.150:0/2474124208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc78c108630 msgr2=0x7fc78c119e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:23.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.860+0000 7fc791182700 1 --2- 172.21.15.150:0/2474124208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc78c108630 0x7fc78c119e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.861+0000 7fc791182700 1 -- 172.21.15.150:0/2474124208 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7880095f0 con 0x7fc78c109300 2024-05-10T04:06:23.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.871+0000 7fc791182700 1 --2- 172.21.15.150:0/2474124208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc78c109300 0x7fc78c11a3a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc788005950 tx=0x7fc78800f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:23.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.881+0000 7fc782ffd700 1 -- 172.21.15.150:0/2474124208 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78801d070 con 0x7fc78c109300 2024-05-10T04:06:23.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.881+0000 7fc782ffd700 1 -- 172.21.15.150:0/2474124208 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc788003e20 con 0x7fc78c109300 2024-05-10T04:06:23.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.881+0000 7fc782ffd700 1 -- 172.21.15.150:0/2474124208 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7880173f0 con 0x7fc78c109300 2024-05-10T04:06:23.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.883+0000 7fc793be7700 1 -- 172.21.15.150:0/2474124208 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78c1192c0 con 0x7fc78c109300 2024-05-10T04:06:23.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.883+0000 7fc793be7700 1 -- 172.21.15.150:0/2474124208 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78c121ec0 con 0x7fc78c109300 2024-05-10T04:06:23.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.902+0000 7fc7a3f2bb80 1 Processor -- start 2024-05-10T04:06:23.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.903+0000 7fc782ffd700 1 -- 172.21.15.150:0/2474124208 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc78800fc20 con 0x7fc78c109300 2024-05-10T04:06:23.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.904+0000 7fc782ffd700 1 --2- 172.21.15.150:0/2474124208 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc7780a13b0 0x7fc7780a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:23.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.904+0000 7fc782ffd700 1 -- 172.21.15.150:0/2474124208 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc7880cdfe0 con 0x7fc78c109300 2024-05-10T04:06:23.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.905+0000 7fc7a3f2bb80 1 -- start start 2024-05-10T04:06:23.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.905+0000 7fc7a3f2bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fd0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:23.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.905+0000 7fc7a3f2bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd1570 0x55b7f7fd1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:23.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.905+0000 7fc7a3f2bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fd87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:23.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.905+0000 7fc7a3f2bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f7ef7150 con 0x55b7f7fd1ec0 2024-05-10T04:06:23.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.905+0000 7fc7a3f2bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f7ef4b10 con 0x55b7f7fd1570 2024-05-10T04:06:23.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.906+0000 7fc7a3f2bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f7ef0400 con 0x55b7f7fd06a0 2024-05-10T04:06:23.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.906+0000 7fc792184700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fd87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:23.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.906+0000 7fc792184700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fd87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59234/0 (socket says 172.21.15.150:59234) 2024-05-10T04:06:23.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.906+0000 7fc792184700 1 -- 172.21.15.150:0/2580900811 learned_addr learned my addr 172.21.15.150:0/2580900811 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:23.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.907+0000 7fc791182700 1 --2- 172.21.15.150:0/2580900811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd1570 0x55b7f7fd1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:23.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.908+0000 7fc791983700 1 --2- 172.21.15.150:0/2580900811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fd0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:23.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.913+0000 7fc791182700 1 -- 172.21.15.150:0/2580900811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd06a0 msgr2=0x55b7f7fd0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:23.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.913+0000 7fc791182700 1 --2- 172.21.15.150:0/2580900811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fd0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.913+0000 7fc791182700 1 -- 172.21.15.150:0/2580900811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd1ec0 msgr2=0x55b7f7fd87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:23.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.913+0000 7fc791182700 1 --2- 172.21.15.150:0/2580900811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fd87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.913+0000 7fc791182700 1 -- 172.21.15.150:0/2580900811 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7f7d713b0 con 0x55b7f7fd1570 2024-05-10T04:06:23.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.915+0000 7fc792184700 1 --2- 172.21.15.150:0/2474124208 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc7780a13b0 0x7fc7780a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:23.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.923+0000 7fc791182700 1 --2- 172.21.15.150:0/2580900811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd1570 0x55b7f7fd1980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc7880038c0 tx=0x7fc788005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aab95a0832bc57f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:23.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.927+0000 7fc777fff700 1 -- 172.21.15.150:0/2580900811 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78809fea0 con 0x55b7f7fd1570 2024-05-10T04:06:23.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.927+0000 7fc777fff700 1 -- 172.21.15.150:0/2580900811 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc78809b150 con 0x55b7f7fd1570 2024-05-10T04:06:23.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.927+0000 7fc777fff700 1 -- 172.21.15.150:0/2580900811 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78809b3b0 con 0x55b7f7fd1570 2024-05-10T04:06:23.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.929+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/2580900811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd1570 msgr2=0x55b7f7fd1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:23.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.929+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/2580900811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd1570 0x55b7f7fd1980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc7880038c0 tx=0x7fc788005f10 comp rx=0 tx=0).stop 2024-05-10T04:06:23.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.929+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/2580900811 shutdown_connections 2024-05-10T04:06:23.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.929+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/2580900811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fd87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.929+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/2580900811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fd0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.929+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/2580900811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd1570 0x55b7f7fd1980 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.929+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/2580900811 >> 172.21.15.150:0/2580900811 conn(0x55b7f7f32a60 msgr2=0x55b7f7fcafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:23.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.935+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/2580900811 shutdown_connections 2024-05-10T04:06:23.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.946+0000 7fc792184700 1 --2- 172.21.15.150:0/2474124208 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc7780a13b0 0x7fc7780a38c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc784004bb0 tx=0x7fc784007040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:23.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.946+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/2580900811 wait complete. 2024-05-10T04:06:23.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.947+0000 7fc7a3f2bb80 1 -- start start 2024-05-10T04:06:23.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.947+0000 7fc7a3f2bb80 1 Processor -- start 2024-05-10T04:06:23.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.955+0000 7fc7a3f2bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fda390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:23.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.955+0000 7fc7a3f2bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd1570 0x55b7f7fda8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:23.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.955+0000 7fc7a3f2bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fe1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:23.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.955+0000 7fc7a3f2bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f7e8c560 con 0x55b7f7fd06a0 2024-05-10T04:06:23.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.955+0000 7fc7a3f2bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f7e8c200 con 0x55b7f7fd1570 2024-05-10T04:06:23.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.955+0000 7fc7a3f2bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f7d736a0 con 0x55b7f7fd1ec0 2024-05-10T04:06:23.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.955+0000 7fc7a3f2bb80 10 client.0 _ll_register_callbacks cb 0x55b7f7e95b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:23.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.965+0000 7fc791182700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd1570 0x55b7f7fda8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:23.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.965+0000 7fc791983700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fda390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:23.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.966+0000 7fc791983700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fda390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59236/0 (socket says 172.21.15.150:59236) 2024-05-10T04:06:23.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.966+0000 7fc791983700 1 -- 172.21.15.150:0/3874565124 learned_addr learned my addr 172.21.15.150:0/3874565124 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:23.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.971+0000 7fc792184700 1 --2- 172.21.15.150:0/3874565124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fe1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:23.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.971+0000 7fc791983700 1 -- 172.21.15.150:0/3874565124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 msgr2=0x55b7f7fe1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:23.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.971+0000 7fc791983700 1 --2- 172.21.15.150:0/3874565124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fe1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.971+0000 7fc791983700 1 -- 172.21.15.150:0/3874565124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd1570 msgr2=0x55b7f7fda8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:23.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.971+0000 7fc791983700 1 --2- 172.21.15.150:0/3874565124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd1570 0x55b7f7fda8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:23.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.971+0000 7fc791983700 1 -- 172.21.15.150:0/3874565124 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78c121ec0 con 0x55b7f7fd06a0 2024-05-10T04:06:23.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.978+0000 7fc791983700 1 --2- 172.21.15.150:0/3874565124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fda390 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00d210 tx=0x7fc77c00d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743374db6badd963 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.982+0000 7fc7757fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:23.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.986+0000 7fc7757fa700 1 -- 172.21.15.150:0/3874565124 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c013480 con 0x55b7f7fd06a0 2024-05-10T04:06:23.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.986+0000 7fc7757fa700 1 -- 172.21.15.150:0/3874565124 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc77c0135e0 con 0x55b7f7fd06a0 2024-05-10T04:06:24.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:23.996+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/3874565124 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7f7e49270 con 0x55b7f7fd06a0 2024-05-10T04:06:24.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.003+0000 7fc7757fa700 1 -- 172.21.15.150:0/3874565124 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc77c0137a0 con 0x55b7f7fd06a0 2024-05-10T04:06:24.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.003+0000 7fc7a3f2bb80 10 client.16170 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:24.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.003+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/3874565124 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b7f7d1db60 con 0x55b7f7fd06a0 2024-05-10T04:06:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.012+0000 7fc7757fa700 1 -- 172.21.15.150:0/3874565124 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b7f7e49270 con 0x55b7f7fd06a0 2024-05-10T04:06:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.012+0000 7fc7757fa700 1 -- 172.21.15.150:0/3874565124 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc77c01da90 con 0x55b7f7fd06a0 2024-05-10T04:06:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.012+0000 7fc7a3f2bb80 10 client.16170 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.012+0000 7fc7a3f2bb80 10 client.16170 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T04:06:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.012+0000 7fc7a3f2bb80 20 client.16170 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.012+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/3874565124 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55b7f7fe6130 0x55b7f7fe8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.012+0000 7fc7a3f2bb80 4 client.16170 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.012+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/3874565124 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7f7f9f780 con 0x55b7f7fe6130 2024-05-10T04:06:24.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.031+0000 7fc791182700 1 --2- 172.21.15.150:0/3874565124 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55b7f7fe6130 0x55b7f7fe8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.063+0000 7fc791182700 1 --2- 172.21.15.150:0/3874565124 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55b7f7fe6130 0x55b7f7fe8580 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa750b8a4ac108a server_cookie=249dc0044edc3603 in_seq=0 out_seq=0 2024-05-10T04:06:24.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.065+0000 7fc7757fa700 10 client.16170 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:06:24.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.069+0000 7fc7757fa700 1 -- 172.21.15.150:0/3874565124 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b7f7f9f780 con 0x55b7f7fe6130 2024-05-10T04:06:24.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.069+0000 7fc7757fa700 10 client.16170 handle_command_reply: tid=0 2024-05-10T04:06:24.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.080+0000 7fc7a3f2bb80 1 client.16170 shutdown 2024-05-10T04:06:24.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.080+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/3874565124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd06a0 msgr2=0x55b7f7fda390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.080+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/3874565124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fda390 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00d210 tx=0x7fc77c00d5d0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.080+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/3874565124 shutdown_connections 2024-05-10T04:06:24.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.080+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/3874565124 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55b7f7fe6130 0x55b7f7fe8580 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.080+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/3874565124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fe1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.080+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/3874565124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fda390 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.080+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/3874565124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd1570 0x55b7f7fda8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.080+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/3874565124 >> 172.21.15.150:0/3874565124 conn(0x55b7f7f32a60 msgr2=0x55b7f7fcafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.081+0000 7fc7757fa700 0 client.16170 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:06:24.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.094+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/3874565124 shutdown_connections 2024-05-10T04:06:24.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.106+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/3874565124 wait complete. 2024-05-10T04:06:24.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.109+0000 7fc7a3f2bb80 20 client.16170 trim_cache size 0 max 16384 2024-05-10T04:06:24.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.157+0000 7fc7a3f2bb80 1 Processor -- start 2024-05-10T04:06:24.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.161+0000 7fc7a3f2bb80 1 -- start start 2024-05-10T04:06:24.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.162+0000 7fc7a3f2bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fdf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.162+0000 7fc7a3f2bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd1570 0x55b7f7fdf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.162+0000 7fc7a3f2bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fd8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.162+0000 7fc7a3f2bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f7fc7d30 con 0x55b7f7fd1570 2024-05-10T04:06:24.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.162+0000 7fc7a3f2bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f7fe3540 con 0x55b7f7fd06a0 2024-05-10T04:06:24.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.162+0000 7fc7a3f2bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f7fed720 con 0x55b7f7fd1ec0 2024-05-10T04:06:24.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.170+0000 7fc791182700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd1570 0x55b7f7fdf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.170+0000 7fc791182700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd1570 0x55b7f7fdf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59238/0 (socket says 172.21.15.150:59238) 2024-05-10T04:06:24.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.170+0000 7fc791182700 1 -- 172.21.15.150:0/1919446713 learned_addr learned my addr 172.21.15.150:0/1919446713 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:24.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.171+0000 7fc791983700 1 --2- 172.21.15.150:0/1919446713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fdf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.172+0000 7fc792184700 1 --2- 172.21.15.150:0/1919446713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fd8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.183+0000 7fc791182700 1 -- 172.21.15.150:0/1919446713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 msgr2=0x55b7f7fd8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.183+0000 7fc791182700 1 --2- 172.21.15.150:0/1919446713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fd8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.183+0000 7fc791182700 1 -- 172.21.15.150:0/1919446713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd06a0 msgr2=0x55b7f7fdf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.183+0000 7fc791182700 1 --2- 172.21.15.150:0/1919446713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fdf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.183+0000 7fc791182700 1 -- 172.21.15.150:0/1919446713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7f7eb6560 con 0x55b7f7fd1570 2024-05-10T04:06:24.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.191+0000 7fc791182700 1 --2- 172.21.15.150:0/1919446713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd1570 0x55b7f7fdf960 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc7880994a0 tx=0x7fc788003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9380ac2f7b854ab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.196+0000 7fc780d7f700 1 -- 172.21.15.150:0/1919446713 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78809fea0 con 0x55b7f7fd1570 2024-05-10T04:06:24.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.196+0000 7fc780d7f700 1 -- 172.21.15.150:0/1919446713 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7880995d0 con 0x55b7f7fd1570 2024-05-10T04:06:24.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.196+0000 7fc780d7f700 1 -- 172.21.15.150:0/1919446713 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7880b9480 con 0x55b7f7fd1570 2024-05-10T04:06:24.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.198+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/1919446713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd1570 msgr2=0x55b7f7fdf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.198+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/1919446713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd1570 0x55b7f7fdf960 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc7880994a0 tx=0x7fc788003f90 comp rx=0 tx=0).stop 2024-05-10T04:06:24.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.199+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/1919446713 shutdown_connections 2024-05-10T04:06:24.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.199+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/1919446713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fd8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.199+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/1919446713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fdf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.199+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/1919446713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd1570 0x55b7f7fdf960 secure :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc7880994a0 tx=0x7fc788003f90 comp rx=0 tx=0).stop 2024-05-10T04:06:24.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.199+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/1919446713 >> 172.21.15.150:0/1919446713 conn(0x55b7f7f32a60 msgr2=0x55b7f7fd47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.216+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/1919446713 shutdown_connections 2024-05-10T04:06:24.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.216+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/1919446713 wait complete. 2024-05-10T04:06:24.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.217+0000 7fc7a3f2bb80 1 -- start start 2024-05-10T04:06:24.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.217+0000 7fc7a3f2bb80 1 Processor -- start 2024-05-10T04:06:24.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.218+0000 7fc7a3f2bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fde5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.218+0000 7fc7a3f2bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fdeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.218+0000 7fc7a3f2bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fdf160 0x55b7f803a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.218+0000 7fc7a3f2bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f803acf0 con 0x55b7f7fd06a0 2024-05-10T04:06:24.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.218+0000 7fc7a3f2bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f803ae60 con 0x55b7f7fdf160 2024-05-10T04:06:24.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.218+0000 7fc7a3f2bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f803b020 con 0x55b7f7fd1ec0 2024-05-10T04:06:24.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.218+0000 7fc7a3f2bb80 10 client.0 _ll_register_callbacks cb 0x55b7f7e95b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:24.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.221+0000 7fc792184700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fdf160 0x55b7f803a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.221+0000 7fc791182700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fdeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.221+0000 7fc792184700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fdf160 0x55b7f803a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:38052/0 (socket says 172.21.15.150:38052) 2024-05-10T04:06:24.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.221+0000 7fc792184700 1 -- 172.21.15.150:0/2247507271 learned_addr learned my addr 172.21.15.150:0/2247507271 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:24.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.224+0000 7fc791983700 1 --2- 172.21.15.150:0/2247507271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fde5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.224+0000 7fc791182700 1 -- 172.21.15.150:0/2247507271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fdf160 msgr2=0x55b7f803a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.224+0000 7fc791182700 1 --2- 172.21.15.150:0/2247507271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fdf160 0x55b7f803a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.224+0000 7fc791182700 1 -- 172.21.15.150:0/2247507271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd06a0 msgr2=0x55b7f7fde5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.225+0000 7fc791182700 1 --2- 172.21.15.150:0/2247507271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fde5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.225+0000 7fc791182700 1 -- 172.21.15.150:0/2247507271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7f7d713b0 con 0x55b7f7fd1ec0 2024-05-10T04:06:24.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.242+0000 7fc791182700 1 --2- 172.21.15.150:0/2247507271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fdeb60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7880b8000 tx=0x7fc788005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d36a081a5edadab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.243+0000 7fc7757fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:06:24.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.255+0000 7fc7757fa700 1 -- 172.21.15.150:0/2247507271 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78809fea0 con 0x55b7f7fd1ec0 2024-05-10T04:06:24.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.255+0000 7fc7757fa700 1 -- 172.21.15.150:0/2247507271 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc7880050b0 con 0x55b7f7fd1ec0 2024-05-10T04:06:24.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.255+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/2247507271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc78809b150 con 0x55b7f7fd1ec0 2024-05-10T04:06:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.266+0000 7fc7757fa700 1 -- 172.21.15.150:0/2247507271 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7880cb9a0 con 0x55b7f7fd1ec0 2024-05-10T04:06:24.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.267+0000 7fc7a3f2bb80 10 client.25753 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:24.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.267+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/2247507271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b7f7d75cd0 con 0x55b7f7fd1ec0 2024-05-10T04:06:24.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.272+0000 7fc7757fa700 1 -- 172.21.15.150:0/2247507271 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc78809b150 con 0x55b7f7fd1ec0 2024-05-10T04:06:24.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.273+0000 7fc7757fa700 1 -- 172.21.15.150:0/2247507271 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc78809b4c0 con 0x55b7f7fd1ec0 2024-05-10T04:06:24.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.273+0000 7fc7a3f2bb80 10 client.25753 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:24.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.273+0000 7fc7a3f2bb80 10 client.25753 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T04:06:24.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.273+0000 7fc7a3f2bb80 20 client.25753 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:24.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.273+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/2247507271 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55b7f803c270 0x55b7f803e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.273+0000 7fc7a3f2bb80 4 client.25753 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:06:24.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.273+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/2247507271 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b7f7ee8ec0 con 0x55b7f803c270 2024-05-10T04:06:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.284+0000 7fc791983700 1 --2- 172.21.15.150:0/2247507271 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55b7f803c270 0x55b7f803e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.300+0000 7fc791983700 1 --2- 172.21.15.150:0/2247507271 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55b7f803c270 0x55b7f803e690 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4a28e48e5d0492c server_cookie=45f8fb39228817d9 in_seq=0 out_seq=0 2024-05-10T04:06:24.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.302+0000 7fc7757fa700 10 client.25753 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:06:24.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.312+0000 7fc7757fa700 1 -- 172.21.15.150:0/2247507271 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b7f7ee8ec0 con 0x55b7f803c270 2024-05-10T04:06:24.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.312+0000 7fc7757fa700 10 client.25753 handle_command_reply: tid=0 2024-05-10T04:06:24.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.312+0000 7fc7a3f2bb80 1 client.25753 shutdown 2024-05-10T04:06:24.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.331+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/2247507271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 msgr2=0x55b7f7fdeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.331+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/2247507271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fdeb60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7880b8000 tx=0x7fc788005f10 comp rx=0 tx=0).stop 2024-05-10T04:06:24.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.332+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/2247507271 shutdown_connections 2024-05-10T04:06:24.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.332+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/2247507271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b7f7fdf160 0x55b7f803a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.332+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/2247507271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b7f7fd06a0 0x55b7f7fde5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.332+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/2247507271 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x55b7f803c270 0x55b7f803e690 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.332+0000 7fc7a3f2bb80 1 --2- 172.21.15.150:0/2247507271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7f7fd1ec0 0x55b7f7fdeb60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.332+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/2247507271 >> 172.21.15.150:0/2247507271 conn(0x55b7f7f32a60 msgr2=0x55b7f7fd47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.332+0000 7fc7757fa700 0 client.25753 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:06:24.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.339+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/2247507271 shutdown_connections 2024-05-10T04:06:24.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.351+0000 7fc7a3f2bb80 1 -- 172.21.15.150:0/2247507271 wait complete. 2024-05-10T04:06:24.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.354+0000 7fc7a3f2bb80 20 client.25753 trim_cache size 0 max 16384 2024-05-10T04:06:24.357 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:06:24.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.363+0000 7fc793be7700 1 -- 172.21.15.150:0/2474124208 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc7780a13b0 msgr2=0x7fc7780a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.363+0000 7fc793be7700 1 --2- 172.21.15.150:0/2474124208 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc7780a13b0 0x7fc7780a38c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc784004bb0 tx=0x7fc784007040 comp rx=0 tx=0).stop 2024-05-10T04:06:24.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.363+0000 7fc793be7700 1 -- 172.21.15.150:0/2474124208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc78c109300 msgr2=0x7fc78c11a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.363+0000 7fc793be7700 1 --2- 172.21.15.150:0/2474124208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc78c109300 0x7fc78c11a3a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc788005950 tx=0x7fc78800f740 comp rx=0 tx=0).stop 2024-05-10T04:06:24.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.366+0000 7fc793be7700 1 -- 172.21.15.150:0/2474124208 shutdown_connections 2024-05-10T04:06:24.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.366+0000 7fc793be7700 1 --2- 172.21.15.150:0/2474124208 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc7780a13b0 0x7fc7780a38c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.366+0000 7fc793be7700 1 --2- 172.21.15.150:0/2474124208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc78c109c50 0x7fc78c118610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.366+0000 7fc793be7700 1 --2- 172.21.15.150:0/2474124208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc78c108630 0x7fc78c119e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.366+0000 7fc793be7700 1 --2- 172.21.15.150:0/2474124208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc78c109300 0x7fc78c11a3a0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc788005950 tx=0x7fc78800f740 comp rx=0 tx=0).stop 2024-05-10T04:06:24.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.366+0000 7fc793be7700 1 -- 172.21.15.150:0/2474124208 >> 172.21.15.150:0/2474124208 conn(0x7fc78c06ea00 msgr2=0x7fc78c10c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.384+0000 7fc793be7700 1 -- 172.21.15.150:0/2474124208 shutdown_connections 2024-05-10T04:06:24.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.387+0000 7fc793be7700 1 -- 172.21.15.150:0/2474124208 wait complete. 2024-05-10T04:06:24.447 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T04:06:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.491+0000 7f45db0a6700 1 -- 172.21.15.150:0/1246055850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d4109c50 msgr2=0x7f45d410e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.491+0000 7f45db0a6700 1 --2- 172.21.15.150:0/1246055850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d4109c50 0x7f45d410e560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f45c40098f0 tx=0x7f45c4009c00 comp rx=0 tx=0).stop 2024-05-10T04:06:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.502+0000 7f45db0a6700 1 -- 172.21.15.150:0/1246055850 shutdown_connections 2024-05-10T04:06:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.502+0000 7f45db0a6700 1 --2- 172.21.15.150:0/1246055850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d4109c50 0x7f45d410e560 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.502+0000 7f45db0a6700 1 --2- 172.21.15.150:0/1246055850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d4108630 0x7f45d4108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.502+0000 7f45db0a6700 1 --2- 172.21.15.150:0/1246055850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45d4109300 0x7f45d4109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.502+0000 7f45db0a6700 1 -- 172.21.15.150:0/1246055850 >> 172.21.15.150:0/1246055850 conn(0x7f45d406ea00 msgr2=0x7f45d4070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.506+0000 7f45db0a6700 1 -- 172.21.15.150:0/1246055850 shutdown_connections 2024-05-10T04:06:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.511+0000 7f45db0a6700 1 -- 172.21.15.150:0/1246055850 wait complete. 2024-05-10T04:06:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.512+0000 7f45db0a6700 1 Processor -- start 2024-05-10T04:06:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.512+0000 7f45db0a6700 1 -- start start 2024-05-10T04:06:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.512+0000 7f45db0a6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45d4108630 0x7f45d41a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.512+0000 7f45db0a6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d4109300 0x7f45d41a7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.513+0000 7f45db0a6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d4109c50 0x7f45d41ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.513+0000 7f45db0a6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d41ae8c0 con 0x7f45d4109300 2024-05-10T04:06:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.513+0000 7f45db0a6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d41101d0 con 0x7f45d4108630 2024-05-10T04:06:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.513+0000 7f45db0a6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d4110390 con 0x7f45d4109c50 2024-05-10T04:06:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.513+0000 7f45d3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d4109300 0x7f45d41a7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.513+0000 7f45d3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d4109300 0x7f45d41a7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59262/0 (socket says 172.21.15.150:59262) 2024-05-10T04:06:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.513+0000 7f45d3fff700 1 -- 172.21.15.150:0/406705077 learned_addr learned my addr 172.21.15.150:0/406705077 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.515+0000 7f45d8e42700 1 --2- 172.21.15.150:0/406705077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45d4108630 0x7f45d41a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.515+0000 7f45d3fff700 1 -- 172.21.15.150:0/406705077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d4109c50 msgr2=0x7f45d41ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.515+0000 7f45d3fff700 1 --2- 172.21.15.150:0/406705077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d4109c50 0x7f45d41ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.515+0000 7f45d3fff700 1 -- 172.21.15.150:0/406705077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45d4108630 msgr2=0x7f45d41a6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.515+0000 7f45d3fff700 1 --2- 172.21.15.150:0/406705077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45d4108630 0x7f45d41a6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.516+0000 7f45d3fff700 1 -- 172.21.15.150:0/406705077 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c4009520 con 0x7f45d4109300 2024-05-10T04:06:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.517+0000 7f45d3fff700 1 --2- 172.21.15.150:0/406705077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d4109300 0x7f45d41a7360 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00ef10 tx=0x7f45cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.517+0000 7f45d1ffb700 1 -- 172.21.15.150:0/406705077 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc00ca10 con 0x7f45d4109300 2024-05-10T04:06:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.517+0000 7f45d1ffb700 1 -- 172.21.15.150:0/406705077 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45cc00f040 con 0x7f45d4109300 2024-05-10T04:06:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.517+0000 7f45d1ffb700 1 -- 172.21.15.150:0/406705077 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0146e0 con 0x7f45d4109300 2024-05-10T04:06:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.520+0000 7f45db0a6700 1 -- 172.21.15.150:0/406705077 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d4110610 con 0x7f45d4109300 2024-05-10T04:06:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.520+0000 7f45db0a6700 1 -- 172.21.15.150:0/406705077 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d4110aa0 con 0x7f45d4109300 2024-05-10T04:06:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.526+0000 7f45eb3eab80 1 Processor -- start 2024-05-10T04:06:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.531+0000 7f45eb3eab80 1 -- start start 2024-05-10T04:06:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.531+0000 7f45eb3eab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570d180 0x55fd1570d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.531+0000 7f45eb3eab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570e050 0x55fd1570e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.531+0000 7f45eb3eab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e9a0 0x55fd157152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.531+0000 7f45eb3eab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd156340f0 con 0x55fd1570e050 2024-05-10T04:06:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.531+0000 7f45eb3eab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd15631ab0 con 0x55fd1570d180 2024-05-10T04:06:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.531+0000 7f45eb3eab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1562d3a0 con 0x55fd1570e9a0 2024-05-10T04:06:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.534+0000 7f45d1ffb700 1 -- 172.21.15.150:0/406705077 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f45cc004ad0 con 0x7f45d4109300 2024-05-10T04:06:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.535+0000 7f45d1ffb700 1 --2- 172.21.15.150:0/406705077 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f45c00a12f0 0x7f45c00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.536+0000 7f45d1ffb700 1 -- 172.21.15.150:0/406705077 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f45cc0cc700 con 0x7f45d4109300 2024-05-10T04:06:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.536+0000 7f45d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e9a0 0x55fd157152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.537+0000 7f45d9643700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570d180 0x55fd1570d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.537+0000 7f45d9643700 1 --2- 172.21.15.150:0/406705077 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f45c00a12f0 0x7f45c00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.539+0000 7f45d9643700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570d180 0x55fd1570d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:38074/0 (socket says 172.21.15.150:38074) 2024-05-10T04:06:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.539+0000 7f45d9643700 1 -- 172.21.15.150:0/3023948840 learned_addr learned my addr 172.21.15.150:0/3023948840 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.539+0000 7f45d8e42700 1 --2- 172.21.15.150:0/3023948840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570e050 0x55fd1570e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.540+0000 7f45d8e42700 1 -- 172.21.15.150:0/3023948840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e9a0 msgr2=0x55fd157152a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.540+0000 7f45d8e42700 1 --2- 172.21.15.150:0/3023948840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e9a0 0x55fd157152a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.540+0000 7f45d8e42700 1 -- 172.21.15.150:0/3023948840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570d180 msgr2=0x55fd1570d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.540+0000 7f45d8e42700 1 --2- 172.21.15.150:0/3023948840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570d180 0x55fd1570d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.540+0000 7f45d8e42700 1 -- 172.21.15.150:0/3023948840 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd154ae350 con 0x55fd1570e050 2024-05-10T04:06:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.540+0000 7f45d8e42700 1 --2- 172.21.15.150:0/3023948840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570e050 0x55fd1570e460 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f45d4072dd0 tx=0x7f45bc011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1846c07e38f4ba9f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.541+0000 7f45caffd700 1 -- 172.21.15.150:0/3023948840 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc011c90 con 0x55fd1570e050 2024-05-10T04:06:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.541+0000 7f45caffd700 1 -- 172.21.15.150:0/3023948840 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45bc011df0 con 0x55fd1570e050 2024-05-10T04:06:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.541+0000 7f45caffd700 1 -- 172.21.15.150:0/3023948840 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc005280 con 0x55fd1570e050 2024-05-10T04:06:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.542+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3023948840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570e050 msgr2=0x55fd1570e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.543+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3023948840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570e050 0x55fd1570e460 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f45d4072dd0 tx=0x7f45bc011440 comp rx=0 tx=0).stop 2024-05-10T04:06:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.543+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3023948840 shutdown_connections 2024-05-10T04:06:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.543+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3023948840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e9a0 0x55fd157152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.543+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3023948840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570d180 0x55fd1570d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.543+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3023948840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570e050 0x55fd1570e460 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.543+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3023948840 >> 172.21.15.150:0/3023948840 conn(0x55fd1566fa00 msgr2=0x55fd15707a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.544+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3023948840 shutdown_connections 2024-05-10T04:06:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.544+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3023948840 wait complete. 2024-05-10T04:06:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.545+0000 7f45eb3eab80 1 -- start start 2024-05-10T04:06:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.546+0000 7f45eb3eab80 1 Processor -- start 2024-05-10T04:06:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.547+0000 7f45eb3eab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570d180 0x55fd15717220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.547+0000 7f45eb3eab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e050 0x55fd15717760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.547+0000 7f45eb3eab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e9a0 0x55fd1571e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.547+0000 7f45eb3eab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd155c9500 con 0x55fd1570d180 2024-05-10T04:06:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.547+0000 7f45eb3eab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd155c91a0 con 0x55fd1570e050 2024-05-10T04:06:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.547+0000 7f45eb3eab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd154b0640 con 0x55fd1570e9a0 2024-05-10T04:06:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.547+0000 7f45eb3eab80 10 client.0 _ll_register_callbacks cb 0x55fd155d2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.548+0000 7f45d3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e050 0x55fd15717760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.548+0000 7f45d3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e050 0x55fd15717760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:38080/0 (socket says 172.21.15.150:38080) 2024-05-10T04:06:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.548+0000 7f45d3fff700 1 -- 172.21.15.150:0/3231220733 learned_addr learned my addr 172.21.15.150:0/3231220733 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.548+0000 7f45d9643700 1 --2- 172.21.15.150:0/3231220733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e9a0 0x55fd1571e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.549+0000 7f45d8e42700 1 --2- 172.21.15.150:0/3231220733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570d180 0x55fd15717220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.549+0000 7f45d3fff700 1 -- 172.21.15.150:0/3231220733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e9a0 msgr2=0x55fd1571e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.549+0000 7f45d3fff700 1 --2- 172.21.15.150:0/3231220733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e9a0 0x55fd1571e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.549+0000 7f45d3fff700 1 -- 172.21.15.150:0/3231220733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570d180 msgr2=0x55fd15717220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.549+0000 7f45d3fff700 1 --2- 172.21.15.150:0/3231220733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570d180 0x55fd15717220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.549+0000 7f45d3fff700 1 -- 172.21.15.150:0/3231220733 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc009920 con 0x55fd1570e050 2024-05-10T04:06:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.550+0000 7f45d3fff700 1 --2- 172.21.15.150:0/3231220733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e050 0x55fd15717760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00c7a0 tx=0x7f45cc099300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef3743dcedb77c65 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.550+0000 7f45b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:06:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.550+0000 7f45b3fff700 1 -- 172.21.15.150:0/3231220733 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc09d6a0 con 0x55fd1570e050 2024-05-10T04:06:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.550+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3231220733 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd15586210 con 0x55fd1570e050 2024-05-10T04:06:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.551+0000 7f45b3fff700 1 -- 172.21.15.150:0/3231220733 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f45cc0a1c60 con 0x55fd1570e050 2024-05-10T04:06:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.551+0000 7f45b3fff700 1 -- 172.21.15.150:0/3231220733 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45cc01bdf0 con 0x55fd1570e050 2024-05-10T04:06:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.551+0000 7f45eb3eab80 10 client.25856 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.551+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3231220733 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fd1545ab00 con 0x55fd1570e050 2024-05-10T04:06:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.556+0000 7f45b3fff700 1 -- 172.21.15.150:0/3231220733 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45cc0a5070 con 0x55fd1570e050 2024-05-10T04:06:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.556+0000 7f45b3fff700 1 -- 172.21.15.150:0/3231220733 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f45cc098d60 con 0x55fd1570e050 2024-05-10T04:06:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.556+0000 7f45eb3eab80 10 client.25856 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.556+0000 7f45eb3eab80 10 client.25856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.556+0000 7f45eb3eab80 20 client.25856 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.556+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3231220733 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55fd15722f50 0x55fd157253a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.557+0000 7f45eb3eab80 4 client.25856 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.557+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3231220733 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd156dc720 con 0x55fd15722f50 2024-05-10T04:06:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.561+0000 7f45d8e42700 1 --2- 172.21.15.150:0/3231220733 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55fd15722f50 0x55fd157253a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.564+0000 7f45d9643700 1 --2- 172.21.15.150:0/406705077 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f45c00a12f0 0x7f45c00a3800 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009fd0 tx=0x7f45c400fb10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.585+0000 7f45b3fff700 10 client.25856 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.585+0000 7f45d8e42700 1 --2- 172.21.15.150:0/3231220733 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55fd15722f50 0x55fd157253a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8183c039db9ed89 server_cookie=cbc6a4618d8d8f3 in_seq=0 out_seq=0 2024-05-10T04:06:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.588+0000 7f45b3fff700 1 -- 172.21.15.150:0/3231220733 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd156dc720 con 0x55fd15722f50 2024-05-10T04:06:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.588+0000 7f45b3fff700 10 client.25856 handle_command_reply: tid=0 2024-05-10T04:06:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.588+0000 7f45eb3eab80 1 client.25856 shutdown 2024-05-10T04:06:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.595+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3231220733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e050 msgr2=0x55fd15717760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.595+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3231220733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e050 0x55fd15717760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00c7a0 tx=0x7f45cc099300 comp rx=0 tx=0).stop 2024-05-10T04:06:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.599+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3231220733 shutdown_connections 2024-05-10T04:06:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.599+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3231220733 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55fd15722f50 0x55fd157253a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.599+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3231220733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e9a0 0x55fd1571e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.599+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3231220733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570d180 0x55fd15717220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.599+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3231220733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e050 0x55fd15717760 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.599+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3231220733 >> 172.21.15.150:0/3231220733 conn(0x55fd1566fa00 msgr2=0x55fd15707a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.599+0000 7f45b3fff700 0 client.25856 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.604+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3231220733 shutdown_connections 2024-05-10T04:06:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.606+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3231220733 wait complete. 2024-05-10T04:06:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.617+0000 7f45eb3eab80 20 client.25856 trim_cache size 0 max 16384 2024-05-10T04:06:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.642+0000 7f45eb3eab80 1 Processor -- start 2024-05-10T04:06:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.643+0000 7f45eb3eab80 1 -- start start 2024-05-10T04:06:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.643+0000 7f45eb3eab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570d180 0x55fd15724f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.643+0000 7f45eb3eab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570e050 0x55fd1571e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.643+0000 7f45eb3eab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e9a0 0x55fd157254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.643+0000 7f45eb3eab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd15702360 con 0x55fd1570e050 2024-05-10T04:06:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.643+0000 7f45eb3eab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd157203d0 con 0x55fd1570e9a0 2024-05-10T04:06:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.643+0000 7f45eb3eab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd15708560 con 0x55fd1570d180 2024-05-10T04:06:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.643+0000 7f45d3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570e050 0x55fd1571e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.643+0000 7f45d3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570e050 0x55fd1571e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59292/0 (socket says 172.21.15.150:59292) 2024-05-10T04:06:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.643+0000 7f45d3fff700 1 -- 172.21.15.150:0/3763463342 learned_addr learned my addr 172.21.15.150:0/3763463342 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.643+0000 7f45d8e42700 1 --2- 172.21.15.150:0/3763463342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570d180 0x55fd15724f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.644+0000 7f45d9643700 1 --2- 172.21.15.150:0/3763463342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e9a0 0x55fd157254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.644+0000 7f45d3fff700 1 -- 172.21.15.150:0/3763463342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570d180 msgr2=0x55fd15724f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.644+0000 7f45d3fff700 1 --2- 172.21.15.150:0/3763463342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570d180 0x55fd15724f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.644+0000 7f45d3fff700 1 -- 172.21.15.150:0/3763463342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e9a0 msgr2=0x55fd157254a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.644+0000 7f45d3fff700 1 --2- 172.21.15.150:0/3763463342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e9a0 0x55fd157254a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.644+0000 7f45d3fff700 1 -- 172.21.15.150:0/3763463342 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd155f3500 con 0x55fd1570e050 2024-05-10T04:06:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.644+0000 7f45d3fff700 1 --2- 172.21.15.150:0/3763463342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570e050 0x55fd1571e680 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f45cc097bc0 tx=0x7f45cc0999f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd39fe5cdff0f3c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.645+0000 7f45cb7fe700 1 -- 172.21.15.150:0/3763463342 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0a1c60 con 0x55fd1570e050 2024-05-10T04:06:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.645+0000 7f45cb7fe700 1 -- 172.21.15.150:0/3763463342 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45cc0a5040 con 0x55fd1570e050 2024-05-10T04:06:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.645+0000 7f45cb7fe700 1 -- 172.21.15.150:0/3763463342 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc098d60 con 0x55fd1570e050 2024-05-10T04:06:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.646+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3763463342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570e050 msgr2=0x55fd1571e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.646+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3763463342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570e050 0x55fd1571e680 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f45cc097bc0 tx=0x7f45cc0999f0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.646+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3763463342 shutdown_connections 2024-05-10T04:06:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.646+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3763463342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e9a0 0x55fd157254a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.646+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3763463342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570d180 0x55fd15724f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.646+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/3763463342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570e050 0x55fd1571e680 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.646+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3763463342 >> 172.21.15.150:0/3763463342 conn(0x55fd1566fa00 msgr2=0x55fd157157e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.646+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3763463342 shutdown_connections 2024-05-10T04:06:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.646+0000 7f45eb3eab80 1 -- 172.21.15.150:0/3763463342 wait complete. 2024-05-10T04:06:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.647+0000 7f45eb3eab80 1 -- start start 2024-05-10T04:06:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.647+0000 7f45eb3eab80 1 Processor -- start 2024-05-10T04:06:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.658+0000 7f45eb3eab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570d180 0x55fd1571b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.658+0000 7f45eb3eab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e050 0x55fd1571bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.658+0000 7f45eb3eab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e9a0 0x55fd1571cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.658+0000 7f45eb3eab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1571d100 con 0x55fd1570d180 2024-05-10T04:06:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.658+0000 7f45eb3eab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1571d270 con 0x55fd1570e9a0 2024-05-10T04:06:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.658+0000 7f45eb3eab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1571d430 con 0x55fd1570e050 2024-05-10T04:06:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.658+0000 7f45eb3eab80 10 client.0 _ll_register_callbacks cb 0x55fd155d2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.660+0000 7f45d8e42700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570d180 0x55fd1571b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.660+0000 7f45d8e42700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570d180 0x55fd1571b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59304/0 (socket says 172.21.15.150:59304) 2024-05-10T04:06:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.660+0000 7f45d8e42700 1 -- 172.21.15.150:0/2172010868 learned_addr learned my addr 172.21.15.150:0/2172010868 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.663+0000 7f45d8e42700 1 -- 172.21.15.150:0/2172010868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e050 msgr2=0x55fd1571bc90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:06:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.663+0000 7f45d8e42700 1 --2- 172.21.15.150:0/2172010868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e050 0x55fd1571bc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.663+0000 7f45d8e42700 1 -- 172.21.15.150:0/2172010868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e9a0 msgr2=0x55fd1571cbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:06:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.663+0000 7f45d8e42700 1 --2- 172.21.15.150:0/2172010868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e9a0 0x55fd1571cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.663+0000 7f45d8e42700 1 -- 172.21.15.150:0/2172010868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd1545ab00 con 0x55fd1570d180 2024-05-10T04:06:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.664+0000 7f45d8e42700 1 --2- 172.21.15.150:0/2172010868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570d180 0x55fd1571b750 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f45d4072dd0 tx=0x7f45bc0095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c6df0c9da23f29 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.664+0000 7f45b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.665+0000 7f45b3fff700 1 -- 172.21.15.150:0/2172010868 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc004a60 con 0x55fd1570d180 2024-05-10T04:06:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.665+0000 7f45b3fff700 1 -- 172.21.15.150:0/2172010868 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f45bc004d10 con 0x55fd1570d180 2024-05-10T04:06:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.666+0000 7f45eb3eab80 1 -- 172.21.15.150:0/2172010868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45bc011df0 con 0x55fd1570d180 2024-05-10T04:06:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.667+0000 7f45b3fff700 1 -- 172.21.15.150:0/2172010868 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45bc004ed0 con 0x55fd1570d180 2024-05-10T04:06:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.669+0000 7f45eb3eab80 10 client.16194 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.669+0000 7f45eb3eab80 1 -- 172.21.15.150:0/2172010868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fd154b2c70 con 0x55fd1570d180 2024-05-10T04:06:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.670+0000 7f45b3fff700 1 -- 172.21.15.150:0/2172010868 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45bc011df0 con 0x55fd1570d180 2024-05-10T04:06:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.670+0000 7f45b3fff700 1 -- 172.21.15.150:0/2172010868 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f45bc03c960 con 0x55fd1570d180 2024-05-10T04:06:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.671+0000 7f45eb3eab80 10 client.16194 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.671+0000 7f45eb3eab80 10 client.16194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.671+0000 7f45eb3eab80 20 client.16194 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.671+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/2172010868 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55fd15779560 0x55fd1577b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.671+0000 7f45eb3eab80 4 client.16194 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:06:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.671+0000 7f45eb3eab80 1 -- 172.21.15.150:0/2172010868 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd15625e60 con 0x55fd15779560 2024-05-10T04:06:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.677+0000 7f45d3fff700 1 --2- 172.21.15.150:0/2172010868 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55fd15779560 0x55fd1577b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.685+0000 7f45b3fff700 10 client.16194 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.685+0000 7f45d3fff700 1 --2- 172.21.15.150:0/2172010868 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55fd15779560 0x55fd1577b980 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae0f41e29f22b2b4 server_cookie=d5228a03489f1aa0 in_seq=0 out_seq=0 2024-05-10T04:06:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.775+0000 7f45b3fff700 1 -- 172.21.15.150:0/2172010868 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd15625e60 con 0x55fd15779560 2024-05-10T04:06:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.775+0000 7f45b3fff700 10 client.16194 handle_command_reply: tid=0 2024-05-10T04:06:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.775+0000 7f45eb3eab80 1 client.16194 shutdown 2024-05-10T04:06:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.787+0000 7f45eb3eab80 1 -- 172.21.15.150:0/2172010868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570d180 msgr2=0x55fd1571b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.787+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/2172010868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570d180 0x55fd1571b750 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f45d4072dd0 tx=0x7f45bc0095e0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.788+0000 7f45eb3eab80 1 -- 172.21.15.150:0/2172010868 shutdown_connections 2024-05-10T04:06:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.788+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/2172010868 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55fd15779560 0x55fd1577b980 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.788+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/2172010868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd1570e9a0 0x55fd1571cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.788+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/2172010868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd1570d180 0x55fd1571b750 secure :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f45d4072dd0 tx=0x7f45bc0095e0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.788+0000 7f45eb3eab80 1 --2- 172.21.15.150:0/2172010868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd1570e050 0x55fd1571bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.788+0000 7f45eb3eab80 1 -- 172.21.15.150:0/2172010868 >> 172.21.15.150:0/2172010868 conn(0x55fd1566fa00 msgr2=0x55fd157154d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.788+0000 7f45b3fff700 0 client.16194 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.804+0000 7f45eb3eab80 1 -- 172.21.15.150:0/2172010868 shutdown_connections 2024-05-10T04:06:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.811+0000 7f45eb3eab80 1 -- 172.21.15.150:0/2172010868 wait complete. 2024-05-10T04:06:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.821+0000 7f45eb3eab80 20 client.16194 trim_cache size 0 max 16384 2024-05-10T04:06:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:06:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:06:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:06:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:06:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.835+0000 7f45db0a6700 1 -- 172.21.15.150:0/406705077 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f45c00a12f0 msgr2=0x7f45c00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.838 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.835+0000 7f45db0a6700 1 --2- 172.21.15.150:0/406705077 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f45c00a12f0 0x7f45c00a3800 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009fd0 tx=0x7f45c400fb10 comp rx=0 tx=0).stop 2024-05-10T04:06:24.838 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.835+0000 7f45db0a6700 1 -- 172.21.15.150:0/406705077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d4109300 msgr2=0x7f45d41a7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.838 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.835+0000 7f45db0a6700 1 --2- 172.21.15.150:0/406705077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d4109300 0x7f45d41a7360 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00ef10 tx=0x7f45cc00c3a0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.837+0000 7f45db0a6700 1 -- 172.21.15.150:0/406705077 shutdown_connections 2024-05-10T04:06:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.837+0000 7f45db0a6700 1 --2- 172.21.15.150:0/406705077 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f45c00a12f0 0x7f45c00a3800 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.837+0000 7f45db0a6700 1 --2- 172.21.15.150:0/406705077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d4109c50 0x7f45d41ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.837+0000 7f45db0a6700 1 --2- 172.21.15.150:0/406705077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45d4108630 0x7f45d41a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.837+0000 7f45db0a6700 1 --2- 172.21.15.150:0/406705077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45d4109300 0x7f45d41a7360 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.837+0000 7f45db0a6700 1 -- 172.21.15.150:0/406705077 >> 172.21.15.150:0/406705077 conn(0x7f45d406ea00 msgr2=0x7f45d406f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.839+0000 7f45db0a6700 1 -- 172.21.15.150:0/406705077 shutdown_connections 2024-05-10T04:06:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:24.841+0000 7f45db0a6700 1 -- 172.21.15.150:0/406705077 wait complete. 2024-05-10T04:06:24.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.875+0000 7f01f0048700 1 -- 172.21.15.150:0/614799956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01e8109c50 msgr2=0x7f01e810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.875+0000 7f01f0048700 1 --2- 172.21.15.150:0/614799956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01e8109c50 0x7f01e810e560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009f90 tx=0x7f01e000b330 comp rx=0 tx=0).stop 2024-05-10T04:06:24.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.876+0000 7f01f0048700 1 -- 172.21.15.150:0/614799956 shutdown_connections 2024-05-10T04:06:24.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.877+0000 7f01f0048700 1 --2- 172.21.15.150:0/614799956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01e8109c50 0x7f01e810e560 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.877+0000 7f01f0048700 1 --2- 172.21.15.150:0/614799956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01e8108630 0x7f01e8108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.877+0000 7f01f0048700 1 --2- 172.21.15.150:0/614799956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e8109300 0x7f01e8109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.877+0000 7f01f0048700 1 -- 172.21.15.150:0/614799956 >> 172.21.15.150:0/614799956 conn(0x7f01e806ea00 msgr2=0x7f01e8070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.890+0000 7f01f0048700 1 -- 172.21.15.150:0/614799956 shutdown_connections 2024-05-10T04:06:24.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.901+0000 7f01f0048700 1 -- 172.21.15.150:0/614799956 wait complete. 2024-05-10T04:06:24.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.902+0000 7f01f0048700 1 Processor -- start 2024-05-10T04:06:24.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.902+0000 7f01f0048700 1 -- start start 2024-05-10T04:06:24.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.902+0000 7f01f0048700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01e8108630 0x7f01e8115350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.902+0000 7f01f0048700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e8109300 0x7f01e8115890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.902+0000 7f01f0048700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01e8109c50 0x7f01e81a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.902+0000 7f01f0048700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81a6090 con 0x7f01e8108630 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.902+0000 7f01f0048700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81a6200 con 0x7f01e8109300 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.902+0000 7f01f0048700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81a63c0 con 0x7f01e8109c50 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.903+0000 7f01edde4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01e8108630 0x7f01e8115350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.903+0000 7f01edde4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01e8108630 0x7f01e8115350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59320/0 (socket says 172.21.15.150:59320) 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.903+0000 7f01edde4700 1 -- 172.21.15.150:0/1180332181 learned_addr learned my addr 172.21.15.150:0/1180332181 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.903+0000 7f01ed5e3700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e8109300 0x7f01e8115890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.904+0000 7f01ed5e3700 1 -- 172.21.15.150:0/1180332181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01e8109c50 msgr2=0x7f01e81a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.904+0000 7f01ed5e3700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01e8109c50 0x7f01e81a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.904+0000 7f01ed5e3700 1 -- 172.21.15.150:0/1180332181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01e8108630 msgr2=0x7f01e8115350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.904+0000 7f01ed5e3700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01e8108630 0x7f01e8115350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.904+0000 7f01ed5e3700 1 -- 172.21.15.150:0/1180332181 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e0009c40 con 0x7f01e8109300 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.904+0000 7f01edde4700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01e8108630 0x7f01e8115350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.904+0000 7f01ed5e3700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e8109300 0x7f01e8115890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f01e4006fd0 tx=0x7f01e400b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.905+0000 7f01deffd700 1 -- 172.21.15.150:0/1180332181 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4011830 con 0x7f01e8109300 2024-05-10T04:06:24.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.905+0000 7f01deffd700 1 -- 172.21.15.150:0/1180332181 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01e400be60 con 0x7f01e8109300 2024-05-10T04:06:24.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.905+0000 7f01deffd700 1 -- 172.21.15.150:0/1180332181 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4010d60 con 0x7f01e8109300 2024-05-10T04:06:24.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.907+0000 7f01f0048700 1 -- 172.21.15.150:0/1180332181 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e81a66a0 con 0x7f01e8109300 2024-05-10T04:06:24.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.908+0000 7f01f0048700 1 -- 172.21.15.150:0/1180332181 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e81a6b30 con 0x7f01e8109300 2024-05-10T04:06:24.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.914+0000 7f020038cb80 1 Processor -- start 2024-05-10T04:06:24.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.915+0000 7f020038cb80 1 -- start start 2024-05-10T04:06:24.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.915+0000 7f020038cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d46f6a0 0x56409d46faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.915+0000 7f020038cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d470570 0x56409d470980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.915+0000 7f020038cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470ec0 0x56409d4777c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.915+0000 7f020038cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409d396150 con 0x56409d470570 2024-05-10T04:06:24.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.915+0000 7f020038cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409d393b10 con 0x56409d470ec0 2024-05-10T04:06:24.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.915+0000 7f020038cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409d38f400 con 0x56409d46f6a0 2024-05-10T04:06:24.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.917+0000 7f01ee5e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d46f6a0 0x56409d46faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.917+0000 7f01ee5e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d46f6a0 0x56409d46faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:35598/0 (socket says 172.21.15.150:35598) 2024-05-10T04:06:24.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.917+0000 7f01ee5e5700 1 -- 172.21.15.150:0/2189444372 learned_addr learned my addr 172.21.15.150:0/2189444372 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:24.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.918+0000 7f01deffd700 1 -- 172.21.15.150:0/1180332181 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f01e4010380 con 0x7f01e8109300 2024-05-10T04:06:24.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.918+0000 7f01ed5e3700 1 --2- 172.21.15.150:0/2189444372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470ec0 0x56409d4777c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.919+0000 7f01ed5e3700 1 -- 172.21.15.150:0/2189444372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d46f6a0 msgr2=0x56409d46faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.919+0000 7f01ed5e3700 1 --2- 172.21.15.150:0/2189444372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d46f6a0 0x56409d46faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.919+0000 7f01ed5e3700 1 -- 172.21.15.150:0/2189444372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d470570 msgr2=0x56409d470980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.919+0000 7f01ed5e3700 1 --2- 172.21.15.150:0/2189444372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d470570 0x56409d470980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.919+0000 7f01ed5e3700 1 -- 172.21.15.150:0/2189444372 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56409d2103b0 con 0x56409d470ec0 2024-05-10T04:06:24.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.919+0000 7f01deffd700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f01d40a1290 0x7f01d40a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.920+0000 7f01deffd700 1 -- 172.21.15.150:0/1180332181 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f01e40ce6c0 con 0x7f01e8109300 2024-05-10T04:06:24.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.920+0000 7f01ed5e3700 1 --2- 172.21.15.150:0/2189444372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470ec0 0x56409d4777c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f01e400bb30 tx=0x7f01e40d27a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=135aaba17e0996b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.924+0000 7f01d3fff700 1 -- 172.21.15.150:0/2189444372 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40092a0 con 0x56409d470ec0 2024-05-10T04:06:24.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.924+0000 7f01d3fff700 1 -- 172.21.15.150:0/2189444372 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01e40d2e70 con 0x56409d470ec0 2024-05-10T04:06:24.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.924+0000 7f01d3fff700 1 -- 172.21.15.150:0/2189444372 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e409b160 con 0x56409d470ec0 2024-05-10T04:06:24.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.926+0000 7f020038cb80 1 -- 172.21.15.150:0/2189444372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470ec0 msgr2=0x56409d4777c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.926+0000 7f020038cb80 1 --2- 172.21.15.150:0/2189444372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470ec0 0x56409d4777c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f01e400bb30 tx=0x7f01e40d27a0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.926+0000 7f01ee5e5700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f01d40a1290 0x7f01d40a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.927+0000 7f020038cb80 1 -- 172.21.15.150:0/2189444372 shutdown_connections 2024-05-10T04:06:24.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.927+0000 7f020038cb80 1 --2- 172.21.15.150:0/2189444372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470ec0 0x56409d4777c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.927+0000 7f020038cb80 1 --2- 172.21.15.150:0/2189444372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d46f6a0 0x56409d46faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.927+0000 7f020038cb80 1 --2- 172.21.15.150:0/2189444372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d470570 0x56409d470980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.927+0000 7f020038cb80 1 -- 172.21.15.150:0/2189444372 >> 172.21.15.150:0/2189444372 conn(0x56409d3d1a60 msgr2=0x56409d469fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.927+0000 7f020038cb80 1 -- 172.21.15.150:0/2189444372 shutdown_connections 2024-05-10T04:06:24.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.927+0000 7f020038cb80 1 -- 172.21.15.150:0/2189444372 wait complete. 2024-05-10T04:06:24.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.927+0000 7f01ee5e5700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f01d40a1290 0x7f01d40a37a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009550 tx=0x7f01e0007ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.928+0000 7f020038cb80 1 -- start start 2024-05-10T04:06:24.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.928+0000 7f020038cb80 1 Processor -- start 2024-05-10T04:06:24.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.929+0000 7f020038cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d479750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.929+0000 7f020038cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470570 0x56409d479c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.929+0000 7f020038cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470ec0 0x56409d480bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.929+0000 7f020038cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409d32b560 con 0x56409d46f6a0 2024-05-10T04:06:24.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.929+0000 7f020038cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409d32b200 con 0x56409d470ec0 2024-05-10T04:06:24.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.929+0000 7f020038cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409d2126a0 con 0x56409d470570 2024-05-10T04:06:24.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.929+0000 7f020038cb80 10 client.0 _ll_register_callbacks cb 0x56409d334b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:24.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.930+0000 7f01ee5e5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470ec0 0x56409d480bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.930+0000 7f01ee5e5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470ec0 0x56409d480bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:38120/0 (socket says 172.21.15.150:38120) 2024-05-10T04:06:24.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.930+0000 7f01ee5e5700 1 -- 172.21.15.150:0/3863499935 learned_addr learned my addr 172.21.15.150:0/3863499935 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:24.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.931+0000 7f01ee5e5700 1 -- 172.21.15.150:0/3863499935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470570 msgr2=0x56409d479c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:06:24.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.931+0000 7f01ee5e5700 1 --2- 172.21.15.150:0/3863499935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470570 0x56409d479c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.931+0000 7f01ee5e5700 1 -- 172.21.15.150:0/3863499935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 msgr2=0x56409d479750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.932+0000 7f01ee5e5700 1 --2- 172.21.15.150:0/3863499935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d479750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.932+0000 7f01ee5e5700 1 -- 172.21.15.150:0/3863499935 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e81a6b30 con 0x56409d470ec0 2024-05-10T04:06:24.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.932+0000 7f01ee5e5700 1 --2- 172.21.15.150:0/3863499935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470ec0 0x56409d480bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f01e0003c80 tx=0x7f01e000b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=450eda741015cef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.933+0000 7f01d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:06:24.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.933+0000 7f01d17fa700 1 -- 172.21.15.150:0/3863499935 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0022040 con 0x56409d470ec0 2024-05-10T04:06:24.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.933+0000 7f01d17fa700 1 -- 172.21.15.150:0/3863499935 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f01e00058e0 con 0x56409d470ec0 2024-05-10T04:06:24.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.933+0000 7f020038cb80 1 -- 172.21.15.150:0/3863499935 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56409d2e8270 con 0x56409d470ec0 2024-05-10T04:06:24.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.934+0000 7f01d17fa700 1 -- 172.21.15.150:0/3863499935 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01e000e040 con 0x56409d470ec0 2024-05-10T04:06:24.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.934+0000 7f020038cb80 10 client.25868 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:24.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.934+0000 7f020038cb80 1 -- 172.21.15.150:0/3863499935 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56409d1bcb60 con 0x56409d470ec0 2024-05-10T04:06:24.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.935+0000 7f01d17fa700 1 -- 172.21.15.150:0/3863499935 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56409d2e8270 con 0x56409d470ec0 2024-05-10T04:06:24.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.935+0000 7f01d17fa700 1 -- 172.21.15.150:0/3863499935 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01e001b430 con 0x56409d470ec0 2024-05-10T04:06:24.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.935+0000 7f020038cb80 10 client.25868 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:24.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.935+0000 7f020038cb80 10 client.25868 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:06:24.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.935+0000 7f020038cb80 20 client.25868 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:24.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.935+0000 7f020038cb80 1 --2- 172.21.15.150:0/3863499935 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x56409d485470 0x56409d4878c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.935+0000 7f020038cb80 4 client.25868 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:24.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.935+0000 7f020038cb80 1 -- 172.21.15.150:0/3863499935 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56409d43e780 con 0x56409d485470 2024-05-10T04:06:24.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.936+0000 7f01edde4700 1 --2- 172.21.15.150:0/3863499935 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x56409d485470 0x56409d4878c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.940+0000 7f01edde4700 1 --2- 172.21.15.150:0/3863499935 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x56409d485470 0x56409d4878c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3fc9558576f6cdd1 server_cookie=8e7cc651098f241d in_seq=0 out_seq=0 2024-05-10T04:06:24.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.940+0000 7f01d17fa700 10 client.25868 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:06:24.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.942+0000 7f01d17fa700 1 -- 172.21.15.150:0/3863499935 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56409d43e780 con 0x56409d485470 2024-05-10T04:06:24.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.942+0000 7f01d17fa700 10 client.25868 handle_command_reply: tid=0 2024-05-10T04:06:24.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.942+0000 7f020038cb80 1 client.25868 shutdown 2024-05-10T04:06:24.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.942+0000 7f020038cb80 1 -- 172.21.15.150:0/3863499935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470ec0 msgr2=0x56409d480bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.942+0000 7f020038cb80 1 --2- 172.21.15.150:0/3863499935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470ec0 0x56409d480bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f01e0003c80 tx=0x7f01e000b730 comp rx=0 tx=0).stop 2024-05-10T04:06:24.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.942+0000 7f020038cb80 1 -- 172.21.15.150:0/3863499935 shutdown_connections 2024-05-10T04:06:24.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.942+0000 7f020038cb80 1 --2- 172.21.15.150:0/3863499935 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x56409d485470 0x56409d4878c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.942+0000 7f020038cb80 1 --2- 172.21.15.150:0/3863499935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470ec0 0x56409d480bf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.942+0000 7f020038cb80 1 --2- 172.21.15.150:0/3863499935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d479750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.942+0000 7f020038cb80 1 --2- 172.21.15.150:0/3863499935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470570 0x56409d479c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.942+0000 7f020038cb80 1 -- 172.21.15.150:0/3863499935 >> 172.21.15.150:0/3863499935 conn(0x56409d3d1a60 msgr2=0x56409d469fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.943+0000 7f01d17fa700 0 client.25868 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:06:24.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.946+0000 7f020038cb80 1 -- 172.21.15.150:0/3863499935 shutdown_connections 2024-05-10T04:06:24.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.946+0000 7f020038cb80 1 -- 172.21.15.150:0/3863499935 wait complete. 2024-05-10T04:06:24.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.946+0000 7f020038cb80 20 client.25868 trim_cache size 0 max 16384 2024-05-10T04:06:24.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.976+0000 7f020038cb80 1 Processor -- start 2024-05-10T04:06:24.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.977+0000 7f020038cb80 1 -- start start 2024-05-10T04:06:24.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.977+0000 7f020038cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d47eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.977+0000 7f020038cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470570 0x56409d47f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.977+0000 7f020038cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470ec0 0x56409d487480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.977+0000 7f020038cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409d481650 con 0x56409d46f6a0 2024-05-10T04:06:24.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.977+0000 7f020038cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409d46b770 con 0x56409d470570 2024-05-10T04:06:24.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.977+0000 7f020038cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409d474da0 con 0x56409d470ec0 2024-05-10T04:06:24.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.978+0000 7f01edde4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d47eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.978+0000 7f01edde4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d47eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59346/0 (socket says 172.21.15.150:59346) 2024-05-10T04:06:24.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.978+0000 7f01edde4700 1 -- 172.21.15.150:0/1103256072 learned_addr learned my addr 172.21.15.150:0/1103256072 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:24.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.978+0000 7f01ee5e5700 1 --2- 172.21.15.150:0/1103256072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470ec0 0x56409d487480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.979+0000 7f01ed5e3700 1 --2- 172.21.15.150:0/1103256072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470570 0x56409d47f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.979+0000 7f01edde4700 1 -- 172.21.15.150:0/1103256072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470ec0 msgr2=0x56409d487480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.979+0000 7f01edde4700 1 --2- 172.21.15.150:0/1103256072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470ec0 0x56409d487480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.979+0000 7f01edde4700 1 -- 172.21.15.150:0/1103256072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470570 msgr2=0x56409d47f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.979+0000 7f01edde4700 1 --2- 172.21.15.150:0/1103256072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470570 0x56409d47f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.979+0000 7f01edde4700 1 -- 172.21.15.150:0/1103256072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56409d46afb0 con 0x56409d46f6a0 2024-05-10T04:06:24.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.980+0000 7f01edde4700 1 --2- 172.21.15.150:0/1103256072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d47eba0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f01d800b750 tx=0x7f01d800b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dcf3f74ebb617 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.980+0000 7f01dcff9700 1 -- 172.21.15.150:0/1103256072 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8016040 con 0x56409d46f6a0 2024-05-10T04:06:24.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.981+0000 7f01dcff9700 1 -- 172.21.15.150:0/1103256072 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01d801d070 con 0x56409d46f6a0 2024-05-10T04:06:24.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.981+0000 7f01dcff9700 1 -- 172.21.15.150:0/1103256072 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80116a0 con 0x56409d46f6a0 2024-05-10T04:06:24.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.982+0000 7f020038cb80 1 -- 172.21.15.150:0/1103256072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 msgr2=0x56409d47eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.982+0000 7f020038cb80 1 --2- 172.21.15.150:0/1103256072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d47eba0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f01d800b750 tx=0x7f01d800b6e0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.983+0000 7f020038cb80 1 -- 172.21.15.150:0/1103256072 shutdown_connections 2024-05-10T04:06:24.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.984+0000 7f020038cb80 1 --2- 172.21.15.150:0/1103256072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470ec0 0x56409d487480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.984+0000 7f020038cb80 1 --2- 172.21.15.150:0/1103256072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d47eba0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.984+0000 7f020038cb80 1 --2- 172.21.15.150:0/1103256072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470570 0x56409d47f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.984+0000 7f020038cb80 1 -- 172.21.15.150:0/1103256072 >> 172.21.15.150:0/1103256072 conn(0x56409d3d1a60 msgr2=0x56409d477d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.984+0000 7f020038cb80 1 -- 172.21.15.150:0/1103256072 shutdown_connections 2024-05-10T04:06:24.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.984+0000 7f020038cb80 1 -- 172.21.15.150:0/1103256072 wait complete. 2024-05-10T04:06:24.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.985+0000 7f020038cb80 1 -- start start 2024-05-10T04:06:24.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.985+0000 7f020038cb80 1 Processor -- start 2024-05-10T04:06:24.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.985+0000 7f020038cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d47e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.985+0000 7f020038cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470570 0x56409d47e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.985+0000 7f020038cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470ec0 0x56409d4d9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.985+0000 7f020038cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409d4da010 con 0x56409d46f6a0 2024-05-10T04:06:24.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.985+0000 7f020038cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409d4da150 con 0x56409d470570 2024-05-10T04:06:24.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.985+0000 7f020038cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409d4da310 con 0x56409d470ec0 2024-05-10T04:06:24.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.986+0000 7f020038cb80 10 client.0 _ll_register_callbacks cb 0x56409d334b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:24.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.986+0000 7f01edde4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d47e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.986+0000 7f01edde4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d47e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59350/0 (socket says 172.21.15.150:59350) 2024-05-10T04:06:24.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.986+0000 7f01edde4700 1 -- 172.21.15.150:0/3130922760 learned_addr learned my addr 172.21.15.150:0/3130922760 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:24.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.986+0000 7f01ee5e5700 1 --2- 172.21.15.150:0/3130922760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470ec0 0x56409d4d9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.986+0000 7f01ed5e3700 1 --2- 172.21.15.150:0/3130922760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470570 0x56409d47e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.987+0000 7f01edde4700 1 -- 172.21.15.150:0/3130922760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470ec0 msgr2=0x56409d4d9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.987+0000 7f01edde4700 1 --2- 172.21.15.150:0/3130922760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470ec0 0x56409d4d9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.987+0000 7f01edde4700 1 -- 172.21.15.150:0/3130922760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470570 msgr2=0x56409d47e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:24.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.987+0000 7f01edde4700 1 --2- 172.21.15.150:0/3130922760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470570 0x56409d47e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.987+0000 7f01edde4700 1 -- 172.21.15.150:0/3130922760 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d8019be0 con 0x56409d46f6a0 2024-05-10T04:06:24.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.988+0000 7f01edde4700 1 --2- 172.21.15.150:0/3130922760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d47e350 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f01d80040b0 tx=0x7f01d8003ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cda1a5ab288f106 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:24.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.988+0000 7f01d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:24.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.988+0000 7f01d17fa700 1 -- 172.21.15.150:0/3130922760 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8019330 con 0x56409d46f6a0 2024-05-10T04:06:24.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.988+0000 7f01d17fa700 1 -- 172.21.15.150:0/3130922760 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f01d8019490 con 0x56409d46f6a0 2024-05-10T04:06:24.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.988+0000 7f020038cb80 1 -- 172.21.15.150:0/3130922760 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01e40d2e70 con 0x56409d46f6a0 2024-05-10T04:06:24.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.988+0000 7f01d17fa700 1 -- 172.21.15.150:0/3130922760 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01d8019670 con 0x56409d46f6a0 2024-05-10T04:06:24.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.988+0000 7f020038cb80 10 client.16206 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:24.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.988+0000 7f020038cb80 1 -- 172.21.15.150:0/3130922760 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56409d355560 con 0x56409d46f6a0 2024-05-10T04:06:24.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.989+0000 7f01d17fa700 1 -- 172.21.15.150:0/3130922760 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01e40d2e70 con 0x56409d46f6a0 2024-05-10T04:06:24.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.989+0000 7f01d17fa700 1 -- 172.21.15.150:0/3130922760 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01d8019050 con 0x56409d46f6a0 2024-05-10T04:06:24.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.989+0000 7f020038cb80 10 client.16206 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:24.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.989+0000 7f020038cb80 10 client.16206 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:06:24.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.989+0000 7f020038cb80 20 client.16206 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:24.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.989+0000 7f020038cb80 1 --2- 172.21.15.150:0/3130922760 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x56409d4dbbd0 0x56409d4ddff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:24.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.989+0000 7f020038cb80 4 client.16206 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:06:24.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.989+0000 7f020038cb80 1 -- 172.21.15.150:0/3130922760 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56409d387ec0 con 0x56409d4dbbd0 2024-05-10T04:06:24.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.990+0000 7f01ed5e3700 1 --2- 172.21.15.150:0/3130922760 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x56409d4dbbd0 0x56409d4ddff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.991+0000 7f01ed5e3700 1 --2- 172.21.15.150:0/3130922760 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x56409d4dbbd0 0x56409d4ddff0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5706e96d3173e205 server_cookie=7f166f29b7606abf in_seq=0 out_seq=0 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.991+0000 7f01d17fa700 10 client.16206 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.996+0000 7f01d17fa700 1 -- 172.21.15.150:0/3130922760 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56409d387ec0 con 0x56409d4dbbd0 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.996+0000 7f01d17fa700 10 client.16206 handle_command_reply: tid=0 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.996+0000 7f020038cb80 1 client.16206 shutdown 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.996+0000 7f020038cb80 1 -- 172.21.15.150:0/3130922760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 msgr2=0x56409d47e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.996+0000 7f020038cb80 1 --2- 172.21.15.150:0/3130922760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d47e350 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f01d80040b0 tx=0x7f01d8003ac0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.997+0000 7f020038cb80 1 -- 172.21.15.150:0/3130922760 shutdown_connections 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.997+0000 7f020038cb80 1 --2- 172.21.15.150:0/3130922760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56409d470ec0 0x56409d4d9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.997+0000 7f020038cb80 1 --2- 172.21.15.150:0/3130922760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409d46f6a0 0x56409d47e350 secure :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f01d80040b0 tx=0x7f01d8003ac0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.997+0000 7f020038cb80 1 --2- 172.21.15.150:0/3130922760 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x56409d4dbbd0 0x56409d4ddff0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.997+0000 7f020038cb80 1 --2- 172.21.15.150:0/3130922760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56409d470570 0x56409d47e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.997+0000 7f020038cb80 1 -- 172.21.15.150:0/3130922760 >> 172.21.15.150:0/3130922760 conn(0x56409d3d1a60 msgr2=0x56409d487480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:24.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.997+0000 7f01d17fa700 0 client.16206 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:06:25.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.997+0000 7f020038cb80 1 -- 172.21.15.150:0/3130922760 shutdown_connections 2024-05-10T04:06:25.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:24.997+0000 7f020038cb80 1 -- 172.21.15.150:0/3130922760 wait complete. 2024-05-10T04:06:25.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.003+0000 7f020038cb80 20 client.16206 trim_cache size 0 max 16384 2024-05-10T04:06:25.005 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:06:25.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.004+0000 7f01f0048700 1 -- 172.21.15.150:0/1180332181 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f01d40a1290 msgr2=0x7f01d40a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:25.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.004+0000 7f01f0048700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f01d40a1290 0x7f01d40a37a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009550 tx=0x7f01e0007ae0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.004+0000 7f01f0048700 1 -- 172.21.15.150:0/1180332181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e8109300 msgr2=0x7f01e8115890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:25.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.004+0000 7f01f0048700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e8109300 0x7f01e8115890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f01e4006fd0 tx=0x7f01e400b730 comp rx=0 tx=0).stop 2024-05-10T04:06:25.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.004+0000 7f01f0048700 1 -- 172.21.15.150:0/1180332181 shutdown_connections 2024-05-10T04:06:25.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.004+0000 7f01f0048700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f01d40a1290 0x7f01d40a37a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.004+0000 7f01f0048700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01e8109c50 0x7f01e81a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.004+0000 7f01f0048700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01e8108630 0x7f01e8115350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.004+0000 7f01f0048700 1 --2- 172.21.15.150:0/1180332181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01e8109300 0x7f01e8115890 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.004+0000 7f01f0048700 1 -- 172.21.15.150:0/1180332181 >> 172.21.15.150:0/1180332181 conn(0x7f01e806ea00 msgr2=0x7f01e810a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:25.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.005+0000 7f01f0048700 1 -- 172.21.15.150:0/1180332181 shutdown_connections 2024-05-10T04:06:25.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.005+0000 7f01f0048700 1 -- 172.21.15.150:0/1180332181 wait complete. 2024-05-10T04:06:25.046 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T04:06:25.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.366+0000 7f03949d2700 1 -- 172.21.15.150:0/1765025178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03901070c0 msgr2=0x7f03901094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:25.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.366+0000 7f03949d2700 1 --2- 172.21.15.150:0/1765025178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03901070c0 0x7f03901094e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f03840098f0 tx=0x7f0384009c00 comp rx=0 tx=0).stop 2024-05-10T04:06:25.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.367+0000 7f03949d2700 1 -- 172.21.15.150:0/1765025178 shutdown_connections 2024-05-10T04:06:25.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.367+0000 7f03949d2700 1 --2- 172.21.15.150:0/1765025178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0390109ab0 0x7f039010bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.367+0000 7f03949d2700 1 --2- 172.21.15.150:0/1765025178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0390069030 0x7f0390106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.367+0000 7f03949d2700 1 --2- 172.21.15.150:0/1765025178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03901070c0 0x7f03901094e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.367+0000 7f03949d2700 1 -- 172.21.15.150:0/1765025178 >> 172.21.15.150:0/1765025178 conn(0x7f03900fc0d0 msgr2=0x7f03900fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:25.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.368+0000 7f03949d2700 1 -- 172.21.15.150:0/1765025178 shutdown_connections 2024-05-10T04:06:25.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.368+0000 7f03949d2700 1 -- 172.21.15.150:0/1765025178 wait complete. 2024-05-10T04:06:25.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.370+0000 7f03949d2700 1 Processor -- start 2024-05-10T04:06:25.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.370+0000 7f03949d2700 1 -- start start 2024-05-10T04:06:25.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.370+0000 7f03949d2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0390069030 0x7f0390114500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.370+0000 7f03949d2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03901070c0 0x7f0390114a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.370+0000 7f03949d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0390109ab0 0x7f0390111050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.370+0000 7f03949d2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0390111700 con 0x7f0390069030 2024-05-10T04:06:25.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.370+0000 7f03949d2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0390111870 con 0x7f03901070c0 2024-05-10T04:06:25.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.370+0000 7f03949d2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0390111a30 con 0x7f0390109ab0 2024-05-10T04:06:25.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.370+0000 7f038e59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0390069030 0x7f0390114500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.370+0000 7f038e59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0390069030 0x7f0390114500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59374/0 (socket says 172.21.15.150:59374) 2024-05-10T04:06:25.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.370+0000 7f038e59c700 1 -- 172.21.15.150:0/3279227429 learned_addr learned my addr 172.21.15.150:0/3279227429 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:25.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.371+0000 7f038dd9b700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03901070c0 0x7f0390114a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.371+0000 7f038ed9d700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0390109ab0 0x7f0390111050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.371+0000 7f038ed9d700 1 -- 172.21.15.150:0/3279227429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03901070c0 msgr2=0x7f0390114a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:25.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.371+0000 7f038ed9d700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03901070c0 0x7f0390114a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.371+0000 7f038ed9d700 1 -- 172.21.15.150:0/3279227429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0390069030 msgr2=0x7f0390114500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:25.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.371+0000 7f038ed9d700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0390069030 0x7f0390114500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.371+0000 7f038ed9d700 1 -- 172.21.15.150:0/3279227429 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03800095f0 con 0x7f0390109ab0 2024-05-10T04:06:25.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.372+0000 7f038ed9d700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0390109ab0 0x7f0390111050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0388007f10 tx=0x7f038800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:25.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.372+0000 7f037f7fe700 1 -- 172.21.15.150:0/3279227429 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388004db0 con 0x7f0390109ab0 2024-05-10T04:06:25.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.372+0000 7f037f7fe700 1 -- 172.21.15.150:0/3279227429 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f038800f040 con 0x7f0390109ab0 2024-05-10T04:06:25.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.372+0000 7f03949d2700 1 -- 172.21.15.150:0/3279227429 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0384009520 con 0x7f0390109ab0 2024-05-10T04:06:25.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.372+0000 7f037f7fe700 1 -- 172.21.15.150:0/3279227429 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388005990 con 0x7f0390109ab0 2024-05-10T04:06:25.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.372+0000 7f03949d2700 1 -- 172.21.15.150:0/3279227429 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03901af1d0 con 0x7f0390109ab0 2024-05-10T04:06:25.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.376+0000 7f037f7fe700 1 -- 172.21.15.150:0/3279227429 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f038800db20 con 0x7f0390109ab0 2024-05-10T04:06:25.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.377+0000 7f037f7fe700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03780a1290 0x7f03780a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.378+0000 7f037f7fe700 1 -- 172.21.15.150:0/3279227429 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f03880cc420 con 0x7f0390109ab0 2024-05-10T04:06:25.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.378+0000 7f038e59c700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03780a1290 0x7f03780a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.379+0000 7f038e59c700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03780a1290 0x7f03780a37a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0380009f60 tx=0x7f0380009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:25.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.382+0000 7f03a4d16b80 1 Processor -- start 2024-05-10T04:06:25.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.383+0000 7f03a4d16b80 1 -- start start 2024-05-10T04:06:25.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.383+0000 7f03a4d16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996dee76a0 0x55996dee7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.383+0000 7f03a4d16b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8570 0x55996dee8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.383+0000 7f03a4d16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8ec0 0x55996deef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.383+0000 7f03a4d16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996de0e150 con 0x55996dee76a0 2024-05-10T04:06:25.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.383+0000 7f03a4d16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996de0bb10 con 0x55996dee8ec0 2024-05-10T04:06:25.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.383+0000 7f03a4d16b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996de07400 con 0x55996dee8570 2024-05-10T04:06:25.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.384+0000 7f038e59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996dee76a0 0x55996dee7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.384+0000 7f038e59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996dee76a0 0x55996dee7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59380/0 (socket says 172.21.15.150:59380) 2024-05-10T04:06:25.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.384+0000 7f038e59c700 1 -- 172.21.15.150:0/3807154506 learned_addr learned my addr 172.21.15.150:0/3807154506 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:25.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.384+0000 7f038dd9b700 1 --2- 172.21.15.150:0/3807154506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8570 0x55996dee8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.384+0000 7f038ed9d700 1 --2- 172.21.15.150:0/3807154506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8ec0 0x55996deef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.384+0000 7f038e59c700 1 -- 172.21.15.150:0/3807154506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8570 msgr2=0x55996dee8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:25.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.384+0000 7f038e59c700 1 --2- 172.21.15.150:0/3807154506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8570 0x55996dee8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.384+0000 7f038e59c700 1 -- 172.21.15.150:0/3807154506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8ec0 msgr2=0x55996deef7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:25.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.384+0000 7f038e59c700 1 --2- 172.21.15.150:0/3807154506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8ec0 0x55996deef7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.384+0000 7f038e59c700 1 -- 172.21.15.150:0/3807154506 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55996dc883b0 con 0x55996dee76a0 2024-05-10T04:06:25.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.385+0000 7f038e59c700 1 --2- 172.21.15.150:0/3807154506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996dee76a0 0x55996dee7af0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f03800043e0 tx=0x7f0380014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a7e13e42cde195 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:25.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.385+0000 7f037cff9700 1 -- 172.21.15.150:0/3807154506 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380003d70 con 0x55996dee76a0 2024-05-10T04:06:25.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.385+0000 7f037cff9700 1 -- 172.21.15.150:0/3807154506 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03800268c0 con 0x55996dee76a0 2024-05-10T04:06:25.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.385+0000 7f037cff9700 1 -- 172.21.15.150:0/3807154506 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038001e830 con 0x55996dee76a0 2024-05-10T04:06:25.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.387+0000 7f03a4d16b80 1 -- 172.21.15.150:0/3807154506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996dee76a0 msgr2=0x55996dee7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:25.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.387+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/3807154506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996dee76a0 0x55996dee7af0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f03800043e0 tx=0x7f0380014000 comp rx=0 tx=0).stop 2024-05-10T04:06:25.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.387+0000 7f03a4d16b80 1 -- 172.21.15.150:0/3807154506 shutdown_connections 2024-05-10T04:06:25.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.387+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/3807154506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8ec0 0x55996deef7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.387+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/3807154506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996dee76a0 0x55996dee7af0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.387+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/3807154506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8570 0x55996dee8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.387+0000 7f03a4d16b80 1 -- 172.21.15.150:0/3807154506 >> 172.21.15.150:0/3807154506 conn(0x55996de49a60 msgr2=0x55996dee1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:25.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.387+0000 7f03a4d16b80 1 -- 172.21.15.150:0/3807154506 shutdown_connections 2024-05-10T04:06:25.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.387+0000 7f03a4d16b80 1 -- 172.21.15.150:0/3807154506 wait complete. 2024-05-10T04:06:25.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.388+0000 7f03a4d16b80 1 -- start start 2024-05-10T04:06:25.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.389+0000 7f03a4d16b80 1 Processor -- start 2024-05-10T04:06:25.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.394+0000 7f03a4d16b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8570 0x55996def18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.394+0000 7f03a4d16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8ec0 0x55996def1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.394+0000 7f03a4d16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996def9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.394+0000 7f03a4d16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996dda3560 con 0x55996def6d50 2024-05-10T04:06:25.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.394+0000 7f03a4d16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996dda3200 con 0x55996dee8ec0 2024-05-10T04:06:25.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.394+0000 7f03a4d16b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996dc8a6a0 con 0x55996dee8570 2024-05-10T04:06:25.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.394+0000 7f03a4d16b80 10 client.0 _ll_register_callbacks cb 0x55996ddacb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:25.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.394+0000 7f038ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996def9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.394+0000 7f038ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996def9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59388/0 (socket says 172.21.15.150:59388) 2024-05-10T04:06:25.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.394+0000 7f038ed9d700 1 -- 172.21.15.150:0/2230453894 learned_addr learned my addr 172.21.15.150:0/2230453894 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:25.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.395+0000 7f038e59c700 1 --2- 172.21.15.150:0/2230453894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8570 0x55996def18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.395+0000 7f038ed9d700 1 -- 172.21.15.150:0/2230453894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8570 msgr2=0x55996def18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:25.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.395+0000 7f038ed9d700 1 --2- 172.21.15.150:0/2230453894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8570 0x55996def18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.395+0000 7f038ed9d700 1 -- 172.21.15.150:0/2230453894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8ec0 msgr2=0x55996def1ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:06:25.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.395+0000 7f038ed9d700 1 --2- 172.21.15.150:0/2230453894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8ec0 0x55996def1ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.395+0000 7f038ed9d700 1 -- 172.21.15.150:0/2230453894 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0380017070 con 0x55996def6d50 2024-05-10T04:06:25.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.396+0000 7f038ed9d700 1 --2- 172.21.15.150:0/2230453894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996def9170 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0388000f80 tx=0x7f03880053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430a019dffb99d09 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:25.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.396+0000 7f0375ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:25.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.396+0000 7f0375ffb700 1 -- 172.21.15.150:0/2230453894 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038809d170 con 0x55996def6d50 2024-05-10T04:06:25.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.396+0000 7f0375ffb700 1 -- 172.21.15.150:0/2230453894 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0388099500 con 0x55996def6d50 2024-05-10T04:06:25.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.396+0000 7f03a4d16b80 1 -- 172.21.15.150:0/2230453894 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55996dd60270 con 0x55996def6d50 2024-05-10T04:06:25.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.396+0000 7f0375ffb700 1 -- 172.21.15.150:0/2230453894 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f038809d330 con 0x55996def6d50 2024-05-10T04:06:25.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.396+0000 7f03a4d16b80 10 client.16218 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:25.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.396+0000 7f03a4d16b80 1 -- 172.21.15.150:0/2230453894 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55996dc34b60 con 0x55996def6d50 2024-05-10T04:06:25.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.397+0000 7f0375ffb700 1 -- 172.21.15.150:0/2230453894 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03880b4020 con 0x55996def6d50 2024-05-10T04:06:25.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.397+0000 7f0375ffb700 1 -- 172.21.15.150:0/2230453894 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0388099760 con 0x55996def6d50 2024-05-10T04:06:25.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.397+0000 7f03a4d16b80 10 client.16218 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:25.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.397+0000 7f03a4d16b80 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:25.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.397+0000 7f03a4d16b80 20 client.16218 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:25.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.397+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/2230453894 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55996defdb00 0x55996defff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.397+0000 7f03a4d16b80 4 client.16218 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:25.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.397+0000 7f03a4d16b80 1 -- 172.21.15.150:0/2230453894 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55996deb6780 con 0x55996defdb00 2024-05-10T04:06:25.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.398+0000 7f038e59c700 1 --2- 172.21.15.150:0/2230453894 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55996defdb00 0x55996defff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.399+0000 7f0375ffb700 10 client.16218 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:25.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.399+0000 7f038e59c700 1 --2- 172.21.15.150:0/2230453894 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55996defdb00 0x55996defff50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=695e1a061c864c0f server_cookie=2ce68dd6753a74dc in_seq=0 out_seq=0 2024-05-10T04:06:25.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.400+0000 7f0375ffb700 1 -- 172.21.15.150:0/2230453894 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55996deb6780 con 0x55996defdb00 2024-05-10T04:06:25.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.400+0000 7f0375ffb700 10 client.16218 handle_command_reply: tid=0 2024-05-10T04:06:25.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.400+0000 7f03a4d16b80 1 client.16218 shutdown 2024-05-10T04:06:25.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.402+0000 7f03a4d16b80 1 -- 172.21.15.150:0/2230453894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 msgr2=0x55996def9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:25.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.402+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/2230453894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996def9170 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0388000f80 tx=0x7f03880053b0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.407+0000 7f03a4d16b80 1 -- 172.21.15.150:0/2230453894 shutdown_connections 2024-05-10T04:06:25.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.407+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/2230453894 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55996defdb00 0x55996defff50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.407+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/2230453894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996def9170 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.407+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/2230453894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8570 0x55996def18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.407+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/2230453894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8ec0 0x55996def1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.407+0000 7f03a4d16b80 1 -- 172.21.15.150:0/2230453894 >> 172.21.15.150:0/2230453894 conn(0x55996de49a60 msgr2=0x55996dee1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:25.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.407+0000 7f0375ffb700 0 client.16218 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:25.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.409+0000 7f03a4d16b80 1 -- 172.21.15.150:0/2230453894 shutdown_connections 2024-05-10T04:06:25.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.409+0000 7f03a4d16b80 1 -- 172.21.15.150:0/2230453894 wait complete. 2024-05-10T04:06:25.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.413+0000 7f03a4d16b80 20 client.16218 trim_cache size 0 max 16384 2024-05-10T04:06:25.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.448+0000 7f03a4d16b80 1 Processor -- start 2024-05-10T04:06:25.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.448+0000 7f03a4d16b80 1 -- start start 2024-05-10T04:06:25.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.448+0000 7f03a4d16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8570 0x55996def9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.448+0000 7f03a4d16b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8ec0 0x55996def9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.448+0000 7f03a4d16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996deeb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.448+0000 7f03a4d16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996def9f80 con 0x55996def6d50 2024-05-10T04:06:25.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.448+0000 7f03a4d16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996defa0f0 con 0x55996dee8570 2024-05-10T04:06:25.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.448+0000 7f03a4d16b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996deebce0 con 0x55996dee8ec0 2024-05-10T04:06:25.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.449+0000 7f038e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8570 0x55996def9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.449+0000 7f038e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8570 0x55996def9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:38168/0 (socket says 172.21.15.150:38168) 2024-05-10T04:06:25.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.449+0000 7f038e59c700 1 -- 172.21.15.150:0/1185893428 learned_addr learned my addr 172.21.15.150:0/1185893428 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:25.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.449+0000 7f038dd9b700 1 --2- 172.21.15.150:0/1185893428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8ec0 0x55996def9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.450+0000 7f038e59c700 1 -- 172.21.15.150:0/1185893428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8ec0 msgr2=0x55996def9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:25.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.450+0000 7f038e59c700 1 --2- 172.21.15.150:0/1185893428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8ec0 0x55996def9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.450+0000 7f038e59c700 1 -- 172.21.15.150:0/1185893428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 msgr2=0x55996deeb7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:06:25.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.450+0000 7f038e59c700 1 --2- 172.21.15.150:0/1185893428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996deeb7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.450+0000 7f038e59c700 1 -- 172.21.15.150:0/1185893428 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55996ddcd560 con 0x55996dee8570 2024-05-10T04:06:25.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.450+0000 7f038e59c700 1 --2- 172.21.15.150:0/1185893428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8570 0x55996def9170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55996dee7380 tx=0x7f0380003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=466a64ff4c70fd89 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:25.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.457+0000 7f037d7fa700 1 -- 172.21.15.150:0/1185893428 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038002a040 con 0x55996dee8570 2024-05-10T04:06:25.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.458+0000 7f037d7fa700 1 -- 172.21.15.150:0/1185893428 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f038002b040 con 0x55996dee8570 2024-05-10T04:06:25.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.458+0000 7f037d7fa700 1 -- 172.21.15.150:0/1185893428 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380005330 con 0x55996dee8570 2024-05-10T04:06:25.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.461+0000 7f03a4d16b80 1 -- 172.21.15.150:0/1185893428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8570 msgr2=0x55996def9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:25.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.461+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/1185893428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8570 0x55996def9170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55996dee7380 tx=0x7f0380003a10 comp rx=0 tx=0).stop 2024-05-10T04:06:25.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.465+0000 7f03a4d16b80 1 -- 172.21.15.150:0/1185893428 shutdown_connections 2024-05-10T04:06:25.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.465+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/1185893428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996deeb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.465+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/1185893428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8570 0x55996def9170 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.465+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/1185893428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8ec0 0x55996def9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.465+0000 7f03a4d16b80 1 -- 172.21.15.150:0/1185893428 >> 172.21.15.150:0/1185893428 conn(0x55996de49a60 msgr2=0x55996dee7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:25.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.466+0000 7f03a4d16b80 1 -- 172.21.15.150:0/1185893428 shutdown_connections 2024-05-10T04:06:25.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.472+0000 7f03a4d16b80 1 -- 172.21.15.150:0/1185893428 wait complete. 2024-05-10T04:06:25.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.472+0000 7f03a4d16b80 1 -- start start 2024-05-10T04:06:25.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.472+0000 7f03a4d16b80 1 Processor -- start 2024-05-10T04:06:25.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.473+0000 7f03a4d16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8570 0x55996df0c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.473+0000 7f03a4d16b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8ec0 0x55996df0cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.473+0000 7f03a4d16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996df52b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.473+0000 7f03a4d16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996df0d390 con 0x55996def6d50 2024-05-10T04:06:25.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.473+0000 7f03a4d16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996df53040 con 0x55996dee8570 2024-05-10T04:06:25.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.473+0000 7f03a4d16b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996df53180 con 0x55996dee8ec0 2024-05-10T04:06:25.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.473+0000 7f03a4d16b80 10 client.0 _ll_register_callbacks cb 0x55996ddacb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:25.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.476+0000 7f038ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996df52b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.476+0000 7f038ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996df52b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59400/0 (socket says 172.21.15.150:59400) 2024-05-10T04:06:25.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.476+0000 7f038ed9d700 1 -- 172.21.15.150:0/1043884238 learned_addr learned my addr 172.21.15.150:0/1043884238 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:25.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.477+0000 7f038ed9d700 1 -- 172.21.15.150:0/1043884238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8ec0 msgr2=0x55996df0cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:25.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.477+0000 7f038ed9d700 1 --2- 172.21.15.150:0/1043884238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8ec0 0x55996df0cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.477+0000 7f038ed9d700 1 -- 172.21.15.150:0/1043884238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8570 msgr2=0x55996df0c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:25.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.477+0000 7f038ed9d700 1 --2- 172.21.15.150:0/1043884238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8570 0x55996df0c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.477+0000 7f038ed9d700 1 -- 172.21.15.150:0/1043884238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55996dc883b0 con 0x55996def6d50 2024-05-10T04:06:25.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.477+0000 7f038ed9d700 1 --2- 172.21.15.150:0/1043884238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996df52b00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0388095f20 tx=0x7f0388004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda14c635ab9cef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:25.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.477+0000 7f0375ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:25.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.477+0000 7f0375ffb700 1 -- 172.21.15.150:0/1043884238 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038809d170 con 0x55996def6d50 2024-05-10T04:06:25.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.477+0000 7f0375ffb700 1 -- 172.21.15.150:0/1043884238 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f03880b5030 con 0x55996def6d50 2024-05-10T04:06:25.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.478+0000 7f03a4d16b80 1 -- 172.21.15.150:0/1043884238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03800268c0 con 0x55996def6d50 2024-05-10T04:06:25.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.478+0000 7f0375ffb700 1 -- 172.21.15.150:0/1043884238 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0388098e60 con 0x55996def6d50 2024-05-10T04:06:25.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.480+0000 7f03a4d16b80 10 client.16221 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:25.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.480+0000 7f03a4d16b80 1 -- 172.21.15.150:0/1043884238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55996dc8ccd0 con 0x55996def6d50 2024-05-10T04:06:25.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.480+0000 7f0375ffb700 1 -- 172.21.15.150:0/1043884238 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03800268c0 con 0x55996def6d50 2024-05-10T04:06:25.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.480+0000 7f0375ffb700 1 -- 172.21.15.150:0/1043884238 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f03880a14f0 con 0x55996def6d50 2024-05-10T04:06:25.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.481+0000 7f03a4d16b80 10 client.16221 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:25.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.481+0000 7f03a4d16b80 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:25.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.481+0000 7f03a4d16b80 20 client.16221 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:25.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.481+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/1043884238 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55996df54620 0x55996df56a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:25.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.481+0000 7f03a4d16b80 4 client.16221 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:06:25.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.481+0000 7f03a4d16b80 1 -- 172.21.15.150:0/1043884238 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55996ddffec0 con 0x55996df54620 2024-05-10T04:06:25.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.482+0000 7f038e59c700 1 --2- 172.21.15.150:0/1043884238 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55996df54620 0x55996df56a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:25.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.485+0000 7f038e59c700 1 --2- 172.21.15.150:0/1043884238 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55996df54620 0x55996df56a40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4f820c548825701 server_cookie=4db4de4b840a3aa1 in_seq=0 out_seq=0 2024-05-10T04:06:25.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.485+0000 7f0375ffb700 10 client.16221 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:25.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.485+0000 7f0375ffb700 1 -- 172.21.15.150:0/1043884238 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55996ddffec0 con 0x55996df54620 2024-05-10T04:06:25.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.485+0000 7f0375ffb700 10 client.16221 handle_command_reply: tid=0 2024-05-10T04:06:25.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.485+0000 7f03a4d16b80 1 client.16221 shutdown 2024-05-10T04:06:25.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.487+0000 7f03a4d16b80 1 -- 172.21.15.150:0/1043884238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 msgr2=0x55996df52b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:25.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.487+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/1043884238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996df52b00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0388095f20 tx=0x7f0388004f10 comp rx=0 tx=0).stop 2024-05-10T04:06:25.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.488+0000 7f03a4d16b80 1 -- 172.21.15.150:0/1043884238 shutdown_connections 2024-05-10T04:06:25.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.488+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/1043884238 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55996df54620 0x55996df56a40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.488+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/1043884238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55996def6d50 0x55996df52b00 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.488+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/1043884238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55996dee8570 0x55996df0c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.488+0000 7f03a4d16b80 1 --2- 172.21.15.150:0/1043884238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55996dee8ec0 0x55996df0cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.488+0000 7f03a4d16b80 1 -- 172.21.15.150:0/1043884238 >> 172.21.15.150:0/1043884238 conn(0x55996de49a60 msgr2=0x55996dee7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:25.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.488+0000 7f0375ffb700 0 client.16221 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:25.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.488+0000 7f03a4d16b80 1 -- 172.21.15.150:0/1043884238 shutdown_connections 2024-05-10T04:06:25.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.488+0000 7f03a4d16b80 1 -- 172.21.15.150:0/1043884238 wait complete. 2024-05-10T04:06:25.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.497+0000 7f03a4d16b80 20 client.16221 trim_cache size 0 max 16384 2024-05-10T04:06:25.500 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:06:25.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.503+0000 7f03949d2700 1 -- 172.21.15.150:0/3279227429 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03780a1290 msgr2=0x7f03780a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:25.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.503+0000 7f03949d2700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03780a1290 0x7f03780a37a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0380009f60 tx=0x7f0380009310 comp rx=0 tx=0).stop 2024-05-10T04:06:25.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.503+0000 7f03949d2700 1 -- 172.21.15.150:0/3279227429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0390109ab0 msgr2=0x7f0390111050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:25.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.503+0000 7f03949d2700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0390109ab0 0x7f0390111050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0388007f10 tx=0x7f038800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.506+0000 7f03949d2700 1 -- 172.21.15.150:0/3279227429 shutdown_connections 2024-05-10T04:06:25.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.506+0000 7f03949d2700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f03780a1290 0x7f03780a37a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.506+0000 7f03949d2700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0390109ab0 0x7f0390111050 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.506+0000 7f03949d2700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0390069030 0x7f0390114500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.506+0000 7f03949d2700 1 --2- 172.21.15.150:0/3279227429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03901070c0 0x7f0390114a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:25.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.506+0000 7f03949d2700 1 -- 172.21.15.150:0/3279227429 >> 172.21.15.150:0/3279227429 conn(0x7f03900fc0d0 msgr2=0x7f0390109f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:25.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.507+0000 7f03949d2700 1 -- 172.21.15.150:0/3279227429 shutdown_connections 2024-05-10T04:06:25.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:25.507+0000 7f03949d2700 1 -- 172.21.15.150:0/3279227429 wait complete. 2024-05-10T04:06:26.545 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T04:06:26.545 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T04:06:26.580 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:26 smithi150 ceph-mon[42477]: pgmap v345: 129 pgs: 129 active+clean; 4.4 GiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 317 MiB/s wr, 103 op/s 2024-05-10T04:06:26.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:26 smithi181 ceph-mon[46602]: pgmap v345: 129 pgs: 129 active+clean; 4.4 GiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 317 MiB/s wr, 103 op/s 2024-05-10T04:06:26.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:26 smithi161 ceph-mon[46918]: pgmap v345: 129 pgs: 129 active+clean; 4.4 GiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 317 MiB/s wr, 103 op/s 2024-05-10T04:06:26.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.749+0000 7fbecb7bd700 1 -- 172.21.15.150:0/2260338183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbec4102060 msgr2=0x7fbec4102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:26.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.749+0000 7fbecb7bd700 1 --2- 172.21.15.150:0/2260338183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbec4102060 0x7fbec4102470 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00b0a0 tx=0x7fbebc00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.749+0000 7fbecb7bd700 1 -- 172.21.15.150:0/2260338183 shutdown_connections 2024-05-10T04:06:26.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.749+0000 7fbecb7bd700 1 --2- 172.21.15.150:0/2260338183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbec41016c0 0x7fbec4108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.749+0000 7fbecb7bd700 1 --2- 172.21.15.150:0/2260338183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbec4102060 0x7fbec4102470 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.749+0000 7fbecb7bd700 1 --2- 172.21.15.150:0/2260338183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbec4100d70 0x7fbec4101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.749+0000 7fbecb7bd700 1 -- 172.21.15.150:0/2260338183 >> 172.21.15.150:0/2260338183 conn(0x7fbec40fb8b0 msgr2=0x7fbec40fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:26.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.750+0000 7fbecb7bd700 1 -- 172.21.15.150:0/2260338183 shutdown_connections 2024-05-10T04:06:26.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.750+0000 7fbecb7bd700 1 -- 172.21.15.150:0/2260338183 wait complete. 2024-05-10T04:06:26.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.750+0000 7fbecb7bd700 1 Processor -- start 2024-05-10T04:06:26.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.750+0000 7fbecb7bd700 1 -- start start 2024-05-10T04:06:26.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbecb7bd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbec4100d70 0x7fbec4104a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbecb7bd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbec41016c0 0x7fbec41030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbecb7bd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbec4102060 0x7fbec4103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbecb7bd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec4103ce0 con 0x7fbec4102060 2024-05-10T04:06:26.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbecb7bd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec4103e50 con 0x7fbec4100d70 2024-05-10T04:06:26.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbecb7bd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec41a2120 con 0x7fbec41016c0 2024-05-10T04:06:26.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbec9d5a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbec4102060 0x7fbec4103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:26.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbec9d5a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbec4102060 0x7fbec4103600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59426/0 (socket says 172.21.15.150:59426) 2024-05-10T04:06:26.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbec9d5a700 1 -- 172.21.15.150:0/3225131839 learned_addr learned my addr 172.21.15.150:0/3225131839 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:26.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbec9d5a700 1 -- 172.21.15.150:0/3225131839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbec41016c0 msgr2=0x7fbec41030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:26.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbec9d5a700 1 --2- 172.21.15.150:0/3225131839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbec41016c0 0x7fbec41030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbec9d5a700 1 -- 172.21.15.150:0/3225131839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbec4100d70 msgr2=0x7fbec4104a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T04:06:26.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbec9d5a700 1 --2- 172.21.15.150:0/3225131839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbec4100d70 0x7fbec4104a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbec9d5a700 1 -- 172.21.15.150:0/3225131839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc009d00 con 0x7fbec4102060 2024-05-10T04:06:26.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.751+0000 7fbec9d5a700 1 --2- 172.21.15.150:0/3225131839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbec4102060 0x7fbec4103600 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbec000b4a0 tx=0x7fbec000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:26.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.752+0000 7fbeba7fc700 1 -- 172.21.15.150:0/3225131839 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0011830 con 0x7fbec4102060 2024-05-10T04:06:26.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.752+0000 7fbecb7bd700 1 -- 172.21.15.150:0/3225131839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec41a2380 con 0x7fbec4102060 2024-05-10T04:06:26.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.752+0000 7fbecb7bd700 1 -- 172.21.15.150:0/3225131839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec41a28e0 con 0x7fbec4102060 2024-05-10T04:06:26.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.752+0000 7fbeba7fc700 1 -- 172.21.15.150:0/3225131839 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbec0011e50 con 0x7fbec4102060 2024-05-10T04:06:26.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.752+0000 7fbeba7fc700 1 -- 172.21.15.150:0/3225131839 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0010e70 con 0x7fbec4102060 2024-05-10T04:06:26.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.755+0000 7fbeba7fc700 1 -- 172.21.15.150:0/3225131839 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbec001a440 con 0x7fbec4102060 2024-05-10T04:06:26.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.755+0000 7fbeba7fc700 1 --2- 172.21.15.150:0/3225131839 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbeb00a12c0 0x7fbeb00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.756+0000 7fbeba7fc700 1 -- 172.21.15.150:0/3225131839 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbec00cc650 con 0x7fbec4102060 2024-05-10T04:06:26.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.756+0000 7fbec9559700 1 --2- 172.21.15.150:0/3225131839 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbeb00a12c0 0x7fbeb00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:26.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.756+0000 7fbedbb01b80 1 Processor -- start 2024-05-10T04:06:26.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.756+0000 7fbedbb01b80 1 -- start start 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.756+0000 7fbec9559700 1 --2- 172.21.15.150:0/3225131839 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbeb00a12c0 0x7fbeb00a37d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00b830 tx=0x7fbebc000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbedbb01b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c193f30 0x55dd9c194380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbedbb01b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c194e00 0x55dd9c195210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbedbb01b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c195750 0x55dd9c19c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbedbb01b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd9c0b21d0 con 0x55dd9c195750 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbedbb01b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd9c0afb90 con 0x55dd9c193f30 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbedbb01b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd9c0ab480 con 0x55dd9c194e00 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbec9d5a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c195750 0x55dd9c19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbec9d5a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c195750 0x55dd9c19c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59442/0 (socket says 172.21.15.150:59442) 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbec9d5a700 1 -- 172.21.15.150:0/2776393068 learned_addr learned my addr 172.21.15.150:0/2776393068 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbec9d5a700 1 -- 172.21.15.150:0/2776393068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c194e00 msgr2=0x55dd9c195210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbec9d5a700 1 --2- 172.21.15.150:0/2776393068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c194e00 0x55dd9c195210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbec9d5a700 1 -- 172.21.15.150:0/2776393068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c193f30 msgr2=0x55dd9c194380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:06:26.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbec9d5a700 1 --2- 172.21.15.150:0/2776393068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c193f30 0x55dd9c194380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbec9d5a700 1 -- 172.21.15.150:0/2776393068 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd9bf2c430 con 0x55dd9c195750 2024-05-10T04:06:26.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.757+0000 7fbec9d5a700 1 --2- 172.21.15.150:0/2776393068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c195750 0x55dd9c19c050 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbec0095c30 tx=0x7fbec0099340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb29147e1f948a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:26.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.758+0000 7fbeaf7fe700 1 -- 172.21.15.150:0/2776393068 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0099730 con 0x55dd9c195750 2024-05-10T04:06:26.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.758+0000 7fbeaf7fe700 1 -- 172.21.15.150:0/2776393068 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbec009d560 con 0x55dd9c195750 2024-05-10T04:06:26.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.758+0000 7fbeaf7fe700 1 -- 172.21.15.150:0/2776393068 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec009d120 con 0x55dd9c195750 2024-05-10T04:06:26.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.759+0000 7fbedbb01b80 1 -- 172.21.15.150:0/2776393068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c195750 msgr2=0x55dd9c19c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:26.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.759+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/2776393068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c195750 0x55dd9c19c050 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbec0095c30 tx=0x7fbec0099340 comp rx=0 tx=0).stop 2024-05-10T04:06:26.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.759+0000 7fbedbb01b80 1 -- 172.21.15.150:0/2776393068 shutdown_connections 2024-05-10T04:06:26.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.759+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/2776393068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c195750 0x55dd9c19c050 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.759+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/2776393068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c193f30 0x55dd9c194380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.759+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/2776393068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c194e00 0x55dd9c195210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.759+0000 7fbedbb01b80 1 -- 172.21.15.150:0/2776393068 >> 172.21.15.150:0/2776393068 conn(0x55dd9c0edae0 msgr2=0x55dd9c186060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:26.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.759+0000 7fbedbb01b80 1 -- 172.21.15.150:0/2776393068 shutdown_connections 2024-05-10T04:06:26.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.759+0000 7fbedbb01b80 1 -- 172.21.15.150:0/2776393068 wait complete. 2024-05-10T04:06:26.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.759+0000 7fbedbb01b80 1 -- start start 2024-05-10T04:06:26.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.759+0000 7fbedbb01b80 1 Processor -- start 2024-05-10T04:06:26.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.760+0000 7fbedbb01b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c193f30 0x55dd9c18ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.760+0000 7fbedbb01b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c18d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.760+0000 7fbedbb01b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c195750 0x55dd9c1922a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.760+0000 7fbedbb01b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd9c0475e0 con 0x55dd9c194e00 2024-05-10T04:06:26.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.760+0000 7fbedbb01b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd9c047280 con 0x55dd9c195750 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.760+0000 7fbedbb01b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd9bf2e720 con 0x55dd9c193f30 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.760+0000 7fbedbb01b80 10 client.0 _ll_register_callbacks cb 0x55dd9c050bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.760+0000 7fbec8d58700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c18d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.760+0000 7fbec8d58700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c18d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59446/0 (socket says 172.21.15.150:59446) 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.760+0000 7fbec8d58700 1 -- 172.21.15.150:0/852464136 learned_addr learned my addr 172.21.15.150:0/852464136 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.761+0000 7fbec8d58700 1 -- 172.21.15.150:0/852464136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c193f30 msgr2=0x55dd9c18ce10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.761+0000 7fbec8d58700 1 --2- 172.21.15.150:0/852464136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c193f30 0x55dd9c18ce10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.761+0000 7fbec8d58700 1 -- 172.21.15.150:0/852464136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c195750 msgr2=0x55dd9c1922a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.761+0000 7fbec8d58700 1 --2- 172.21.15.150:0/852464136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c195750 0x55dd9c1922a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.761+0000 7fbec8d58700 1 -- 172.21.15.150:0/852464136 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec41a28e0 con 0x55dd9c194e00 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.761+0000 7fbec8d58700 1 --2- 172.21.15.150:0/852464136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c18d350 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009e80 tx=0x7fbeb400b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9df68566cfdfa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.761+0000 7fbeacff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.761+0000 7fbeacff9700 1 -- 172.21.15.150:0/852464136 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4011420 con 0x55dd9c194e00 2024-05-10T04:06:26.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.761+0000 7fbeacff9700 1 -- 172.21.15.150:0/852464136 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbeb4011580 con 0x55dd9c194e00 2024-05-10T04:06:26.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.761+0000 7fbedbb01b80 1 -- 172.21.15.150:0/852464136 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd9c0042f0 con 0x55dd9c194e00 2024-05-10T04:06:26.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.761+0000 7fbeacff9700 1 -- 172.21.15.150:0/852464136 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbeb4011740 con 0x55dd9c194e00 2024-05-10T04:06:26.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.761+0000 7fbedbb01b80 10 client.16233 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:26.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.761+0000 7fbedbb01b80 1 -- 172.21.15.150:0/852464136 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dd9bed8be0 con 0x55dd9c194e00 2024-05-10T04:06:26.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.762+0000 7fbeacff9700 1 -- 172.21.15.150:0/852464136 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd9c0042f0 con 0x55dd9c194e00 2024-05-10T04:06:26.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.762+0000 7fbeacff9700 1 -- 172.21.15.150:0/852464136 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbeb40199d0 con 0x55dd9c194e00 2024-05-10T04:06:26.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.762+0000 7fbedbb01b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:26.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.762+0000 7fbedbb01b80 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:26.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.762+0000 7fbedbb01b80 20 client.16233 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:26.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.762+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/852464136 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55dd9c1b26a0 0x55dd9c1b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.762+0000 7fbedbb01b80 4 client.16233 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:26.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.762+0000 7fbedbb01b80 1 -- 172.21.15.150:0/852464136 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd9c15a800 con 0x55dd9c1b26a0 2024-05-10T04:06:26.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.763+0000 7fbec9559700 1 --2- 172.21.15.150:0/852464136 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55dd9c1b26a0 0x55dd9c1b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:26.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.764+0000 7fbec9559700 1 --2- 172.21.15.150:0/852464136 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55dd9c1b26a0 0x55dd9c1b4b10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84e3ee958cfbcb92 server_cookie=57ca29edd3e64972 in_seq=0 out_seq=0 2024-05-10T04:06:26.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.764+0000 7fbeacff9700 10 client.16233 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:26.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbeacff9700 1 -- 172.21.15.150:0/852464136 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd9c15a800 con 0x55dd9c1b26a0 2024-05-10T04:06:26.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbeacff9700 10 client.16233 handle_command_reply: tid=0 2024-05-10T04:06:26.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbedbb01b80 1 client.16233 shutdown 2024-05-10T04:06:26.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbedbb01b80 1 -- 172.21.15.150:0/852464136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 msgr2=0x55dd9c18d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:26.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/852464136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c18d350 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009e80 tx=0x7fbeb400b3c0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbedbb01b80 1 -- 172.21.15.150:0/852464136 shutdown_connections 2024-05-10T04:06:26.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/852464136 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55dd9c1b26a0 0x55dd9c1b4b10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/852464136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c195750 0x55dd9c1922a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/852464136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c193f30 0x55dd9c18ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbeacff9700 0 client.16233 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:26.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/852464136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c18d350 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbedbb01b80 1 -- 172.21.15.150:0/852464136 >> 172.21.15.150:0/852464136 conn(0x55dd9c0edae0 msgr2=0x55dd9c186030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:26.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbedbb01b80 1 -- 172.21.15.150:0/852464136 shutdown_connections 2024-05-10T04:06:26.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.766+0000 7fbedbb01b80 1 -- 172.21.15.150:0/852464136 wait complete. 2024-05-10T04:06:26.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.767+0000 7fbedbb01b80 20 client.16233 trim_cache size 0 max 16384 2024-05-10T04:06:26.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.787+0000 7fbedbb01b80 1 Processor -- start 2024-05-10T04:06:26.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.787+0000 7fbedbb01b80 1 -- start start 2024-05-10T04:06:26.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbedbb01b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c0edae0 0x55dd9c185d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbedbb01b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c193f30 0x55dd9c1c2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbedbb01b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c1c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbedbb01b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd9c187290 con 0x55dd9c194e00 2024-05-10T04:06:26.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbedbb01b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd9c1b22d0 con 0x55dd9c0edae0 2024-05-10T04:06:26.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbedbb01b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd9c1947e0 con 0x55dd9c193f30 2024-05-10T04:06:26.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbec9d5a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c1c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:26.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbec9d5a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c1c2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59460/0 (socket says 172.21.15.150:59460) 2024-05-10T04:06:26.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbec9d5a700 1 -- 172.21.15.150:0/1210141644 learned_addr learned my addr 172.21.15.150:0/1210141644 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:26.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbec9559700 1 --2- 172.21.15.150:0/1210141644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c0edae0 0x55dd9c185d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:26.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbec9d5a700 1 -- 172.21.15.150:0/1210141644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c193f30 msgr2=0x55dd9c1c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:26.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbec9d5a700 1 --2- 172.21.15.150:0/1210141644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c193f30 0x55dd9c1c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbec9d5a700 1 -- 172.21.15.150:0/1210141644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c0edae0 msgr2=0x55dd9c185d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:26.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbec9d5a700 1 --2- 172.21.15.150:0/1210141644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c0edae0 0x55dd9c185d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.788+0000 7fbec9d5a700 1 -- 172.21.15.150:0/1210141644 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd9c0715e0 con 0x55dd9c194e00 2024-05-10T04:06:26.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.789+0000 7fbec9d5a700 1 --2- 172.21.15.150:0/1210141644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c1c2ea0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbec00961a0 tx=0x7fbec00c9ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f725e9cbd79cc89e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:26.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.789+0000 7fbeaffff700 1 -- 172.21.15.150:0/1210141644 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0099730 con 0x55dd9c194e00 2024-05-10T04:06:26.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.789+0000 7fbeaffff700 1 -- 172.21.15.150:0/1210141644 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbec00a1bb0 con 0x55dd9c194e00 2024-05-10T04:06:26.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.789+0000 7fbeaffff700 1 -- 172.21.15.150:0/1210141644 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec00104d0 con 0x55dd9c194e00 2024-05-10T04:06:26.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.790+0000 7fbedbb01b80 1 -- 172.21.15.150:0/1210141644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 msgr2=0x55dd9c1c2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:26.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.790+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/1210141644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c1c2ea0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbec00961a0 tx=0x7fbec00c9ef0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.790+0000 7fbedbb01b80 1 -- 172.21.15.150:0/1210141644 shutdown_connections 2024-05-10T04:06:26.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.790+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/1210141644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c1c2ea0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.790+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/1210141644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c0edae0 0x55dd9c185d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.790+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/1210141644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c193f30 0x55dd9c1c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.790+0000 7fbedbb01b80 1 -- 172.21.15.150:0/1210141644 >> 172.21.15.150:0/1210141644 conn(0x55dd9c1824d0 msgr2=0x55dd9c198be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:26.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.790+0000 7fbedbb01b80 1 -- 172.21.15.150:0/1210141644 shutdown_connections 2024-05-10T04:06:26.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.790+0000 7fbedbb01b80 1 -- 172.21.15.150:0/1210141644 wait complete. 2024-05-10T04:06:26.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.791+0000 7fbedbb01b80 1 -- start start 2024-05-10T04:06:26.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.791+0000 7fbedbb01b80 1 Processor -- start 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.791+0000 7fbedbb01b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c0edae0 0x55dd9c203600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.791+0000 7fbedbb01b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c193f30 0x55dd9c205b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.791+0000 7fbedbb01b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c2080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.791+0000 7fbedbb01b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd9c2086c0 con 0x55dd9c194e00 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.791+0000 7fbedbb01b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd9c208830 con 0x55dd9c193f30 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.791+0000 7fbedbb01b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd9c2089f0 con 0x55dd9c0edae0 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.791+0000 7fbedbb01b80 10 client.0 _ll_register_callbacks cb 0x55dd9c050bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.791+0000 7fbec9d5a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c2080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.792+0000 7fbec9d5a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c2080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59466/0 (socket says 172.21.15.150:59466) 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.792+0000 7fbec9d5a700 1 -- 172.21.15.150:0/3279228891 learned_addr learned my addr 172.21.15.150:0/3279228891 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.792+0000 7fbec8d58700 1 --2- 172.21.15.150:0/3279228891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c193f30 0x55dd9c205b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.792+0000 7fbec9559700 1 --2- 172.21.15.150:0/3279228891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c0edae0 0x55dd9c203600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.792+0000 7fbec9d5a700 1 -- 172.21.15.150:0/3279228891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c0edae0 msgr2=0x55dd9c203600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.792+0000 7fbec9d5a700 1 --2- 172.21.15.150:0/3279228891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c0edae0 0x55dd9c203600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.792+0000 7fbec9d5a700 1 -- 172.21.15.150:0/3279228891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c193f30 msgr2=0x55dd9c205b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.792+0000 7fbec9d5a700 1 --2- 172.21.15.150:0/3279228891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c193f30 0x55dd9c205b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.792+0000 7fbec9d5a700 1 -- 172.21.15.150:0/3279228891 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd9bf2c430 con 0x55dd9c194e00 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.792+0000 7fbec9d5a700 1 --2- 172.21.15.150:0/3279228891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c2080a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbec00a1680 tx=0x7fbec0010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c19a0643c897ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.792+0000 7fbeacff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.792+0000 7fbeacff9700 1 -- 172.21.15.150:0/3279228891 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0097ad0 con 0x55dd9c194e00 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.792+0000 7fbeacff9700 1 -- 172.21.15.150:0/3279228891 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbec00104d0 con 0x55dd9c194e00 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.793+0000 7fbedbb01b80 1 -- 172.21.15.150:0/3279228891 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbec009d560 con 0x55dd9c194e00 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.793+0000 7fbeacff9700 1 -- 172.21.15.150:0/3279228891 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbec0010690 con 0x55dd9c194e00 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.793+0000 7fbedbb01b80 10 client.16239 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.793+0000 7fbedbb01b80 1 -- 172.21.15.150:0/3279228891 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dd9bf30d50 con 0x55dd9c194e00 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.793+0000 7fbeacff9700 1 -- 172.21.15.150:0/3279228891 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbec009d560 con 0x55dd9c194e00 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.793+0000 7fbeacff9700 1 -- 172.21.15.150:0/3279228891 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbec0098c20 con 0x55dd9c194e00 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.793+0000 7fbedbb01b80 10 client.16239 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:26.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.793+0000 7fbedbb01b80 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:26.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.793+0000 7fbedbb01b80 20 client.16239 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:26.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.793+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/3279228891 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55dd9c195750 0x55dd9c20beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:26.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.793+0000 7fbedbb01b80 4 client.16239 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T04:06:26.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.793+0000 7fbedbb01b80 1 -- 172.21.15.150:0/3279228891 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dd9c0a3f40 con 0x55dd9c195750 2024-05-10T04:06:26.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.794+0000 7fbec9559700 1 --2- 172.21.15.150:0/3279228891 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55dd9c195750 0x55dd9c20beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:26.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.795+0000 7fbeacff9700 10 client.16239 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:26.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.795+0000 7fbec9559700 1 --2- 172.21.15.150:0/3279228891 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55dd9c195750 0x55dd9c20beb0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d4caff43636607a server_cookie=7b706575326a3b54 in_seq=0 out_seq=0 2024-05-10T04:06:26.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.798+0000 7fbeacff9700 1 -- 172.21.15.150:0/3279228891 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dd9c0a3f40 con 0x55dd9c195750 2024-05-10T04:06:26.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.798+0000 7fbeacff9700 10 client.16239 handle_command_reply: tid=0 2024-05-10T04:06:26.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.798+0000 7fbedbb01b80 1 client.16239 shutdown 2024-05-10T04:06:26.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.799+0000 7fbedbb01b80 1 -- 172.21.15.150:0/3279228891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 msgr2=0x55dd9c2080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:26.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.799+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/3279228891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c2080a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbec00a1680 tx=0x7fbec0010910 comp rx=0 tx=0).stop 2024-05-10T04:06:26.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.799+0000 7fbedbb01b80 1 -- 172.21.15.150:0/3279228891 shutdown_connections 2024-05-10T04:06:26.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.799+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/3279228891 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55dd9c195750 0x55dd9c20beb0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.799+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/3279228891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dd9c194e00 0x55dd9c2080a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.799+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/3279228891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd9c0edae0 0x55dd9c203600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.799+0000 7fbeacff9700 0 client.16239 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:26.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.799+0000 7fbedbb01b80 1 --2- 172.21.15.150:0/3279228891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd9c193f30 0x55dd9c205b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.799+0000 7fbedbb01b80 1 -- 172.21.15.150:0/3279228891 >> 172.21.15.150:0/3279228891 conn(0x55dd9c1824d0 msgr2=0x55dd9c18ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:26.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.799+0000 7fbedbb01b80 1 -- 172.21.15.150:0/3279228891 shutdown_connections 2024-05-10T04:06:26.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.799+0000 7fbedbb01b80 1 -- 172.21.15.150:0/3279228891 wait complete. 2024-05-10T04:06:26.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.799+0000 7fbedbb01b80 20 client.16239 trim_cache size 0 max 16384 2024-05-10T04:06:26.802 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:06:26.803 INFO:teuthology.orchestra.run.smithi150.stdout: "return_code": 0, 2024-05-10T04:06:26.803 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_tag": "f2a72b07-4da3-4b55-8f9c-ed98939a6c85", 2024-05-10T04:06:26.803 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": "asynchronous" 2024-05-10T04:06:26.803 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:06:26.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.800+0000 7fbecb7bd700 1 -- 172.21.15.150:0/3225131839 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbeb00a12c0 msgr2=0x7fbeb00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:26.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.800+0000 7fbecb7bd700 1 --2- 172.21.15.150:0/3225131839 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbeb00a12c0 0x7fbeb00a37d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00b830 tx=0x7fbebc000f40 comp rx=0 tx=0).stop 2024-05-10T04:06:26.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.800+0000 7fbecb7bd700 1 -- 172.21.15.150:0/3225131839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbec4102060 msgr2=0x7fbec4103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:26.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.800+0000 7fbecb7bd700 1 --2- 172.21.15.150:0/3225131839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbec4102060 0x7fbec4103600 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbec000b4a0 tx=0x7fbec000b860 comp rx=0 tx=0).stop 2024-05-10T04:06:26.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.800+0000 7fbecb7bd700 1 -- 172.21.15.150:0/3225131839 shutdown_connections 2024-05-10T04:06:26.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.800+0000 7fbecb7bd700 1 --2- 172.21.15.150:0/3225131839 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fbeb00a12c0 0x7fbeb00a37d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.800+0000 7fbecb7bd700 1 --2- 172.21.15.150:0/3225131839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbec4102060 0x7fbec4103600 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.800+0000 7fbecb7bd700 1 --2- 172.21.15.150:0/3225131839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbec4100d70 0x7fbec4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.800+0000 7fbecb7bd700 1 --2- 172.21.15.150:0/3225131839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbec41016c0 0x7fbec41030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:26.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.800+0000 7fbecb7bd700 1 -- 172.21.15.150:0/3225131839 >> 172.21.15.150:0/3225131839 conn(0x7fbec40fb8b0 msgr2=0x7fbec40ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:26.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.801+0000 7fbecb7bd700 1 -- 172.21.15.150:0/3225131839 shutdown_connections 2024-05-10T04:06:26.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:26.801+0000 7fbecb7bd700 1 -- 172.21.15.150:0/3225131839 wait complete. 2024-05-10T04:06:26.823 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:06:27.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.075+0000 7ff7c4362700 1 -- 172.21.15.150:0/3855424762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7bc1013e0 msgr2=0x7ff7bc1017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:27.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.075+0000 7ff7c4362700 1 --2- 172.21.15.150:0/3855424762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7bc1013e0 0x7ff7bc1017f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009990 tx=0x7ff7ac009ca0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.076+0000 7ff7c4362700 1 -- 172.21.15.150:0/3855424762 shutdown_connections 2024-05-10T04:06:27.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.076+0000 7ff7c4362700 1 --2- 172.21.15.150:0/3855424762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7bc101d80 0x7ff7bc108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.076+0000 7ff7c4362700 1 --2- 172.21.15.150:0/3855424762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7bc1002c0 0x7ff7bc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.076+0000 7ff7c4362700 1 --2- 172.21.15.150:0/3855424762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7bc1013e0 0x7ff7bc1017f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.076+0000 7ff7c4362700 1 -- 172.21.15.150:0/3855424762 >> 172.21.15.150:0/3855424762 conn(0x7ff7bc0fb8f0 msgr2=0x7ff7bc0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:27.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.076+0000 7ff7c4362700 1 -- 172.21.15.150:0/3855424762 shutdown_connections 2024-05-10T04:06:27.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.076+0000 7ff7c4362700 1 -- 172.21.15.150:0/3855424762 wait complete. 2024-05-10T04:06:27.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c4362700 1 Processor -- start 2024-05-10T04:06:27.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c4362700 1 -- start start 2024-05-10T04:06:27.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c4362700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7bc1002c0 0x7ff7bc10b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c4362700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7bc1013e0 0x7ff7bc109e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c4362700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7bc101d80 0x7ff7bc10a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c4362700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc10a9d0 con 0x7ff7bc101d80 2024-05-10T04:06:27.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c4362700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc10ab40 con 0x7ff7bc1013e0 2024-05-10T04:06:27.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c4362700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc10ad00 con 0x7ff7bc1002c0 2024-05-10T04:06:27.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c28ff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7bc101d80 0x7ff7bc10a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:27.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c28ff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7bc101d80 0x7ff7bc10a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59484/0 (socket says 172.21.15.150:59484) 2024-05-10T04:06:27.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c28ff700 1 -- 172.21.15.150:0/3006550429 learned_addr learned my addr 172.21.15.150:0/3006550429 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:27.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c28ff700 1 -- 172.21.15.150:0/3006550429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7bc1002c0 msgr2=0x7ff7bc10b790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T04:06:27.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c28ff700 1 --2- 172.21.15.150:0/3006550429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7bc1002c0 0x7ff7bc10b790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.077+0000 7ff7c28ff700 1 -- 172.21.15.150:0/3006550429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7bc1013e0 msgr2=0x7ff7bc109e30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T04:06:27.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.078+0000 7ff7c28ff700 1 --2- 172.21.15.150:0/3006550429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7bc1013e0 0x7ff7bc109e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.078+0000 7ff7c28ff700 1 -- 172.21.15.150:0/3006550429 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac0095f0 con 0x7ff7bc101d80 2024-05-10T04:06:27.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.078+0000 7ff7c28ff700 1 --2- 172.21.15.150:0/3006550429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7bc101d80 0x7ff7bc10a370 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400b970 tx=0x7ff7b400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:27.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.078+0000 7ff7b37fe700 1 -- 172.21.15.150:0/3006550429 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b400c9b0 con 0x7ff7bc101d80 2024-05-10T04:06:27.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.078+0000 7ff7b37fe700 1 -- 172.21.15.150:0/3006550429 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b4014440 con 0x7ff7bc101d80 2024-05-10T04:06:27.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.078+0000 7ff7c4362700 1 -- 172.21.15.150:0/3006550429 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7bc1a21b0 con 0x7ff7bc101d80 2024-05-10T04:06:27.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.078+0000 7ff7b37fe700 1 -- 172.21.15.150:0/3006550429 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40075d0 con 0x7ff7bc101d80 2024-05-10T04:06:27.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.078+0000 7ff7c4362700 1 -- 172.21.15.150:0/3006550429 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7bc1a2670 con 0x7ff7bc101d80 2024-05-10T04:06:27.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.082+0000 7ff7b37fe700 1 -- 172.21.15.150:0/3006550429 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff7b4007730 con 0x7ff7bc101d80 2024-05-10T04:06:27.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.083+0000 7ff7b37fe700 1 --2- 172.21.15.150:0/3006550429 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff7a80a11c0 0x7ff7a80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.083+0000 7ff7c20fe700 1 --2- 172.21.15.150:0/3006550429 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff7a80a11c0 0x7ff7a80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:27.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.083+0000 7ff7b37fe700 1 -- 172.21.15.150:0/3006550429 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff7b4006e40 con 0x7ff7bc101d80 2024-05-10T04:06:27.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.084+0000 7ff7d46a6b80 1 Processor -- start 2024-05-10T04:06:27.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.084+0000 7ff7d46a6b80 1 -- start start 2024-05-10T04:06:27.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.084+0000 7ff7d46a6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb084d5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.084+0000 7ff7d46a6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 0x55cb084d6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.084+0000 7ff7d46a6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb084dcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.084+0000 7ff7d46a6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb083f3150 con 0x55cb084d4eb0 2024-05-10T04:06:27.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.084+0000 7ff7d46a6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb083f0b10 con 0x55cb084d5d80 2024-05-10T04:06:27.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.084+0000 7ff7d46a6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb083ec400 con 0x55cb084d66d0 2024-05-10T04:06:27.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.085+0000 7ff7c20fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb084d5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:27.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.085+0000 7ff7c20fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb084d5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59486/0 (socket says 172.21.15.150:59486) 2024-05-10T04:06:27.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.085+0000 7ff7c20fe700 1 -- 172.21.15.150:0/3557787989 learned_addr learned my addr 172.21.15.150:0/3557787989 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:27.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.085+0000 7ff7c18fd700 1 --2- 172.21.15.150:0/3557787989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 0x55cb084d6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:27.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.086+0000 7ff7c28ff700 1 --2- 172.21.15.150:0/3557787989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb084dcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:27.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.086+0000 7ff7c20fe700 1 --2- 172.21.15.150:0/3006550429 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff7a80a11c0 0x7ff7a80a36d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc1012e0 tx=0x7ff7b8005cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:27.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.086+0000 7ff7c20fe700 1 -- 172.21.15.150:0/3557787989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 msgr2=0x55cb084dcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:27.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.086+0000 7ff7c20fe700 1 --2- 172.21.15.150:0/3557787989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb084dcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.086+0000 7ff7c20fe700 1 -- 172.21.15.150:0/3557787989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 msgr2=0x55cb084d6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:27.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.086+0000 7ff7c20fe700 1 --2- 172.21.15.150:0/3557787989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 0x55cb084d6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.086+0000 7ff7c20fe700 1 -- 172.21.15.150:0/3557787989 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb0826d3b0 con 0x55cb084d4eb0 2024-05-10T04:06:27.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.086+0000 7ff7c20fe700 1 --2- 172.21.15.150:0/3557787989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb084d5300 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800cb80 tx=0x7ff7b800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6825106a121eafb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:27.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.087+0000 7ff7b0d7f700 1 -- 172.21.15.150:0/3557787989 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8004230 con 0x55cb084d4eb0 2024-05-10T04:06:27.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.087+0000 7ff7b0d7f700 1 -- 172.21.15.150:0/3557787989 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b800fda0 con 0x55cb084d4eb0 2024-05-10T04:06:27.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.087+0000 7ff7b0d7f700 1 -- 172.21.15.150:0/3557787989 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b801c440 con 0x55cb084d4eb0 2024-05-10T04:06:27.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.088+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/3557787989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 msgr2=0x55cb084d5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:27.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.088+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/3557787989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb084d5300 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800cb80 tx=0x7ff7b800f000 comp rx=0 tx=0).stop 2024-05-10T04:06:27.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.088+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/3557787989 shutdown_connections 2024-05-10T04:06:27.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.088+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/3557787989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb084dcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.088+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/3557787989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb084d5300 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.088+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/3557787989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 0x55cb084d6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.088+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/3557787989 >> 172.21.15.150:0/3557787989 conn(0x55cb0842ea60 msgr2=0x55cb084c6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:27.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.088+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/3557787989 shutdown_connections 2024-05-10T04:06:27.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.088+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/3557787989 wait complete. 2024-05-10T04:06:27.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.089+0000 7ff7d46a6b80 1 -- start start 2024-05-10T04:06:27.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.089+0000 7ff7d46a6b80 1 Processor -- start 2024-05-10T04:06:27.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.089+0000 7ff7d46a6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb084cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.089+0000 7ff7d46a6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 0x55cb084ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.089+0000 7ff7d46a6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb084d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.089+0000 7ff7d46a6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb08388560 con 0x55cb084d4eb0 2024-05-10T04:06:27.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.089+0000 7ff7d46a6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb08388200 con 0x55cb084d5d80 2024-05-10T04:06:27.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.089+0000 7ff7d46a6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb0826f6a0 con 0x55cb084d66d0 2024-05-10T04:06:27.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.089+0000 7ff7d46a6b80 10 client.0 _ll_register_callbacks cb 0x55cb08391b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:27.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.090+0000 7ff7c20fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb084cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:27.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.090+0000 7ff7c20fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb084cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59488/0 (socket says 172.21.15.150:59488) 2024-05-10T04:06:27.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.090+0000 7ff7c20fe700 1 -- 172.21.15.150:0/994603541 learned_addr learned my addr 172.21.15.150:0/994603541 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:27.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.090+0000 7ff7c18fd700 1 --2- 172.21.15.150:0/994603541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 0x55cb084ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:27.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.090+0000 7ff7c28ff700 1 --2- 172.21.15.150:0/994603541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb084d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:27.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.091+0000 7ff7c18fd700 1 -- 172.21.15.150:0/994603541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 msgr2=0x55cb084d3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:27.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.091+0000 7ff7c18fd700 1 --2- 172.21.15.150:0/994603541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb084d3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.091+0000 7ff7c18fd700 1 -- 172.21.15.150:0/994603541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 msgr2=0x55cb084cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:27.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.091+0000 7ff7c18fd700 1 --2- 172.21.15.150:0/994603541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb084cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.091+0000 7ff7c18fd700 1 -- 172.21.15.150:0/994603541 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b800c830 con 0x55cb084d5d80 2024-05-10T04:06:27.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.092+0000 7ff7c18fd700 1 --2- 172.21.15.150:0/994603541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 0x55cb084ce320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0098e0 tx=0x7ff7ac005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b210fe9fab37fc72 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:27.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.092+0000 7ff7a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:06:27.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.093+0000 7ff7a5ffb700 1 -- 172.21.15.150:0/994603541 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac00f870 con 0x55cb084d5d80 2024-05-10T04:06:27.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.093+0000 7ff7a5ffb700 1 -- 172.21.15.150:0/994603541 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff7ac00f9d0 con 0x55cb084d5d80 2024-05-10T04:06:27.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.093+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/994603541 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb08345270 con 0x55cb084d5d80 2024-05-10T04:06:27.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.093+0000 7ff7a5ffb700 1 -- 172.21.15.150:0/994603541 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7ac00fb90 con 0x55cb084d5d80 2024-05-10T04:06:27.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.094+0000 7ff7d46a6b80 10 client.25895 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:27.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.094+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/994603541 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cb08219b60 con 0x55cb084d5d80 2024-05-10T04:06:27.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.094+0000 7ff7a5ffb700 1 -- 172.21.15.150:0/994603541 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb08345270 con 0x55cb084d5d80 2024-05-10T04:06:27.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.094+0000 7ff7a5ffb700 1 -- 172.21.15.150:0/994603541 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff7ac00e3e0 con 0x55cb084d5d80 2024-05-10T04:06:27.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.094+0000 7ff7d46a6b80 10 client.25895 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:27.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.094+0000 7ff7d46a6b80 10 client.25895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:27.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.094+0000 7ff7d46a6b80 20 client.25895 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:27.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.094+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/994603541 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55cb084f33f0 0x55cb084f5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.094+0000 7ff7d46a6b80 4 client.25895 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:27.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.094+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/994603541 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb0849b780 con 0x55cb084f33f0 2024-05-10T04:06:27.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.095+0000 7ff7c20fe700 1 --2- 172.21.15.150:0/994603541 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55cb084f33f0 0x55cb084f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:27.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.097+0000 7ff7a5ffb700 10 client.25895 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:27.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.097+0000 7ff7c20fe700 1 --2- 172.21.15.150:0/994603541 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55cb084f33f0 0x55cb084f5840 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4449e9180fa5f595 server_cookie=4c801031a54f57e0 in_seq=0 out_seq=0 2024-05-10T04:06:27.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.098+0000 7ff7a5ffb700 1 -- 172.21.15.150:0/994603541 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb0849b780 con 0x55cb084f33f0 2024-05-10T04:06:27.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.098+0000 7ff7a5ffb700 10 client.25895 handle_command_reply: tid=0 2024-05-10T04:06:27.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.098+0000 7ff7d46a6b80 1 client.25895 shutdown 2024-05-10T04:06:27.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.099+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/994603541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 msgr2=0x55cb084ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:27.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.099+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/994603541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 0x55cb084ce320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0098e0 tx=0x7ff7ac005f10 comp rx=0 tx=0).stop 2024-05-10T04:06:27.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.099+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/994603541 shutdown_connections 2024-05-10T04:06:27.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.099+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/994603541 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55cb084f33f0 0x55cb084f5840 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.099+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/994603541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb084d3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.099+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/994603541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb084cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.099+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/994603541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 0x55cb084ce320 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.099+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/994603541 >> 172.21.15.150:0/994603541 conn(0x55cb0842ea60 msgr2=0x55cb084c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:27.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.099+0000 7ff7a5ffb700 0 client.25895 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:27.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.099+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/994603541 shutdown_connections 2024-05-10T04:06:27.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.099+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/994603541 wait complete. 2024-05-10T04:06:27.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.099+0000 7ff7d46a6b80 20 client.25895 trim_cache size 0 max 16384 2024-05-10T04:06:27.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.128+0000 7ff7d46a6b80 1 Processor -- start 2024-05-10T04:06:27.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.128+0000 7ff7d46a6b80 1 -- start start 2024-05-10T04:06:27.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.128+0000 7ff7d46a6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d4eb0 0x55cb084d8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7d46a6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d5d80 0x55cb084cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7d46a6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb08503670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7d46a6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb084d6190 con 0x55cb084d5d80 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7d46a6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb084d6300 con 0x55cb084d4eb0 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7d46a6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb084d3860 con 0x55cb084d66d0 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7c18fd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d5d80 0x55cb084cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7c18fd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d5d80 0x55cb084cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59496/0 (socket says 172.21.15.150:59496) 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7c18fd700 1 -- 172.21.15.150:0/3851290115 learned_addr learned my addr 172.21.15.150:0/3851290115 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7c18fd700 1 -- 172.21.15.150:0/3851290115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 msgr2=0x55cb08503670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7c18fd700 1 --2- 172.21.15.150:0/3851290115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb08503670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7c18fd700 1 -- 172.21.15.150:0/3851290115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d4eb0 msgr2=0x55cb084d8fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7c18fd700 1 --2- 172.21.15.150:0/3851290115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d4eb0 0x55cb084d8fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7c18fd700 1 -- 172.21.15.150:0/3851290115 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb083b2560 con 0x55cb084d5d80 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.129+0000 7ff7c18fd700 1 --2- 172.21.15.150:0/3851290115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d5d80 0x55cb084cd3c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009990 tx=0x7ff7ac0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f06b92288d7638 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.130+0000 7ff7b1580700 1 -- 172.21.15.150:0/3851290115 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac00ea60 con 0x55cb084d5d80 2024-05-10T04:06:27.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.130+0000 7ff7b1580700 1 -- 172.21.15.150:0/3851290115 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7ac01e040 con 0x55cb084d5d80 2024-05-10T04:06:27.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.130+0000 7ff7b1580700 1 -- 172.21.15.150:0/3851290115 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac020710 con 0x55cb084d5d80 2024-05-10T04:06:27.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.131+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/3851290115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d5d80 msgr2=0x55cb084cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:27.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.131+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/3851290115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d5d80 0x55cb084cd3c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009990 tx=0x7ff7ac0036f0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.131+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/3851290115 shutdown_connections 2024-05-10T04:06:27.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.131+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/3851290115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb08503670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.131+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/3851290115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d4eb0 0x55cb084d8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.131+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/3851290115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d5d80 0x55cb084cd3c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.131+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/3851290115 >> 172.21.15.150:0/3851290115 conn(0x55cb0842ea60 msgr2=0x55cb084c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:27.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.131+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/3851290115 shutdown_connections 2024-05-10T04:06:27.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.131+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/3851290115 wait complete. 2024-05-10T04:06:27.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.132+0000 7ff7d46a6b80 1 -- start start 2024-05-10T04:06:27.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.132+0000 7ff7d46a6b80 1 Processor -- start 2024-05-10T04:06:27.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.133+0000 7ff7d46a6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb08502ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.133+0000 7ff7d46a6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 0x55cb08503020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.133+0000 7ff7d46a6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb08548330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.133+0000 7ff7d46a6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb08503640 con 0x55cb084d4eb0 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.133+0000 7ff7d46a6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb08548870 con 0x55cb084d5d80 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.133+0000 7ff7d46a6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb085489b0 con 0x55cb084d66d0 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.133+0000 7ff7d46a6b80 10 client.0 _ll_register_callbacks cb 0x55cb08391b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.133+0000 7ff7c20fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb08502ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.133+0000 7ff7c20fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb08502ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59510/0 (socket says 172.21.15.150:59510) 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.133+0000 7ff7c20fe700 1 -- 172.21.15.150:0/4079065798 learned_addr learned my addr 172.21.15.150:0/4079065798 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.134+0000 7ff7c18fd700 1 --2- 172.21.15.150:0/4079065798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 0x55cb08503020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.134+0000 7ff7c20fe700 1 -- 172.21.15.150:0/4079065798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 msgr2=0x55cb08548330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.134+0000 7ff7c20fe700 1 --2- 172.21.15.150:0/4079065798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb08548330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.134+0000 7ff7c20fe700 1 -- 172.21.15.150:0/4079065798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 msgr2=0x55cb08503020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.134+0000 7ff7c20fe700 1 --2- 172.21.15.150:0/4079065798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 0x55cb08503020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.134+0000 7ff7c20fe700 1 -- 172.21.15.150:0/4079065798 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb08219b60 con 0x55cb084d4eb0 2024-05-10T04:06:27.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.134+0000 7ff7c20fe700 1 --2- 172.21.15.150:0/4079065798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb08502ae0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800ae70 tx=0x7ff7b800fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ce1375f58ac205 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:27.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.134+0000 7ff7a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:27.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.135+0000 7ff7a5ffb700 1 -- 172.21.15.150:0/4079065798 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b802fd00 con 0x55cb084d4eb0 2024-05-10T04:06:27.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.135+0000 7ff7a5ffb700 1 -- 172.21.15.150:0/4079065798 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff7b8005250 con 0x55cb084d4eb0 2024-05-10T04:06:27.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.135+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/4079065798 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7b800fda0 con 0x55cb084d4eb0 2024-05-10T04:06:27.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.135+0000 7ff7a5ffb700 1 -- 172.21.15.150:0/4079065798 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7b8005450 con 0x55cb084d4eb0 2024-05-10T04:06:27.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.135+0000 7ff7d46a6b80 10 client.16257 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:27.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.135+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/4079065798 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cb08271cd0 con 0x55cb084d4eb0 2024-05-10T04:06:27.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.136+0000 7ff7a5ffb700 1 -- 172.21.15.150:0/4079065798 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b800fda0 con 0x55cb084d4eb0 2024-05-10T04:06:27.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.136+0000 7ff7a5ffb700 1 -- 172.21.15.150:0/4079065798 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff7b8036050 con 0x55cb084d4eb0 2024-05-10T04:06:27.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.136+0000 7ff7d46a6b80 10 client.16257 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:27.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.136+0000 7ff7d46a6b80 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:27.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.136+0000 7ff7d46a6b80 20 client.16257 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:27.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.136+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/4079065798 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55cb0854a250 0x55cb0854c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:27.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.136+0000 7ff7d46a6b80 4 client.16257 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:06:27.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.136+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/4079065798 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cb083e4ec0 con 0x55cb0854a250 2024-05-10T04:06:27.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.138+0000 7ff7c18fd700 1 --2- 172.21.15.150:0/4079065798 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55cb0854a250 0x55cb0854c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:27.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.141+0000 7ff7a5ffb700 10 client.16257 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:27.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.141+0000 7ff7c18fd700 1 --2- 172.21.15.150:0/4079065798 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55cb0854a250 0x55cb0854c670 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831f35be5f8814f6 server_cookie=420ea831184efb85 in_seq=0 out_seq=0 2024-05-10T04:06:27.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.141+0000 7ff7a5ffb700 1 -- 172.21.15.150:0/4079065798 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cb083e4ec0 con 0x55cb0854a250 2024-05-10T04:06:27.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.141+0000 7ff7a5ffb700 10 client.16257 handle_command_reply: tid=0 2024-05-10T04:06:27.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.141+0000 7ff7d46a6b80 1 client.16257 shutdown 2024-05-10T04:06:27.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.142+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/4079065798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 msgr2=0x55cb08502ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:27.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.142+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/4079065798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb08502ae0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800ae70 tx=0x7ff7b800fb70 comp rx=0 tx=0).stop 2024-05-10T04:06:27.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.142+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/4079065798 shutdown_connections 2024-05-10T04:06:27.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.142+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/4079065798 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55cb0854a250 0x55cb0854c670 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.142+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/4079065798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb084d66d0 0x55cb08548330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.142+0000 7ff7a5ffb700 0 client.16257 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:27.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.142+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/4079065798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb084d4eb0 0x55cb08502ae0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.142+0000 7ff7d46a6b80 1 --2- 172.21.15.150:0/4079065798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb084d5d80 0x55cb08503020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.142+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/4079065798 >> 172.21.15.150:0/4079065798 conn(0x55cb0842ea60 msgr2=0x55cb084d9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:27.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.142+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/4079065798 shutdown_connections 2024-05-10T04:06:27.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.142+0000 7ff7d46a6b80 1 -- 172.21.15.150:0/4079065798 wait complete. 2024-05-10T04:06:27.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.142+0000 7ff7d46a6b80 20 client.16257 trim_cache size 0 max 16384 2024-05-10T04:06:27.145 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:06:27.146 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T04:06:27.146 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": { 2024-05-10T04:06:27.146 INFO:teuthology.orchestra.run.smithi150.stdout: "f2a72b07-4da3-4b55-8f9c-ed98939a6c85": { 2024-05-10T04:06:27.146 INFO:teuthology.orchestra.run.smithi150.stdout: "path": "/", 2024-05-10T04:06:27.146 INFO:teuthology.orchestra.run.smithi150.stdout: "tag": "f2a72b07-4da3-4b55-8f9c-ed98939a6c85", 2024-05-10T04:06:27.146 INFO:teuthology.orchestra.run.smithi150.stdout: "options": "recursive,force" 2024-05-10T04:06:27.146 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:06:27.146 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:06:27.146 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:06:27.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.143+0000 7ff7c4362700 1 -- 172.21.15.150:0/3006550429 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff7a80a11c0 msgr2=0x7ff7a80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:27.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.143+0000 7ff7c4362700 1 --2- 172.21.15.150:0/3006550429 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff7a80a11c0 0x7ff7a80a36d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc1012e0 tx=0x7ff7b8005cb0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.143+0000 7ff7c4362700 1 -- 172.21.15.150:0/3006550429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7bc101d80 msgr2=0x7ff7bc10a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:27.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.143+0000 7ff7c4362700 1 --2- 172.21.15.150:0/3006550429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7bc101d80 0x7ff7bc10a370 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400b970 tx=0x7ff7b400bd30 comp rx=0 tx=0).stop 2024-05-10T04:06:27.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.143+0000 7ff7c4362700 1 -- 172.21.15.150:0/3006550429 shutdown_connections 2024-05-10T04:06:27.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.143+0000 7ff7c4362700 1 --2- 172.21.15.150:0/3006550429 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff7a80a11c0 0x7ff7a80a36d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.143+0000 7ff7c4362700 1 --2- 172.21.15.150:0/3006550429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7bc101d80 0x7ff7bc10a370 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.143+0000 7ff7c4362700 1 --2- 172.21.15.150:0/3006550429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7bc1002c0 0x7ff7bc10b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.143+0000 7ff7c4362700 1 --2- 172.21.15.150:0/3006550429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7bc1013e0 0x7ff7bc109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:27.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.143+0000 7ff7c4362700 1 -- 172.21.15.150:0/3006550429 >> 172.21.15.150:0/3006550429 conn(0x7ff7bc0fb8f0 msgr2=0x7ff7bc104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:27.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.143+0000 7ff7c4362700 1 -- 172.21.15.150:0/3006550429 shutdown_connections 2024-05-10T04:06:27.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:27.143+0000 7ff7c4362700 1 -- 172.21.15.150:0/3006550429 wait complete. 2024-05-10T04:06:27.161 INFO:tasks.cephfs.filesystem:scrub status for tag:f2a72b07-4da3-4b55-8f9c-ed98939a6c85 - {'path': '/', 'tag': 'f2a72b07-4da3-4b55-8f9c-ed98939a6c85', 'options': 'recursive,force'} 2024-05-10T04:06:27.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:27 smithi150 ceph-mon[42477]: scrub queued for path: / 2024-05-10T04:06:27.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:27 smithi150 ceph-mon[42477]: scrub summary: idle+waiting paths [/] 2024-05-10T04:06:27.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:27 smithi150 ceph-mon[42477]: scrub summary: active paths [/] 2024-05-10T04:06:27.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:27 smithi150 ceph-mon[42477]: scrub summary: idle+waiting paths [/] 2024-05-10T04:06:27.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:27 smithi181 ceph-mon[46602]: scrub queued for path: / 2024-05-10T04:06:27.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:27 smithi181 ceph-mon[46602]: scrub summary: idle+waiting paths [/] 2024-05-10T04:06:27.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:27 smithi181 ceph-mon[46602]: scrub summary: active paths [/] 2024-05-10T04:06:27.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:27 smithi181 ceph-mon[46602]: scrub summary: idle+waiting paths [/] 2024-05-10T04:06:27.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:27 smithi161 ceph-mon[46918]: scrub queued for path: / 2024-05-10T04:06:27.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:27 smithi161 ceph-mon[46918]: scrub summary: idle+waiting paths [/] 2024-05-10T04:06:27.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:27 smithi161 ceph-mon[46918]: scrub summary: active paths [/] 2024-05-10T04:06:27.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:27 smithi161 ceph-mon[46918]: scrub summary: idle+waiting paths [/] 2024-05-10T04:06:28.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:28 smithi150 ceph-mon[42477]: pgmap v346: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 506 MiB/s wr, 155 op/s 2024-05-10T04:06:28.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:28 smithi181 ceph-mon[46602]: pgmap v346: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 506 MiB/s wr, 155 op/s 2024-05-10T04:06:28.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:28 smithi161 ceph-mon[46918]: pgmap v346: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 506 MiB/s wr, 155 op/s 2024-05-10T04:06:28.742 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:06:28.742 INFO:tasks.workunit.client.0.smithi150.stdout: Children see throughput for 1 initial writers = 1009862.25 kB/sec 2024-05-10T04:06:28.742 INFO:tasks.workunit.client.0.smithi150.stdout: Parent sees throughput for 1 initial writers = 1009833.73 kB/sec 2024-05-10T04:06:28.742 INFO:tasks.workunit.client.0.smithi150.stdout: Min throughput per process = 1009862.25 kB/sec 2024-05-10T04:06:28.742 INFO:tasks.workunit.client.0.smithi150.stdout: Max throughput per process = 1009862.25 kB/sec 2024-05-10T04:06:28.742 INFO:tasks.workunit.client.0.smithi150.stdout: Avg throughput per process = 1009862.25 kB/sec 2024-05-10T04:06:28.742 INFO:tasks.workunit.client.0.smithi150.stdout: Min xfer = 10485760.00 kB 2024-05-10T04:06:30.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:30 smithi150 ceph-mon[42477]: pgmap v347: 129 pgs: 129 active+clean; 9.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 803 MiB/s wr, 237 op/s 2024-05-10T04:06:30.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:30 smithi181 ceph-mon[46602]: pgmap v347: 129 pgs: 129 active+clean; 9.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 803 MiB/s wr, 237 op/s 2024-05-10T04:06:30.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:30 smithi161 ceph-mon[46918]: pgmap v347: 129 pgs: 129 active+clean; 9.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 803 MiB/s wr, 237 op/s 2024-05-10T04:06:31.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:31 smithi150 ceph-mon[42477]: scrub summary: idle 2024-05-10T04:06:31.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:31 smithi181 ceph-mon[46602]: scrub summary: idle 2024-05-10T04:06:31.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:31 smithi161 ceph-mon[46918]: scrub summary: idle 2024-05-10T04:06:32.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:32 smithi150 ceph-mon[42477]: pgmap v348: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 815 MiB/s wr, 242 op/s 2024-05-10T04:06:32.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:32 smithi181 ceph-mon[46602]: pgmap v348: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 815 MiB/s wr, 242 op/s 2024-05-10T04:06:32.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:32 smithi161 ceph-mon[46918]: pgmap v348: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 815 MiB/s wr, 242 op/s 2024-05-10T04:06:33.721 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:33 smithi150 ceph-mon[42477]: pgmap v349: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 846 MiB/s wr, 247 op/s 2024-05-10T04:06:33.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:33 smithi181 ceph-mon[46602]: pgmap v349: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 846 MiB/s wr, 247 op/s 2024-05-10T04:06:33.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:33 smithi161 ceph-mon[46918]: pgmap v349: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 846 MiB/s wr, 247 op/s 2024-05-10T04:06:34.967 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:34 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:06:34.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:34 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:06:34.995 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:34 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:06:35.964 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:35 smithi150 ceph-mon[42477]: pgmap v350: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 757 MiB/s wr, 226 op/s 2024-05-10T04:06:35.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:35 smithi181 ceph-mon[46602]: pgmap v350: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 757 MiB/s wr, 226 op/s 2024-05-10T04:06:35.991 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:35 smithi161 ceph-mon[46918]: pgmap v350: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 757 MiB/s wr, 226 op/s 2024-05-10T04:06:36.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:36 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:06:36.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:36 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:06:36.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:36 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:06:36.991 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:36 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:06:36.991 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:36 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:06:36.991 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:36 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:06:37.219 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:36 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:06:37.220 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:36 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:06:37.220 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:36 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:06:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.171+0000 7f6f02b2a700 1 -- 172.21.15.150:0/1057297316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6efc0ffe10 msgr2=0x7f6efc100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.171+0000 7f6f02b2a700 1 --2- 172.21.15.150:0/1057297316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6efc0ffe10 0x7f6efc100220 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c310 tx=0x7f6eec00c620 comp rx=0 tx=0).stop 2024-05-10T04:06:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.176+0000 7f6f02b2a700 1 -- 172.21.15.150:0/1057297316 shutdown_connections 2024-05-10T04:06:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.176+0000 7f6f02b2a700 1 --2- 172.21.15.150:0/1057297316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6efc1018d0 0x7f6efc108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.176+0000 7f6f02b2a700 1 --2- 172.21.15.150:0/1057297316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6efc0ffe10 0x7f6efc100220 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.176+0000 7f6f02b2a700 1 --2- 172.21.15.150:0/1057297316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc100f30 0x7f6efc101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.176+0000 7f6f02b2a700 1 -- 172.21.15.150:0/1057297316 >> 172.21.15.150:0/1057297316 conn(0x7f6efc0fb420 msgr2=0x7f6efc0fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.194+0000 7f6f02b2a700 1 -- 172.21.15.150:0/1057297316 shutdown_connections 2024-05-10T04:06:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.196+0000 7f6f02b2a700 1 -- 172.21.15.150:0/1057297316 wait complete. 2024-05-10T04:06:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.197+0000 7f6f02b2a700 1 Processor -- start 2024-05-10T04:06:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.197+0000 7f6f02b2a700 1 -- start start 2024-05-10T04:06:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.197+0000 7f6f02b2a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc0ffe10 0x7f6efc111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.197+0000 7f6f02b2a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6efc100f30 0x7f6efc111ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.197+0000 7f6f02b2a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6efc1018d0 0x7f6efc10fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.197+0000 7f6f02b2a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc110450 con 0x7f6efc100f30 2024-05-10T04:06:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.197+0000 7f6f02b2a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc1105c0 con 0x7f6efc1018d0 2024-05-10T04:06:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.197+0000 7f6f02b2a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc110780 con 0x7f6efc0ffe10 2024-05-10T04:06:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.199+0000 7f6f010c7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6efc1018d0 0x7f6efc10fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.199+0000 7f6f010c7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6efc1018d0 0x7f6efc10fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:33426/0 (socket says 172.21.15.150:33426) 2024-05-10T04:06:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.199+0000 7f6f010c7700 1 -- 172.21.15.150:0/1146003265 learned_addr learned my addr 172.21.15.150:0/1146003265 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.199+0000 7f6f008c6700 1 --2- 172.21.15.150:0/1146003265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc0ffe10 0x7f6efc111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.200+0000 7f6f010c7700 1 -- 172.21.15.150:0/1146003265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc0ffe10 msgr2=0x7f6efc111590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.200+0000 7f6f010c7700 1 --2- 172.21.15.150:0/1146003265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc0ffe10 0x7f6efc111590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.200+0000 7f6f010c7700 1 -- 172.21.15.150:0/1146003265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6efc100f30 msgr2=0x7f6efc111ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T04:06:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.200+0000 7f6f010c7700 1 --2- 172.21.15.150:0/1146003265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6efc100f30 0x7f6efc111ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.200+0000 7f6f010c7700 1 -- 172.21.15.150:0/1146003265 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec00c050 con 0x7f6efc1018d0 2024-05-10T04:06:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.200+0000 7f6f010c7700 1 --2- 172.21.15.150:0/1146003265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6efc1018d0 0x7f6efc10fd70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400b4a0 tx=0x7f6ef400b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.201+0000 7f6ef9ffb700 1 -- 172.21.15.150:0/1146003265 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4011830 con 0x7f6efc1018d0 2024-05-10T04:06:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.201+0000 7f6f02b2a700 1 -- 172.21.15.150:0/1146003265 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6efc110a60 con 0x7f6efc1018d0 2024-05-10T04:06:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.201+0000 7f6f02b2a700 1 -- 172.21.15.150:0/1146003265 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6efc1af5d0 con 0x7f6efc1018d0 2024-05-10T04:06:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.201+0000 7f6ef9ffb700 1 -- 172.21.15.150:0/1146003265 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ef4011e50 con 0x7f6efc1018d0 2024-05-10T04:06:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.201+0000 7f6ef9ffb700 1 -- 172.21.15.150:0/1146003265 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400f2e0 con 0x7f6efc1018d0 2024-05-10T04:06:38.211 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.207+0000 7f6ef9ffb700 1 -- 172.21.15.150:0/1146003265 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f6ef4011990 con 0x7f6efc1018d0 2024-05-10T04:06:38.211 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.208+0000 7f6f12e6eb80 1 Processor -- start 2024-05-10T04:06:38.211 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.208+0000 7f6ef9ffb700 1 --2- 172.21.15.150:0/1146003265 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6ee40a1270 0x7f6ee40a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.209+0000 7f6ef9ffb700 1 -- 172.21.15.150:0/1146003265 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6ef40cc690 con 0x7f6efc1018d0 2024-05-10T04:06:38.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:37 smithi150 ceph-mon[42477]: pgmap v351: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 661 MiB/s wr, 190 op/s 2024-05-10T04:06:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.210+0000 7f6efbfff700 1 --2- 172.21.15.150:0/1146003265 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6ee40a1270 0x7f6ee40a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.210+0000 7f6efbfff700 1 --2- 172.21.15.150:0/1146003265 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6ee40a1270 0x7f6ee40a3780 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6efc110f30 tx=0x7f6ef0006c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.212+0000 7f6f12e6eb80 1 -- start start 2024-05-10T04:06:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.212+0000 7f6f12e6eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561987891180 0x5619878915b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.212+0000 7f6f12e6eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x561987892460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.212+0000 7f6f12e6eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5619878929a0 0x5619878992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.212+0000 7f6f12e6eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619877b80f0 con 0x561987892050 2024-05-10T04:06:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.212+0000 7f6f12e6eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619877b5ab0 con 0x561987891180 2024-05-10T04:06:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.212+0000 7f6f12e6eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619877b13a0 con 0x5619878929a0 2024-05-10T04:06:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.212+0000 7f6efbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x561987892460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.212+0000 7f6efbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x561987892460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55504/0 (socket says 172.21.15.150:55504) 2024-05-10T04:06:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.212+0000 7f6efbfff700 1 -- 172.21.15.150:0/1743814067 learned_addr learned my addr 172.21.15.150:0/1743814067 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.213+0000 7f6f008c6700 1 --2- 172.21.15.150:0/1743814067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561987891180 0x5619878915b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.213+0000 7f6efbfff700 1 -- 172.21.15.150:0/1743814067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5619878929a0 msgr2=0x5619878992a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:06:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.213+0000 7f6efbfff700 1 --2- 172.21.15.150:0/1743814067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5619878929a0 0x5619878992a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.213+0000 7f6efbfff700 1 -- 172.21.15.150:0/1743814067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561987891180 msgr2=0x5619878915b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.213+0000 7f6efbfff700 1 --2- 172.21.15.150:0/1743814067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561987891180 0x5619878915b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.213+0000 7f6efbfff700 1 -- 172.21.15.150:0/1743814067 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561987632350 con 0x561987892050 2024-05-10T04:06:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.213+0000 7f6efbfff700 1 --2- 172.21.15.150:0/1743814067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x561987892460 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00158f0 tx=0x7f6ef000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49a6867381439c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.215+0000 7f6ee2ffd700 1 -- 172.21.15.150:0/1743814067 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef001a070 con 0x561987892050 2024-05-10T04:06:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.215+0000 7f6ee2ffd700 1 -- 172.21.15.150:0/1743814067 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ef0012970 con 0x561987892050 2024-05-10T04:06:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.215+0000 7f6ee2ffd700 1 -- 172.21.15.150:0/1743814067 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00138b0 con 0x561987892050 2024-05-10T04:06:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.224+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/1743814067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 msgr2=0x561987892460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.224+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/1743814067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x561987892460 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00158f0 tx=0x7f6ef000e000 comp rx=0 tx=0).stop 2024-05-10T04:06:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.231+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/1743814067 shutdown_connections 2024-05-10T04:06:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.231+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/1743814067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5619878929a0 0x5619878992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.231+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/1743814067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561987891180 0x5619878915b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.231+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/1743814067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x561987892460 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.231+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/1743814067 >> 172.21.15.150:0/1743814067 conn(0x5619877f3a00 msgr2=0x56198788ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:38.237 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:37 smithi181 ceph-mon[46602]: pgmap v351: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 661 MiB/s wr, 190 op/s 2024-05-10T04:06:38.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:37 smithi161 ceph-mon[46918]: pgmap v351: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 661 MiB/s wr, 190 op/s 2024-05-10T04:06:38.251 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.250+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/1743814067 shutdown_connections 2024-05-10T04:06:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.250+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/1743814067 wait complete. 2024-05-10T04:06:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.250+0000 7f6f12e6eb80 1 -- start start 2024-05-10T04:06:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.251+0000 7f6f12e6eb80 1 Processor -- start 2024-05-10T04:06:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.252+0000 7f6f12e6eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561987891180 0x56198789b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.252+0000 7f6f12e6eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x56198789b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.252+0000 7f6f12e6eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619878929a0 0x5619878a2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.252+0000 7f6f12e6eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198774d500 con 0x561987892050 2024-05-10T04:06:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.252+0000 7f6f12e6eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198774d1a0 con 0x5619878929a0 2024-05-10T04:06:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.252+0000 7f6f12e6eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561987634640 con 0x561987891180 2024-05-10T04:06:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.252+0000 7f6f12e6eb80 10 client.0 _ll_register_callbacks cb 0x561987756ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.254+0000 7f6f008c6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561987891180 0x56198789b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.256+0000 7f6f010c7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619878929a0 0x5619878a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.256+0000 7f6f010c7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619878929a0 0x5619878a2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:33438/0 (socket says 172.21.15.150:33438) 2024-05-10T04:06:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.256+0000 7f6f010c7700 1 -- 172.21.15.150:0/1309291640 learned_addr learned my addr 172.21.15.150:0/1309291640 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.256+0000 7f6efbfff700 1 --2- 172.21.15.150:0/1309291640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x56198789b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.258+0000 7f6f010c7700 1 -- 172.21.15.150:0/1309291640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561987891180 msgr2=0x56198789b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.258+0000 7f6f010c7700 1 --2- 172.21.15.150:0/1309291640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561987891180 0x56198789b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.258+0000 7f6f010c7700 1 -- 172.21.15.150:0/1309291640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 msgr2=0x56198789b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.258+0000 7f6f010c7700 1 --2- 172.21.15.150:0/1309291640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x56198789b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.258+0000 7f6f010c7700 1 -- 172.21.15.150:0/1309291640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef0015660 con 0x5619878929a0 2024-05-10T04:06:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.264+0000 7f6f010c7700 1 --2- 172.21.15.150:0/1309291640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619878929a0 0x5619878a2720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40961e0 tx=0x7f6ef4099150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75093aa68dafe7ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.267+0000 7f6edbfff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:06:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.267+0000 7f6edbfff700 1 -- 172.21.15.150:0/1309291640 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef409d5a0 con 0x5619878929a0 2024-05-10T04:06:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.267+0000 7f6edbfff700 1 -- 172.21.15.150:0/1309291640 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6ef40a1bf0 con 0x5619878929a0 2024-05-10T04:06:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.267+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/1309291640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56198770a210 con 0x5619878929a0 2024-05-10T04:06:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.279+0000 7f6edbfff700 1 -- 172.21.15.150:0/1309291640 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ef40c9f30 con 0x5619878929a0 2024-05-10T04:06:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.279+0000 7f6f12e6eb80 10 client.25907 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.279+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/1309291640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5619875deb00 con 0x5619878929a0 2024-05-10T04:06:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.279+0000 7f6edbfff700 1 -- 172.21.15.150:0/1309291640 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56198770a210 con 0x5619878929a0 2024-05-10T04:06:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.280+0000 7f6edbfff700 1 -- 172.21.15.150:0/1309291640 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ef4098c60 con 0x5619878929a0 2024-05-10T04:06:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.280+0000 7f6f12e6eb80 10 client.25907 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.280+0000 7f6f12e6eb80 10 client.25907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.280+0000 7f6f12e6eb80 20 client.25907 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.280+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/1309291640 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5619878a6ee0 0x5619878a9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.280+0000 7f6f12e6eb80 4 client.25907 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.280+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/1309291640 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561987860720 con 0x5619878a6ee0 2024-05-10T04:06:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.282+0000 7f6f008c6700 1 --2- 172.21.15.150:0/1309291640 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5619878a6ee0 0x5619878a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.292+0000 7f6f008c6700 1 --2- 172.21.15.150:0/1309291640 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5619878a6ee0 0x5619878a9330 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f80f6ea21a16b169 server_cookie=65cd9f86a613a10c in_seq=0 out_seq=0 2024-05-10T04:06:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.294+0000 7f6edbfff700 10 client.25907 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.294+0000 7f6edbfff700 1 -- 172.21.15.150:0/1309291640 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561987860720 con 0x5619878a6ee0 2024-05-10T04:06:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.294+0000 7f6edbfff700 10 client.25907 handle_command_reply: tid=0 2024-05-10T04:06:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.297+0000 7f6f12e6eb80 1 client.25907 shutdown 2024-05-10T04:06:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.297+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/1309291640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619878929a0 msgr2=0x5619878a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.297+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/1309291640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619878929a0 0x5619878a2720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40961e0 tx=0x7f6ef4099150 comp rx=0 tx=0).stop 2024-05-10T04:06:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.305+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/1309291640 shutdown_connections 2024-05-10T04:06:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.305+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/1309291640 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5619878a6ee0 0x5619878a9330 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.305+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/1309291640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619878929a0 0x5619878a2720 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.305+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/1309291640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561987891180 0x56198789b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.305+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/1309291640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x56198789b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.305+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/1309291640 >> 172.21.15.150:0/1309291640 conn(0x5619877f3a00 msgr2=0x56198788ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.306+0000 7f6edbfff700 0 client.25907 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.308+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/1309291640 shutdown_connections 2024-05-10T04:06:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.312+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/1309291640 wait complete. 2024-05-10T04:06:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.313+0000 7f6f12e6eb80 20 client.25907 trim_cache size 0 max 16384 2024-05-10T04:06:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.365+0000 7f6f12e6eb80 1 Processor -- start 2024-05-10T04:06:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.365+0000 7f6f12e6eb80 1 -- start start 2024-05-10T04:06:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.366+0000 7f6f12e6eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561987891180 0x56198789b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.366+0000 7f6f12e6eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x5619878a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.366+0000 7f6f12e6eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5619878929a0 0x56198789b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.366+0000 7f6f12e6eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619878965c0 con 0x561987892050 2024-05-10T04:06:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.366+0000 7f6f12e6eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619878995c0 con 0x561987891180 2024-05-10T04:06:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.366+0000 7f6f12e6eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561987886360 con 0x5619878929a0 2024-05-10T04:06:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.366+0000 7f6efbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x5619878a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.366+0000 7f6efbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x5619878a25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55528/0 (socket says 172.21.15.150:55528) 2024-05-10T04:06:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.366+0000 7f6efbfff700 1 -- 172.21.15.150:0/2004483223 learned_addr learned my addr 172.21.15.150:0/2004483223 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.366+0000 7f6f008c6700 1 --2- 172.21.15.150:0/2004483223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561987891180 0x56198789b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.367+0000 7f6f010c7700 1 --2- 172.21.15.150:0/2004483223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5619878929a0 0x56198789b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.367+0000 7f6efbfff700 1 -- 172.21.15.150:0/2004483223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5619878929a0 msgr2=0x56198789b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.367+0000 7f6efbfff700 1 --2- 172.21.15.150:0/2004483223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5619878929a0 0x56198789b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.367+0000 7f6efbfff700 1 -- 172.21.15.150:0/2004483223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561987891180 msgr2=0x56198789b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.367+0000 7f6efbfff700 1 --2- 172.21.15.150:0/2004483223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561987891180 0x56198789b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.367+0000 7f6efbfff700 1 -- 172.21.15.150:0/2004483223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561987777500 con 0x561987892050 2024-05-10T04:06:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.367+0000 7f6efbfff700 1 --2- 172.21.15.150:0/2004483223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x5619878a25d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00158c0 tx=0x7f6ef0006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225aa77951e7fa83 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.368+0000 7f6ee37fe700 1 -- 172.21.15.150:0/2004483223 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef001a040 con 0x561987892050 2024-05-10T04:06:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.368+0000 7f6ee37fe700 1 -- 172.21.15.150:0/2004483223 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ef0013d70 con 0x561987892050 2024-05-10T04:06:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.368+0000 7f6ee37fe700 1 -- 172.21.15.150:0/2004483223 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0004030 con 0x561987892050 2024-05-10T04:06:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.370+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/2004483223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 msgr2=0x5619878a25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.370+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/2004483223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x5619878a25d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00158c0 tx=0x7f6ef0006e10 comp rx=0 tx=0).stop 2024-05-10T04:06:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.370+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/2004483223 shutdown_connections 2024-05-10T04:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.370+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/2004483223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5619878929a0 0x56198789b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.370+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/2004483223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561987891180 0x56198789b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.370+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/2004483223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987892050 0x5619878a25d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.370+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/2004483223 >> 172.21.15.150:0/2004483223 conn(0x5619877f3a00 msgr2=0x561987895eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.371+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/2004483223 shutdown_connections 2024-05-10T04:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.371+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/2004483223 wait complete. 2024-05-10T04:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.371+0000 7f6f12e6eb80 1 -- start start 2024-05-10T04:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.371+0000 7f6f12e6eb80 1 Processor -- start 2024-05-10T04:06:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.372+0000 7f6f12e6eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987891180 0x56198789f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.372+0000 7f6f12e6eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561987892050 0x56198789fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.372+0000 7f6f12e6eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619878929a0 0x5619878a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.372+0000 7f6f12e6eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619878a1310 con 0x561987891180 2024-05-10T04:06:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.372+0000 7f6f12e6eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619878a1480 con 0x5619878929a0 2024-05-10T04:06:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.372+0000 7f6f12e6eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619878a1640 con 0x561987892050 2024-05-10T04:06:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.372+0000 7f6f12e6eb80 10 client.0 _ll_register_callbacks cb 0x561987756ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.373+0000 7f6f008c6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987891180 0x56198789f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.373+0000 7f6f008c6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987891180 0x56198789f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55536/0 (socket says 172.21.15.150:55536) 2024-05-10T04:06:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.373+0000 7f6f008c6700 1 -- 172.21.15.150:0/2085161808 learned_addr learned my addr 172.21.15.150:0/2085161808 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.373+0000 7f6f010c7700 1 --2- 172.21.15.150:0/2085161808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619878929a0 0x5619878a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.373+0000 7f6efbfff700 1 --2- 172.21.15.150:0/2085161808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561987892050 0x56198789fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.374+0000 7f6f010c7700 1 -- 172.21.15.150:0/2085161808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561987892050 msgr2=0x56198789fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.374+0000 7f6f010c7700 1 --2- 172.21.15.150:0/2085161808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561987892050 0x56198789fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.374+0000 7f6f010c7700 1 -- 172.21.15.150:0/2085161808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987891180 msgr2=0x56198789f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.374+0000 7f6f010c7700 1 --2- 172.21.15.150:0/2085161808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987891180 0x56198789f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.374+0000 7f6f010c7700 1 -- 172.21.15.150:0/2085161808 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561987632350 con 0x5619878929a0 2024-05-10T04:06:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.374+0000 7f6f010c7700 1 --2- 172.21.15.150:0/2085161808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619878929a0 0x5619878a0dd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4095c20 tx=0x7f6ef4095460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80c007527b43634f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.375+0000 7f6edbfff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:06:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.375+0000 7f6edbfff700 1 -- 172.21.15.150:0/2085161808 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400d040 con 0x5619878929a0 2024-05-10T04:06:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.375+0000 7f6edbfff700 1 -- 172.21.15.150:0/2085161808 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6ef40a1bf0 con 0x5619878929a0 2024-05-10T04:06:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.375+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/2085161808 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ef0012970 con 0x5619878929a0 2024-05-10T04:06:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.375+0000 7f6edbfff700 1 -- 172.21.15.150:0/2085161808 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ef409ced0 con 0x5619878929a0 2024-05-10T04:06:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.376+0000 7f6f12e6eb80 10 client.25913 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.376+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/2085161808 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561987636c70 con 0x5619878929a0 2024-05-10T04:06:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.376+0000 7f6edbfff700 1 -- 172.21.15.150:0/2085161808 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ef0012970 con 0x5619878929a0 2024-05-10T04:06:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.379+0000 7f6edbfff700 1 -- 172.21.15.150:0/2085161808 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ef4098c60 con 0x5619878929a0 2024-05-10T04:06:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.379+0000 7f6f12e6eb80 10 client.25913 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.379+0000 7f6f12e6eb80 10 client.25913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.379+0000 7f6f12e6eb80 20 client.25913 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.379+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/2085161808 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5619878fd390 0x5619878ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.379+0000 7f6f12e6eb80 4 client.25913 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:06:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.379+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/2085161808 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5619877a9e60 con 0x5619878fd390 2024-05-10T04:06:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.380+0000 7f6f008c6700 1 --2- 172.21.15.150:0/2085161808 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5619878fd390 0x5619878ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.383+0000 7f6f008c6700 1 --2- 172.21.15.150:0/2085161808 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5619878fd390 0x5619878ff7b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d086bec1ba938ed1 server_cookie=ea5794340e2b358b in_seq=0 out_seq=0 2024-05-10T04:06:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.383+0000 7f6edbfff700 10 client.25913 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.528+0000 7f6edbfff700 1 -- 172.21.15.150:0/2085161808 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5619877a9e60 con 0x5619878fd390 2024-05-10T04:06:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.528+0000 7f6edbfff700 10 client.25913 handle_command_reply: tid=0 2024-05-10T04:06:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.528+0000 7f6f12e6eb80 1 client.25913 shutdown 2024-05-10T04:06:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.535+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/2085161808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619878929a0 msgr2=0x5619878a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.535+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/2085161808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619878929a0 0x5619878a0dd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4095c20 tx=0x7f6ef4095460 comp rx=0 tx=0).stop 2024-05-10T04:06:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.535+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/2085161808 shutdown_connections 2024-05-10T04:06:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.535+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/2085161808 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5619878fd390 0x5619878ff7b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.535+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/2085161808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5619878929a0 0x5619878a0dd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.535+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/2085161808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561987891180 0x56198789f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.535+0000 7f6f12e6eb80 1 --2- 172.21.15.150:0/2085161808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561987892050 0x56198789fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.535+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/2085161808 >> 172.21.15.150:0/2085161808 conn(0x5619877f3a00 msgr2=0x561987895eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.536+0000 7f6edbfff700 0 client.25913 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.547+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/2085161808 shutdown_connections 2024-05-10T04:06:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.550+0000 7f6f12e6eb80 1 -- 172.21.15.150:0/2085161808 wait complete. 2024-05-10T04:06:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.552+0000 7f6f12e6eb80 20 client.25913 trim_cache size 0 max 16384 2024-05-10T04:06:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:06:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:06:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:06:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:06:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.554+0000 7f6f02b2a700 1 -- 172.21.15.150:0/1146003265 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6ee40a1270 msgr2=0x7f6ee40a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.554+0000 7f6f02b2a700 1 --2- 172.21.15.150:0/1146003265 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6ee40a1270 0x7f6ee40a3780 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6efc110f30 tx=0x7f6ef0006c60 comp rx=0 tx=0).stop 2024-05-10T04:06:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.554+0000 7f6f02b2a700 1 -- 172.21.15.150:0/1146003265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6efc1018d0 msgr2=0x7f6efc10fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.554+0000 7f6f02b2a700 1 --2- 172.21.15.150:0/1146003265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6efc1018d0 0x7f6efc10fd70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400b4a0 tx=0x7f6ef400b860 comp rx=0 tx=0).stop 2024-05-10T04:06:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.565+0000 7f6f02b2a700 1 -- 172.21.15.150:0/1146003265 shutdown_connections 2024-05-10T04:06:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.565+0000 7f6f02b2a700 1 --2- 172.21.15.150:0/1146003265 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6ee40a1270 0x7f6ee40a3780 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.565+0000 7f6f02b2a700 1 --2- 172.21.15.150:0/1146003265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6efc1018d0 0x7f6efc10fd70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.565+0000 7f6f02b2a700 1 --2- 172.21.15.150:0/1146003265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc0ffe10 0x7f6efc111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.565+0000 7f6f02b2a700 1 --2- 172.21.15.150:0/1146003265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6efc100f30 0x7f6efc111ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.565+0000 7f6f02b2a700 1 -- 172.21.15.150:0/1146003265 >> 172.21.15.150:0/1146003265 conn(0x7f6efc0fb420 msgr2=0x7f6efc1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:38.577 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.575+0000 7f6f02b2a700 1 -- 172.21.15.150:0/1146003265 shutdown_connections 2024-05-10T04:06:38.577 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:38.575+0000 7f6f02b2a700 1 -- 172.21.15.150:0/1146003265 wait complete. 2024-05-10T04:06:40.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:40 smithi150 ceph-mon[42477]: pgmap v352: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 706 MiB/s wr, 194 op/s 2024-05-10T04:06:40.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:40 smithi181 ceph-mon[46602]: pgmap v352: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 706 MiB/s wr, 194 op/s 2024-05-10T04:06:40.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:40 smithi161 ceph-mon[46918]: pgmap v352: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 706 MiB/s wr, 194 op/s 2024-05-10T04:06:42.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:42 smithi150 ceph-mon[42477]: pgmap v353: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 421 MiB/s wr, 116 op/s 2024-05-10T04:06:42.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:42 smithi181 ceph-mon[46602]: pgmap v353: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 421 MiB/s wr, 116 op/s 2024-05-10T04:06:42.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:42 smithi161 ceph-mon[46918]: pgmap v353: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 421 MiB/s wr, 116 op/s 2024-05-10T04:06:42.975 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:06:42.975 INFO:tasks.workunit.client.0.smithi150.stdout: Children see throughput for 1 rewriters = 1023302.00 kB/sec 2024-05-10T04:06:42.975 INFO:tasks.workunit.client.0.smithi150.stdout: Parent sees throughput for 1 rewriters = 1023275.82 kB/sec 2024-05-10T04:06:42.975 INFO:tasks.workunit.client.0.smithi150.stdout: Min throughput per process = 1023302.00 kB/sec 2024-05-10T04:06:42.975 INFO:tasks.workunit.client.0.smithi150.stdout: Max throughput per process = 1023302.00 kB/sec 2024-05-10T04:06:42.975 INFO:tasks.workunit.client.0.smithi150.stdout: Avg throughput per process = 1023302.00 kB/sec 2024-05-10T04:06:42.975 INFO:tasks.workunit.client.0.smithi150.stdout: Min xfer = 10485760.00 kB 2024-05-10T04:06:44.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:44 smithi150 ceph-mon[42477]: pgmap v354: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 670 MiB/s wr, 177 op/s 2024-05-10T04:06:44.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:44 smithi181 ceph-mon[46602]: pgmap v354: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 670 MiB/s wr, 177 op/s 2024-05-10T04:06:44.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:44 smithi161 ceph-mon[46918]: pgmap v354: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 670 MiB/s wr, 177 op/s 2024-05-10T04:06:46.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:46 smithi150 ceph-mon[42477]: pgmap v355: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 839 MiB/s wr, 220 op/s 2024-05-10T04:06:46.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:46 smithi181 ceph-mon[46602]: pgmap v355: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 839 MiB/s wr, 220 op/s 2024-05-10T04:06:46.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:46 smithi161 ceph-mon[46918]: pgmap v355: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 839 MiB/s wr, 220 op/s 2024-05-10T04:06:48.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:48 smithi150 ceph-mon[42477]: pgmap v356: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 841 MiB/s wr, 219 op/s 2024-05-10T04:06:48.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:48 smithi181 ceph-mon[46602]: pgmap v356: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 841 MiB/s wr, 219 op/s 2024-05-10T04:06:48.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:48 smithi161 ceph-mon[46918]: pgmap v356: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 841 MiB/s wr, 219 op/s 2024-05-10T04:06:50.246 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:06:50.246 INFO:tasks.workunit.client.0.smithi150.stdout: Children see throughput for 1 readers = 7487715.00 kB/sec 2024-05-10T04:06:50.246 INFO:tasks.workunit.client.0.smithi150.stdout: Parent sees throughput for 1 readers = 7485807.51 kB/sec 2024-05-10T04:06:50.246 INFO:tasks.workunit.client.0.smithi150.stdout: Min throughput per process = 7487715.00 kB/sec 2024-05-10T04:06:50.246 INFO:tasks.workunit.client.0.smithi150.stdout: Max throughput per process = 7487715.00 kB/sec 2024-05-10T04:06:50.246 INFO:tasks.workunit.client.0.smithi150.stdout: Avg throughput per process = 7487715.00 kB/sec 2024-05-10T04:06:50.247 INFO:tasks.workunit.client.0.smithi150.stdout: Min xfer = 10485760.00 kB 2024-05-10T04:06:50.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:50 smithi150 ceph-mon[42477]: pgmap v357: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 726 MiB/s wr, 190 op/s 2024-05-10T04:06:50.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:50 smithi181 ceph-mon[46602]: pgmap v357: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 726 MiB/s wr, 190 op/s 2024-05-10T04:06:50.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:50 smithi161 ceph-mon[46918]: pgmap v357: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 726 MiB/s wr, 190 op/s 2024-05-10T04:06:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.747+0000 7fe0c9a99700 1 -- 172.21.15.150:0/3323484683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c4101020 msgr2=0x7fe0c4101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.747+0000 7fe0c9a99700 1 --2- 172.21.15.150:0/3323484683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c4101020 0x7fe0c4101430 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009990 tx=0x7fe0b8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.747+0000 7fe0c9a99700 1 -- 172.21.15.150:0/3323484683 shutdown_connections 2024-05-10T04:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.747+0000 7fe0c9a99700 1 --2- 172.21.15.150:0/3323484683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0c4102560 0x7fe0c4108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.747+0000 7fe0c9a99700 1 --2- 172.21.15.150:0/3323484683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c4101020 0x7fe0c4101430 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.747+0000 7fe0c9a99700 1 --2- 172.21.15.150:0/3323484683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0c4101a00 0x7fe0c4101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.747+0000 7fe0c9a99700 1 -- 172.21.15.150:0/3323484683 >> 172.21.15.150:0/3323484683 conn(0x7fe0c40fb840 msgr2=0x7fe0c40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.747+0000 7fe0c9a99700 1 -- 172.21.15.150:0/3323484683 shutdown_connections 2024-05-10T04:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.747+0000 7fe0c9a99700 1 -- 172.21.15.150:0/3323484683 wait complete. 2024-05-10T04:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.748+0000 7fe0c9a99700 1 Processor -- start 2024-05-10T04:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.748+0000 7fe0c9a99700 1 -- start start 2024-05-10T04:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.748+0000 7fe0c9a99700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0c4101020 0x7fe0c4196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.748+0000 7fe0c9a99700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c4101a00 0x7fe0c41974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.748+0000 7fe0c9a99700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0c4102560 0x7fe0c419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.748+0000 7fe0c9a99700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c419ea20 con 0x7fe0c4101a00 2024-05-10T04:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.748+0000 7fe0c9a99700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c419eb60 con 0x7fe0c4101020 2024-05-10T04:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.748+0000 7fe0c9a99700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c419ecf0 con 0x7fe0c4102560 2024-05-10T04:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.748+0000 7fe0c2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c4101a00 0x7fe0c41974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.748+0000 7fe0c2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c4101a00 0x7fe0c41974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52354/0 (socket says 172.21.15.150:52354) 2024-05-10T04:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.748+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/852152279 learned_addr learned my addr 172.21.15.150:0/852152279 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.748+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0c4101020 0x7fe0c4196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.749+0000 7fe0c3fff700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0c4102560 0x7fe0c419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.749+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/852152279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0c4102560 msgr2=0x7fe0c419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.749+0000 7fe0c2ffd700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0c4102560 0x7fe0c419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.749+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/852152279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0c4101020 msgr2=0x7fe0c4196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.749+0000 7fe0c2ffd700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0c4101020 0x7fe0c4196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.749+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/852152279 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b80095f0 con 0x7fe0c4101a00 2024-05-10T04:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.749+0000 7fe0c3fff700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0c4102560 0x7fe0c419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.749+0000 7fe0c2ffd700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c4101a00 0x7fe0c41974c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400e9d0 tx=0x7fe0b400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.749+0000 7fe0c0ff9700 1 -- 172.21.15.150:0/852152279 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4009790 con 0x7fe0c4101a00 2024-05-10T04:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.749+0000 7fe0c0ff9700 1 -- 172.21.15.150:0/852152279 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0b400cbd0 con 0x7fe0c4101a00 2024-05-10T04:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.749+0000 7fe0c9a99700 1 -- 172.21.15.150:0/852152279 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c419efd0 con 0x7fe0c4101a00 2024-05-10T04:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.749+0000 7fe0c0ff9700 1 -- 172.21.15.150:0/852152279 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4018680 con 0x7fe0c4101a00 2024-05-10T04:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.749+0000 7fe0c9a99700 1 -- 172.21.15.150:0/852152279 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c419f400 con 0x7fe0c4101a00 2024-05-10T04:06:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.751+0000 7fe0c0ff9700 1 -- 172.21.15.150:0/852152279 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe0b40187e0 con 0x7fe0c4101a00 2024-05-10T04:06:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.752+0000 7fe0c0ff9700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe0ac0a1120 0x7fe0ac0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.752+0000 7fe0c0ff9700 1 -- 172.21.15.150:0/852152279 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe0b4014070 con 0x7fe0c4101a00 2024-05-10T04:06:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.752+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe0ac0a1120 0x7fe0ac0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.753+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe0ac0a1120 0x7fe0ac0a3630 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8006010 tx=0x7fe0b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.753+0000 7fe0d9dddb80 1 Processor -- start 2024-05-10T04:06:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.753+0000 7fe0d9dddb80 1 -- start start 2024-05-10T04:06:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.753+0000 7fe0d9dddb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc6390 0x555c5acc67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.753+0000 7fe0d9dddb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acc7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.753+0000 7fe0d9dddb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.753+0000 7fe0d9dddb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5abeb0f0 con 0x555c5acc7260 2024-05-10T04:06:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.753+0000 7fe0d9dddb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5abe8ab0 con 0x555c5acc7bb0 2024-05-10T04:06:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.753+0000 7fe0d9dddb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5abe43a0 con 0x555c5acc6390 2024-05-10T04:06:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.753+0000 7fe0c2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acc7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.753+0000 7fe0c2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acc7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52364/0 (socket says 172.21.15.150:52364) 2024-05-10T04:06:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.753+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/3248580685 learned_addr learned my addr 172.21.15.150:0/3248580685 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.754+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/3248580685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc6390 0x555c5acc67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.754+0000 7fe0c3fff700 1 --2- 172.21.15.150:0/3248580685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.754+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/3248580685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc6390 msgr2=0x555c5acc67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.754+0000 7fe0c2ffd700 1 --2- 172.21.15.150:0/3248580685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc6390 0x555c5acc67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.754+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/3248580685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 msgr2=0x555c5acce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.754+0000 7fe0c2ffd700 1 --2- 172.21.15.150:0/3248580685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.754+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/3248580685 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c5aa65350 con 0x555c5acc7260 2024-05-10T04:06:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.754+0000 7fe0c3fff700 1 --2- 172.21.15.150:0/3248580685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:06:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.754+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/3248580685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc6390 0x555c5acc67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:06:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.754+0000 7fe0c2ffd700 1 --2- 172.21.15.150:0/3248580685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acc7670 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4099c70 tx=0x7fe0b40998d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e035c768d499d5d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.754+0000 7fe0a9ffb700 1 -- 172.21.15.150:0/3248580685 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4099f70 con 0x555c5acc7260 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.754+0000 7fe0a9ffb700 1 -- 172.21.15.150:0/3248580685 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0b409dda0 con 0x555c5acc7260 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.754+0000 7fe0a9ffb700 1 -- 172.21.15.150:0/3248580685 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40a1f60 con 0x555c5acc7260 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.755+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/3248580685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 msgr2=0x555c5acc7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.755+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/3248580685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acc7670 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4099c70 tx=0x7fe0b40998d0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.755+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/3248580685 shutdown_connections 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.755+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/3248580685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.755+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/3248580685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc6390 0x555c5acc67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.755+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/3248580685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acc7670 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.755+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/3248580685 >> 172.21.15.150:0/3248580685 conn(0x555c5ac26a00 msgr2=0x555c5acbea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.755+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/3248580685 shutdown_connections 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.755+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/3248580685 wait complete. 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.755+0000 7fe0d9dddb80 1 -- start start 2024-05-10T04:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0d9dddb80 1 Processor -- start 2024-05-10T04:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0d9dddb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc6390 0x555c5acd0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0d9dddb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acd09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0d9dddb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acd7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0d9dddb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5ab80500 con 0x555c5acc7260 2024-05-10T04:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0d9dddb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5ab801a0 con 0x555c5acc7bb0 2024-05-10T04:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0d9dddb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5aa67640 con 0x555c5acc6390 2024-05-10T04:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0d9dddb80 10 client.0 _ll_register_callbacks cb 0x555c5ab89ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0c2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acd09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0c2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acd09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52372/0 (socket says 172.21.15.150:52372) 2024-05-10T04:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/212005781 learned_addr learned my addr 172.21.15.150:0/212005781 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/212005781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc6390 0x555c5acd0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0c3fff700 1 --2- 172.21.15.150:0/212005781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acd7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/212005781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc6390 msgr2=0x555c5acd0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0c2ffd700 1 --2- 172.21.15.150:0/212005781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc6390 0x555c5acd0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/212005781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 msgr2=0x555c5acd7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0c2ffd700 1 --2- 172.21.15.150:0/212005781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acd7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/212005781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c419f400 con 0x555c5acc7260 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0c2ffd700 1 --2- 172.21.15.150:0/212005781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acd09a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4096470 tx=0x7fe0b4010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa10da22e8b9ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.756+0000 7fe0a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0a2ffd700 1 -- 172.21.15.150:0/212005781 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4099f70 con 0x555c5acc7260 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0a2ffd700 1 -- 172.21.15.150:0/212005781 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe0b40ade40 con 0x555c5acc7260 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/212005781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c5ab3d210 con 0x555c5acc7260 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0a2ffd700 1 -- 172.21.15.150:0/212005781 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0b40a1f60 con 0x555c5acc7260 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0d9dddb80 10 client.16284 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/212005781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555c5aa11b00 con 0x555c5acc7260 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0a2ffd700 1 -- 172.21.15.150:0/212005781 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c5ab3d210 con 0x555c5acc7260 2024-05-10T04:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0a2ffd700 1 -- 172.21.15.150:0/212005781 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe0b40caae0 con 0x555c5acc7260 2024-05-10T04:06:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0d9dddb80 10 client.16284 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0d9dddb80 10 client.16284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0d9dddb80 20 client.16284 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/212005781 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x555c5acdbd40 0x555c5acde190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0d9dddb80 4 client.16284 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.757+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/212005781 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c5ac93720 con 0x555c5acdbd40 2024-05-10T04:06:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.758+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/212005781 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x555c5acdbd40 0x555c5acde190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.758+0000 7fe0a2ffd700 10 client.16284 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.758+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/212005781 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x555c5acdbd40 0x555c5acde190 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee62e75d85290b4 server_cookie=71ffbb8f50e86329 in_seq=0 out_seq=0 2024-05-10T04:06:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0a2ffd700 1 -- 172.21.15.150:0/212005781 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c5ac93720 con 0x555c5acdbd40 2024-05-10T04:06:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0a2ffd700 10 client.16284 handle_command_reply: tid=0 2024-05-10T04:06:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0d9dddb80 1 client.16284 shutdown 2024-05-10T04:06:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/212005781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 msgr2=0x555c5acd09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/212005781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acd09a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4096470 tx=0x7fe0b4010b10 comp rx=0 tx=0).stop 2024-05-10T04:06:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/212005781 shutdown_connections 2024-05-10T04:06:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/212005781 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x555c5acdbd40 0x555c5acde190 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/212005781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acd7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/212005781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc6390 0x555c5acd0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0a2ffd700 0 client.16284 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/212005781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acd09a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/212005781 >> 172.21.15.150:0/212005781 conn(0x555c5ac26a00 msgr2=0x555c5acbea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/212005781 shutdown_connections 2024-05-10T04:06:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/212005781 wait complete. 2024-05-10T04:06:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.759+0000 7fe0d9dddb80 20 client.16284 trim_cache size 0 max 16384 2024-05-10T04:06:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.774+0000 7fe0d9dddb80 1 Processor -- start 2024-05-10T04:06:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.774+0000 7fe0d9dddb80 1 -- start start 2024-05-10T04:06:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.774+0000 7fe0d9dddb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc6390 0x555c5acc7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.774+0000 7fe0d9dddb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc7260 0x555c5acd78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.774+0000 7fe0d9dddb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acde160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.774+0000 7fe0d9dddb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5acc7fc0 con 0x555c5acc6390 2024-05-10T04:06:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.774+0000 7fe0d9dddb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5accbd70 con 0x555c5acc7bb0 2024-05-10T04:06:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.774+0000 7fe0d9dddb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5acd00e0 con 0x555c5acc7260 2024-05-10T04:06:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.774+0000 7fe0c37fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc6390 0x555c5acc7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.774+0000 7fe0c37fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc6390 0x555c5acc7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52380/0 (socket says 172.21.15.150:52380) 2024-05-10T04:06:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.774+0000 7fe0c37fe700 1 -- 172.21.15.150:0/1892448810 learned_addr learned my addr 172.21.15.150:0/1892448810 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.774+0000 7fe0c2ffd700 1 --2- 172.21.15.150:0/1892448810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc7260 0x555c5acd78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.775+0000 7fe0c37fe700 1 -- 172.21.15.150:0/1892448810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc7260 msgr2=0x555c5acd78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.775+0000 7fe0c3fff700 1 --2- 172.21.15.150:0/1892448810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acde160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.775+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/1892448810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc7260 0x555c5acd78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.775+0000 7fe0c37fe700 1 -- 172.21.15.150:0/1892448810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 msgr2=0x555c5acde160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.775+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/1892448810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acde160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.775+0000 7fe0c37fe700 1 -- 172.21.15.150:0/1892448810 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c5abaa500 con 0x555c5acc6390 2024-05-10T04:06:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.775+0000 7fe0c3fff700 1 --2- 172.21.15.150:0/1892448810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acde160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:06:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.775+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/1892448810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc6390 0x555c5acc7670 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8005f50 tx=0x7fe0b801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8feb34a6f6ddbe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.775+0000 7fe0aa7fc700 1 -- 172.21.15.150:0/1892448810 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8024b20 con 0x555c5acc6390 2024-05-10T04:06:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.775+0000 7fe0aa7fc700 1 -- 172.21.15.150:0/1892448810 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0b802c040 con 0x555c5acc6390 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.775+0000 7fe0aa7fc700 1 -- 172.21.15.150:0/1892448810 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800f6c0 con 0x555c5acc6390 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/1892448810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc6390 msgr2=0x555c5acc7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/1892448810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc6390 0x555c5acc7670 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8005f50 tx=0x7fe0b801f000 comp rx=0 tx=0).stop 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/1892448810 shutdown_connections 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/1892448810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc7bb0 0x555c5acde160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/1892448810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc6390 0x555c5acc7670 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/1892448810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc7260 0x555c5acd78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/1892448810 >> 172.21.15.150:0/1892448810 conn(0x555c5ac26a00 msgr2=0x555c5accf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/1892448810 shutdown_connections 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/1892448810 wait complete. 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 -- start start 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 Processor -- start 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc6390 0x555c5acd4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acd53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc7bb0 0x555c5acd6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5acd6850 con 0x555c5acc7260 2024-05-10T04:06:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5acd69c0 con 0x555c5acc6390 2024-05-10T04:06:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.776+0000 7fe0d9dddb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5acd6b80 con 0x555c5acc7bb0 2024-05-10T04:06:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0d9dddb80 10 client.0 _ll_register_callbacks cb 0x555c5ab89ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0c2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acd53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0c2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acd53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52390/0 (socket says 172.21.15.150:52390) 2024-05-10T04:06:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/2661124608 learned_addr learned my addr 172.21.15.150:0/2661124608 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/2661124608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc6390 0x555c5acd4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0c3fff700 1 --2- 172.21.15.150:0/2661124608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc7bb0 0x555c5acd6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/2661124608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc7bb0 msgr2=0x555c5acd6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0c2ffd700 1 --2- 172.21.15.150:0/2661124608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc7bb0 0x555c5acd6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/2661124608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc6390 msgr2=0x555c5acd4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0c2ffd700 1 --2- 172.21.15.150:0/2661124608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc6390 0x555c5acd4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0c2ffd700 1 -- 172.21.15.150:0/2661124608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c41098d0 con 0x555c5acc7260 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0c2ffd700 1 --2- 172.21.15.150:0/2661124608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acd53e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40caec0 tx=0x7fe0b40998d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58466a9d8ceb0ce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0a2ffd700 1 -- 172.21.15.150:0/2661124608 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4095db0 con 0x555c5acc7260 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0a2ffd700 1 -- 172.21.15.150:0/2661124608 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe0b40ade40 con 0x555c5acc7260 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/2661124608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0b409dda0 con 0x555c5acc7260 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0a2ffd700 1 -- 172.21.15.150:0/2661124608 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0b40056c0 con 0x555c5acc7260 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0d9dddb80 10 client.16290 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.777+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/2661124608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555c5aa69c70 con 0x555c5acc7260 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.778+0000 7fe0a2ffd700 1 -- 172.21.15.150:0/2661124608 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0b409dda0 con 0x555c5acc7260 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.778+0000 7fe0a2ffd700 1 -- 172.21.15.150:0/2661124608 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe0b4094b80 con 0x555c5acc7260 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.778+0000 7fe0d9dddb80 10 client.16290 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.778+0000 7fe0d9dddb80 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.778+0000 7fe0d9dddb80 20 client.16290 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.778+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/2661124608 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x555c5ad32240 0x555c5ad34660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.778+0000 7fe0d9dddb80 4 client.16290 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:06:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.778+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/2661124608 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555c5abdce60 con 0x555c5ad32240 2024-05-10T04:06:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.778+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/2661124608 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x555c5ad32240 0x555c5ad34660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.779+0000 7fe0a2ffd700 10 client.16290 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.779+0000 7fe0c37fe700 1 --2- 172.21.15.150:0/2661124608 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x555c5ad32240 0x555c5ad34660 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=974277e1bc91b407 server_cookie=5ea0c11c1d4acf67 in_seq=0 out_seq=0 2024-05-10T04:06:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0a2ffd700 1 -- 172.21.15.150:0/2661124608 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555c5abdce60 con 0x555c5ad32240 2024-05-10T04:06:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0a2ffd700 10 client.16290 handle_command_reply: tid=0 2024-05-10T04:06:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0d9dddb80 1 client.16290 shutdown 2024-05-10T04:06:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/2661124608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 msgr2=0x555c5acd53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/2661124608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acd53e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40caec0 tx=0x7fe0b40998d0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/2661124608 shutdown_connections 2024-05-10T04:06:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/2661124608 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x555c5ad32240 0x555c5ad34660 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/2661124608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c5acc7bb0 0x555c5acd6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0a2ffd700 0 client.16290 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/2661124608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c5acc6390 0x555c5acd4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0d9dddb80 1 --2- 172.21.15.150:0/2661124608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c5acc7260 0x555c5acd53e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/2661124608 >> 172.21.15.150:0/2661124608 conn(0x555c5ac26a00 msgr2=0x555c5accf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/2661124608 shutdown_connections 2024-05-10T04:06:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0d9dddb80 1 -- 172.21.15.150:0/2661124608 wait complete. 2024-05-10T04:06:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.787+0000 7fe0d9dddb80 20 client.16290 trim_cache size 0 max 16384 2024-05-10T04:06:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:06:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:06:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:06:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:06:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.788+0000 7fe0c9a99700 1 -- 172.21.15.150:0/852152279 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe0ac0a1120 msgr2=0x7fe0ac0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.788+0000 7fe0c9a99700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe0ac0a1120 0x7fe0ac0a3630 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8006010 tx=0x7fe0b801a040 comp rx=0 tx=0).stop 2024-05-10T04:06:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.788+0000 7fe0c9a99700 1 -- 172.21.15.150:0/852152279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c4101a00 msgr2=0x7fe0c41974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.788+0000 7fe0c9a99700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c4101a00 0x7fe0c41974c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400e9d0 tx=0x7fe0b400ece0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.788+0000 7fe0c9a99700 1 -- 172.21.15.150:0/852152279 shutdown_connections 2024-05-10T04:06:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.788+0000 7fe0c9a99700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe0ac0a1120 0x7fe0ac0a3630 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.788+0000 7fe0c9a99700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0c4102560 0x7fe0c419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.788+0000 7fe0c9a99700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0c4101020 0x7fe0c4196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.788+0000 7fe0c9a99700 1 --2- 172.21.15.150:0/852152279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c4101a00 0x7fe0c41974c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.788+0000 7fe0c9a99700 1 -- 172.21.15.150:0/852152279 >> 172.21.15.150:0/852152279 conn(0x7fe0c40fb840 msgr2=0x7fe0c4104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.788+0000 7fe0c9a99700 1 -- 172.21.15.150:0/852152279 shutdown_connections 2024-05-10T04:06:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:06:51.788+0000 7fe0c9a99700 1 -- 172.21.15.150:0/852152279 wait complete. 2024-05-10T04:06:52.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:52 smithi150 ceph-mon[42477]: pgmap v358: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 492 MiB/s wr, 131 op/s 2024-05-10T04:06:52.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:52 smithi181 ceph-mon[46602]: pgmap v358: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 492 MiB/s wr, 131 op/s 2024-05-10T04:06:52.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:52 smithi161 ceph-mon[46918]: pgmap v358: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 492 MiB/s wr, 131 op/s 2024-05-10T04:06:54.037 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:06:54.037 INFO:tasks.workunit.client.0.smithi150.stdout: Children see throughput for 1 re-readers = 6097099.00 kB/sec 2024-05-10T04:06:54.037 INFO:tasks.workunit.client.0.smithi150.stdout: Parent sees throughput for 1 re-readers = 6096007.12 kB/sec 2024-05-10T04:06:54.037 INFO:tasks.workunit.client.0.smithi150.stdout: Min throughput per process = 6097099.00 kB/sec 2024-05-10T04:06:54.037 INFO:tasks.workunit.client.0.smithi150.stdout: Max throughput per process = 6097099.00 kB/sec 2024-05-10T04:06:54.038 INFO:tasks.workunit.client.0.smithi150.stdout: Avg throughput per process = 6097099.00 kB/sec 2024-05-10T04:06:54.038 INFO:tasks.workunit.client.0.smithi150.stdout: Min xfer = 10485760.00 kB 2024-05-10T04:06:54.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:54 smithi150 ceph-mon[42477]: pgmap v359: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 480 MiB/s wr, 127 op/s 2024-05-10T04:06:54.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:54 smithi181 ceph-mon[46602]: pgmap v359: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 480 MiB/s wr, 127 op/s 2024-05-10T04:06:54.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:54 smithi161 ceph-mon[46918]: pgmap v359: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 480 MiB/s wr, 127 op/s 2024-05-10T04:06:56.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:56 smithi150 ceph-mon[42477]: pgmap v360: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 220 MiB/s wr, 63 op/s 2024-05-10T04:06:56.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:56 smithi181 ceph-mon[46602]: pgmap v360: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 220 MiB/s wr, 63 op/s 2024-05-10T04:06:56.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:56 smithi161 ceph-mon[46918]: pgmap v360: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 220 MiB/s wr, 63 op/s 2024-05-10T04:06:57.162 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:06:57.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.310+0000 7f5eef3cd700 1 -- 172.21.15.150:0/3729919221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ee8102950 msgr2=0x7f5ee8109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.310+0000 7f5eef3cd700 1 --2- 172.21.15.150:0/3729919221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ee8102950 0x7f5ee8109250 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000b0a0 tx=0x7f5ee000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.310+0000 7f5eef3cd700 1 -- 172.21.15.150:0/3729919221 shutdown_connections 2024-05-10T04:06:57.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.310+0000 7f5eef3cd700 1 --2- 172.21.15.150:0/3729919221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ee8102950 0x7f5ee8109250 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.310+0000 7f5eef3cd700 1 --2- 172.21.15.150:0/3729919221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ee8101110 0x7f5ee8101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.310+0000 7f5eef3cd700 1 --2- 172.21.15.150:0/3729919221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee8102000 0x7f5ee8102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.310+0000 7f5eef3cd700 1 -- 172.21.15.150:0/3729919221 >> 172.21.15.150:0/3729919221 conn(0x7f5ee80fb8f0 msgr2=0x7f5ee80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.310+0000 7f5eef3cd700 1 -- 172.21.15.150:0/3729919221 shutdown_connections 2024-05-10T04:06:57.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.310+0000 7f5eef3cd700 1 -- 172.21.15.150:0/3729919221 wait complete. 2024-05-10T04:06:57.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.311+0000 7f5eef3cd700 1 Processor -- start 2024-05-10T04:06:57.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.311+0000 7f5eef3cd700 1 -- start start 2024-05-10T04:06:57.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eef3cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ee8101110 0x7f5ee8197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eef3cd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ee8102000 0x7f5ee8197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eef3cd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee8102950 0x7f5ee819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eef3cd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee819ebe0 con 0x7f5ee8102000 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eef3cd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee819ed50 con 0x7f5ee8102950 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eef3cd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee819ef10 con 0x7f5ee8101110 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eec968700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ee8102000 0x7f5ee8197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eec968700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ee8102000 0x7f5ee8197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52398/0 (socket says 172.21.15.150:52398) 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eec968700 1 -- 172.21.15.150:0/979070206 learned_addr learned my addr 172.21.15.150:0/979070206 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eed169700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ee8101110 0x7f5ee8197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eed96a700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee8102950 0x7f5ee819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eec968700 1 -- 172.21.15.150:0/979070206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ee8101110 msgr2=0x7f5ee8197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eec968700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ee8101110 0x7f5ee8197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eec968700 1 -- 172.21.15.150:0/979070206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee8102950 msgr2=0x7f5ee819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eec968700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee8102950 0x7f5ee819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eec968700 1 -- 172.21.15.150:0/979070206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee0009d00 con 0x7f5ee8102000 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eed96a700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee8102950 0x7f5ee819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eec968700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ee8102000 0x7f5ee8197640 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400d950 tx=0x7f5ee400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5ede7fc700 1 -- 172.21.15.150:0/979070206 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4009940 con 0x7f5ee8102000 2024-05-10T04:06:57.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5ede7fc700 1 -- 172.21.15.150:0/979070206 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ee4010460 con 0x7f5ee8102000 2024-05-10T04:06:57.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5ede7fc700 1 -- 172.21.15.150:0/979070206 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee400f620 con 0x7f5ee8102000 2024-05-10T04:06:57.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.312+0000 7f5eef3cd700 1 -- 172.21.15.150:0/979070206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee819f1f0 con 0x7f5ee8102000 2024-05-10T04:06:57.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.313+0000 7f5eef3cd700 1 -- 172.21.15.150:0/979070206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee819f6b0 con 0x7f5ee8102000 2024-05-10T04:06:57.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.315+0000 7f5ede7fc700 1 -- 172.21.15.150:0/979070206 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5ee40108d0 con 0x7f5ee8102000 2024-05-10T04:06:57.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.316+0000 7f5ede7fc700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5ed40a1240 0x7f5ed40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.316+0000 7f5eed169700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5ed40a1240 0x7f5ed40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.316+0000 7f5ede7fc700 1 -- 172.21.15.150:0/979070206 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5ee40cca40 con 0x7f5ee8102000 2024-05-10T04:06:57.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.316+0000 7f5eed169700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5ed40a1240 0x7f5ed40a3750 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8005950 tx=0x7f5ed800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eff711b80 1 Processor -- start 2024-05-10T04:06:57.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eff711b80 1 -- start start 2024-05-10T04:06:57.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eff711b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b0076a0 0x55ec3b007af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eff711b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 0x55ec3b008980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eff711b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b008ec0 0x55ec3b00f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eff711b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3af2e150 con 0x55ec3b008570 2024-05-10T04:06:57.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eff711b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3af2bb10 con 0x55ec3b0076a0 2024-05-10T04:06:57.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eff711b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3af27400 con 0x55ec3b008ec0 2024-05-10T04:06:57.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eec968700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 0x55ec3b008980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eec968700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 0x55ec3b008980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52400/0 (socket says 172.21.15.150:52400) 2024-05-10T04:06:57.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eec968700 1 -- 172.21.15.150:0/1930489262 learned_addr learned my addr 172.21.15.150:0/1930489262 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eed169700 1 --2- 172.21.15.150:0/1930489262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b0076a0 0x55ec3b007af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eed96a700 1 --2- 172.21.15.150:0/1930489262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b008ec0 0x55ec3b00f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eec968700 1 -- 172.21.15.150:0/1930489262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b008ec0 msgr2=0x55ec3b00f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eec968700 1 --2- 172.21.15.150:0/1930489262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b008ec0 0x55ec3b00f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.317+0000 7f5eec968700 1 -- 172.21.15.150:0/1930489262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b0076a0 msgr2=0x55ec3b007af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.318+0000 7f5eec968700 1 --2- 172.21.15.150:0/1930489262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b0076a0 0x55ec3b007af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.318+0000 7f5eec968700 1 -- 172.21.15.150:0/1930489262 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3ada83b0 con 0x55ec3b008570 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.318+0000 7f5eec968700 1 --2- 172.21.15.150:0/1930489262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 0x55ec3b008980 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4096020 tx=0x7f5ee4099580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582e5334b1f98d5f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.318+0000 7f5ed37fe700 1 -- 172.21.15.150:0/1930489262 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40a1fa0 con 0x55ec3b008570 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.318+0000 7f5ed37fe700 1 -- 172.21.15.150:0/1930489262 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ee4097ec0 con 0x55ec3b008570 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.318+0000 7f5ed37fe700 1 -- 172.21.15.150:0/1930489262 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40a1b10 con 0x55ec3b008570 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.319+0000 7f5eff711b80 1 -- 172.21.15.150:0/1930489262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 msgr2=0x55ec3b008980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.319+0000 7f5eff711b80 1 --2- 172.21.15.150:0/1930489262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 0x55ec3b008980 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4096020 tx=0x7f5ee4099580 comp rx=0 tx=0).stop 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.319+0000 7f5eff711b80 1 -- 172.21.15.150:0/1930489262 shutdown_connections 2024-05-10T04:06:57.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.319+0000 7f5eff711b80 1 --2- 172.21.15.150:0/1930489262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b008ec0 0x55ec3b00f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.319+0000 7f5eff711b80 1 --2- 172.21.15.150:0/1930489262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b0076a0 0x55ec3b007af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.319+0000 7f5eff711b80 1 --2- 172.21.15.150:0/1930489262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 0x55ec3b008980 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.319+0000 7f5eff711b80 1 -- 172.21.15.150:0/1930489262 >> 172.21.15.150:0/1930489262 conn(0x55ec3af69a60 msgr2=0x55ec3b001fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.319+0000 7f5eff711b80 1 -- 172.21.15.150:0/1930489262 shutdown_connections 2024-05-10T04:06:57.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.319+0000 7f5eff711b80 1 -- 172.21.15.150:0/1930489262 wait complete. 2024-05-10T04:06:57.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.319+0000 7f5eff711b80 1 -- start start 2024-05-10T04:06:57.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.319+0000 7f5eff711b80 1 Processor -- start 2024-05-10T04:06:57.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eff711b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b0076a0 0x55ec3b0115c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eff711b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 0x55ec3b011b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eff711b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b008ec0 0x55ec3b018a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eff711b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3aec3560 con 0x55ec3b008570 2024-05-10T04:06:57.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eff711b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3aec3200 con 0x55ec3b008ec0 2024-05-10T04:06:57.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eff711b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3adaa6a0 con 0x55ec3b0076a0 2024-05-10T04:06:57.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eff711b80 10 client.0 _ll_register_callbacks cb 0x55ec3aeccb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:57.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eec968700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 0x55ec3b011b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eec968700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 0x55ec3b011b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52414/0 (socket says 172.21.15.150:52414) 2024-05-10T04:06:57.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eec968700 1 -- 172.21.15.150:0/491961460 learned_addr learned my addr 172.21.15.150:0/491961460 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:57.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eed96a700 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b008ec0 0x55ec3b018a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eed169700 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b0076a0 0x55ec3b0115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eec968700 1 -- 172.21.15.150:0/491961460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b0076a0 msgr2=0x55ec3b0115c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eec968700 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b0076a0 0x55ec3b0115c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eec968700 1 -- 172.21.15.150:0/491961460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b008ec0 msgr2=0x55ec3b018a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eec968700 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b008ec0 0x55ec3b018a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eec968700 1 -- 172.21.15.150:0/491961460 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee819f6b0 con 0x55ec3b008570 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eed96a700 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b008ec0 0x55ec3b018a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eed169700 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b0076a0 0x55ec3b0115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eec968700 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 0x55ec3b011b00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400b2f0 tx=0x7f5ee40a1fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a6b40b8854a0d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5ed0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5ed0ff9700 1 -- 172.21.15.150:0/491961460 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40a1d10 con 0x55ec3b008570 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5ed0ff9700 1 -- 172.21.15.150:0/491961460 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5ee409ab30 con 0x55ec3b008570 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.320+0000 7f5eff711b80 1 -- 172.21.15.150:0/491961460 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec3ae80270 con 0x55ec3b008570 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.321+0000 7f5ed0ff9700 1 -- 172.21.15.150:0/491961460 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ee4099f90 con 0x55ec3b008570 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.321+0000 7f5eff711b80 10 client.16302 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.321+0000 7f5eff711b80 1 -- 172.21.15.150:0/491961460 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ec3ad54b60 con 0x55ec3b008570 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.321+0000 7f5ed0ff9700 1 -- 172.21.15.150:0/491961460 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec3ae80270 con 0x55ec3b008570 2024-05-10T04:06:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.321+0000 7f5ed0ff9700 1 -- 172.21.15.150:0/491961460 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5ee4098a80 con 0x55ec3b008570 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.321+0000 7f5eff711b80 10 client.16302 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.321+0000 7f5eff711b80 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.321+0000 7f5eff711b80 20 client.16302 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.321+0000 7f5eff711b80 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55ec3b01cd40 0x55ec3b01f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.321+0000 7f5eff711b80 4 client.16302 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.321+0000 7f5eff711b80 1 -- 172.21.15.150:0/491961460 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec3afd6780 con 0x55ec3b01cd40 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.321+0000 7f5eed169700 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55ec3b01cd40 0x55ec3b01f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.322+0000 7f5ed0ff9700 10 client.16302 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.322+0000 7f5eed169700 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55ec3b01cd40 0x55ec3b01f160 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb7f710c69ea031 server_cookie=bc520b48eb806c76 in_seq=0 out_seq=0 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5ed0ff9700 1 -- 172.21.15.150:0/491961460 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec3afd6780 con 0x55ec3b01cd40 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5ed0ff9700 10 client.16302 handle_command_reply: tid=0 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5eff711b80 1 client.16302 shutdown 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5eff711b80 1 -- 172.21.15.150:0/491961460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 msgr2=0x55ec3b011b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5eff711b80 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 0x55ec3b011b00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400b2f0 tx=0x7f5ee40a1fa0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5eff711b80 1 -- 172.21.15.150:0/491961460 shutdown_connections 2024-05-10T04:06:57.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5eff711b80 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55ec3b01cd40 0x55ec3b01f160 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5eff711b80 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b008ec0 0x55ec3b018a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5eff711b80 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b0076a0 0x55ec3b0115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5ed0ff9700 0 client.16302 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:57.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5eff711b80 1 --2- 172.21.15.150:0/491961460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b008570 0x55ec3b011b00 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5eff711b80 1 -- 172.21.15.150:0/491961460 >> 172.21.15.150:0/491961460 conn(0x55ec3af69a60 msgr2=0x55ec3b001fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5eff711b80 1 -- 172.21.15.150:0/491961460 shutdown_connections 2024-05-10T04:06:57.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5eff711b80 1 -- 172.21.15.150:0/491961460 wait complete. 2024-05-10T04:06:57.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.323+0000 7f5eff711b80 20 client.16302 trim_cache size 0 max 16384 2024-05-10T04:06:57.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.340+0000 7f5eff711b80 1 Processor -- start 2024-05-10T04:06:57.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.340+0000 7f5eff711b80 1 -- start start 2024-05-10T04:06:57.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.340+0000 7f5eff711b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3af69a60 0x55ec3b00b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.340+0000 7f5eff711b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 0x55ec3b018a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.340+0000 7f5eff711b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b008570 0x55ec3b011580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.340+0000 7f5eff711b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3b019b60 con 0x55ec3b0076a0 2024-05-10T04:06:57.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.340+0000 7f5eff711b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3affc7c0 con 0x55ec3b008570 2024-05-10T04:06:57.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.340+0000 7f5eff711b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3b0092d0 con 0x55ec3af69a60 2024-05-10T04:06:57.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.340+0000 7f5eec968700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 0x55ec3b018a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.340+0000 7f5eec968700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 0x55ec3b018a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52420/0 (socket says 172.21.15.150:52420) 2024-05-10T04:06:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.340+0000 7f5eec968700 1 -- 172.21.15.150:0/4203070081 learned_addr learned my addr 172.21.15.150:0/4203070081 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.340+0000 7f5eed169700 1 --2- 172.21.15.150:0/4203070081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3af69a60 0x55ec3b00b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.341+0000 7f5eed96a700 1 --2- 172.21.15.150:0/4203070081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b008570 0x55ec3b011580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.341+0000 7f5eec968700 1 -- 172.21.15.150:0/4203070081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3af69a60 msgr2=0x55ec3b00b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.341+0000 7f5eec968700 1 --2- 172.21.15.150:0/4203070081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3af69a60 0x55ec3b00b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.341+0000 7f5eec968700 1 -- 172.21.15.150:0/4203070081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b008570 msgr2=0x55ec3b011580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.341+0000 7f5eec968700 1 --2- 172.21.15.150:0/4203070081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b008570 0x55ec3b011580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.341+0000 7f5eec968700 1 -- 172.21.15.150:0/4203070081 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3aeed560 con 0x55ec3b0076a0 2024-05-10T04:06:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.341+0000 7f5eed169700 1 --2- 172.21.15.150:0/4203070081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3af69a60 0x55ec3b00b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:06:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.341+0000 7f5eec968700 1 --2- 172.21.15.150:0/4203070081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 0x55ec3b018a20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009520 tx=0x7f5ee4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd53fe95d591bc77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.341+0000 7f5ed3fff700 1 -- 172.21.15.150:0/4203070081 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4099f90 con 0x55ec3b0076a0 2024-05-10T04:06:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.341+0000 7f5ed3fff700 1 -- 172.21.15.150:0/4203070081 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ee4099010 con 0x55ec3b0076a0 2024-05-10T04:06:57.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.341+0000 7f5ed3fff700 1 -- 172.21.15.150:0/4203070081 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40a1d70 con 0x55ec3b0076a0 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.342+0000 7f5eff711b80 1 -- 172.21.15.150:0/4203070081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 msgr2=0x55ec3b018a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.342+0000 7f5eff711b80 1 --2- 172.21.15.150:0/4203070081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 0x55ec3b018a20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009520 tx=0x7f5ee4000bc0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.342+0000 7f5eff711b80 1 -- 172.21.15.150:0/4203070081 shutdown_connections 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.342+0000 7f5eff711b80 1 --2- 172.21.15.150:0/4203070081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3b008570 0x55ec3b011580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.342+0000 7f5eff711b80 1 --2- 172.21.15.150:0/4203070081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3af69a60 0x55ec3b00b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.342+0000 7f5eff711b80 1 --2- 172.21.15.150:0/4203070081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 0x55ec3b018a20 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.342+0000 7f5eff711b80 1 -- 172.21.15.150:0/4203070081 >> 172.21.15.150:0/4203070081 conn(0x55ec3affe450 msgr2=0x55ec3b007d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.342+0000 7f5eff711b80 1 -- 172.21.15.150:0/4203070081 shutdown_connections 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.342+0000 7f5eff711b80 1 -- 172.21.15.150:0/4203070081 wait complete. 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.342+0000 7f5eff711b80 1 -- start start 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.342+0000 7f5eff711b80 1 Processor -- start 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.342+0000 7f5eff711b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3af69a60 0x55ec3b016470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.342+0000 7f5eff711b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 0x55ec3b0169b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eff711b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b008570 0x55ec3b0178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eff711b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3b017e20 con 0x55ec3b0076a0 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eff711b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3b017f90 con 0x55ec3af69a60 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eff711b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3b018150 con 0x55ec3b008570 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eff711b80 10 client.0 _ll_register_callbacks cb 0x55ec3aeccb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eec968700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 0x55ec3b0169b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eec968700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 0x55ec3b0169b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52430/0 (socket says 172.21.15.150:52430) 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eec968700 1 -- 172.21.15.150:0/1444449586 learned_addr learned my addr 172.21.15.150:0/1444449586 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eed169700 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3af69a60 0x55ec3b016470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eed96a700 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b008570 0x55ec3b0178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eec968700 1 -- 172.21.15.150:0/1444449586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b008570 msgr2=0x55ec3b0178e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eec968700 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b008570 0x55ec3b0178e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eec968700 1 -- 172.21.15.150:0/1444449586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3af69a60 msgr2=0x55ec3b016470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eec968700 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3af69a60 0x55ec3b016470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eec968700 1 -- 172.21.15.150:0/1444449586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec3ad54b60 con 0x55ec3b0076a0 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eed96a700 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b008570 0x55ec3b0178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eec968700 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 0x55ec3b0169b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4099ab0 tx=0x7f5ee40a1fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904a76e1505e213c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5ed0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5ed0ff9700 1 -- 172.21.15.150:0/1444449586 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4099f90 con 0x55ec3b0076a0 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5ed0ff9700 1 -- 172.21.15.150:0/1444449586 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5ee409ab30 con 0x55ec3b0076a0 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eff711b80 1 -- 172.21.15.150:0/1444449586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ee4097ec0 con 0x55ec3b0076a0 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5ed0ff9700 1 -- 172.21.15.150:0/1444449586 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ee409ad30 con 0x55ec3b0076a0 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eff711b80 10 client.16308 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.343+0000 7f5eff711b80 1 -- 172.21.15.150:0/1444449586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ec3adaccd0 con 0x55ec3b0076a0 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.344+0000 7f5ed0ff9700 1 -- 172.21.15.150:0/1444449586 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ee4097ec0 con 0x55ec3b0076a0 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.344+0000 7f5ed0ff9700 1 -- 172.21.15.150:0/1444449586 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5ee4095a30 con 0x55ec3b0076a0 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.344+0000 7f5eff711b80 10 client.16308 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.344+0000 7f5eff711b80 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.344+0000 7f5eff711b80 20 client.16308 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:57.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.344+0000 7f5eff711b80 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55ec3b008ec0 0x55ec3b0757e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.344+0000 7f5eff711b80 4 client.16308 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.344+0000 7f5eff711b80 1 -- 172.21.15.150:0/1444449586 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ec3af1fec0 con 0x55ec3b008ec0 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.344+0000 7f5eed169700 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55ec3b008ec0 0x55ec3b0757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5ed0ff9700 10 client.16308 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5eed169700 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55ec3b008ec0 0x55ec3b0757e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7642a2bec88c89c3 server_cookie=9cf83800646ceb7 in_seq=0 out_seq=0 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5ed0ff9700 1 -- 172.21.15.150:0/1444449586 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ec3af1fec0 con 0x55ec3b008ec0 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5ed0ff9700 10 client.16308 handle_command_reply: tid=0 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5eff711b80 1 client.16308 shutdown 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5eff711b80 1 -- 172.21.15.150:0/1444449586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 msgr2=0x55ec3b0169b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5eff711b80 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 0x55ec3b0169b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4099ab0 tx=0x7f5ee40a1fa0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5eff711b80 1 -- 172.21.15.150:0/1444449586 shutdown_connections 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5eff711b80 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x55ec3b008ec0 0x55ec3b0757e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5eff711b80 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec3b008570 0x55ec3b0178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5eff711b80 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec3af69a60 0x55ec3b016470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5ed0ff9700 0 client.16308 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:57.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5eff711b80 1 --2- 172.21.15.150:0/1444449586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec3b0076a0 0x55ec3b0169b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5eff711b80 1 -- 172.21.15.150:0/1444449586 >> 172.21.15.150:0/1444449586 conn(0x55ec3affe450 msgr2=0x55ec3b007d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5eff711b80 1 -- 172.21.15.150:0/1444449586 shutdown_connections 2024-05-10T04:06:57.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5eff711b80 1 -- 172.21.15.150:0/1444449586 wait complete. 2024-05-10T04:06:57.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.345+0000 7f5eff711b80 20 client.16308 trim_cache size 0 max 16384 2024-05-10T04:06:57.354 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:06:57.354 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "no active scrubs running", 2024-05-10T04:06:57.354 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": {} 2024-05-10T04:06:57.354 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:06:57.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.346+0000 7f5eef3cd700 1 -- 172.21.15.150:0/979070206 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5ed40a1240 msgr2=0x7f5ed40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.346+0000 7f5eef3cd700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5ed40a1240 0x7f5ed40a3750 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8005950 tx=0x7f5ed800b410 comp rx=0 tx=0).stop 2024-05-10T04:06:57.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.346+0000 7f5eef3cd700 1 -- 172.21.15.150:0/979070206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ee8102000 msgr2=0x7f5ee8197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.346+0000 7f5eef3cd700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ee8102000 0x7f5ee8197640 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400d950 tx=0x7f5ee400dc60 comp rx=0 tx=0).stop 2024-05-10T04:06:57.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.346+0000 7f5eef3cd700 1 -- 172.21.15.150:0/979070206 shutdown_connections 2024-05-10T04:06:57.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.346+0000 7f5eef3cd700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5ed40a1240 0x7f5ed40a3750 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.346+0000 7f5eef3cd700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee8102950 0x7f5ee819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.346+0000 7f5eef3cd700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ee8101110 0x7f5ee8197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.346+0000 7f5eef3cd700 1 --2- 172.21.15.150:0/979070206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ee8102000 0x7f5ee8197640 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.346+0000 7f5eef3cd700 1 -- 172.21.15.150:0/979070206 >> 172.21.15.150:0/979070206 conn(0x7f5ee80fb8f0 msgr2=0x7f5ee8105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.346+0000 7f5eef3cd700 1 -- 172.21.15.150:0/979070206 shutdown_connections 2024-05-10T04:06:57.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.346+0000 7f5eef3cd700 1 -- 172.21.15.150:0/979070206 wait complete. 2024-05-10T04:06:57.360 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T04:06:57.360 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:06:57.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.491+0000 7ff455868700 1 -- 172.21.15.150:0/1049463451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450069090 msgr2=0x7ff450104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.491+0000 7ff455868700 1 --2- 172.21.15.150:0/1049463451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450069090 0x7ff450104720 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff440009990 tx=0x7ff440009ca0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.491+0000 7ff455868700 1 -- 172.21.15.150:0/1049463451 shutdown_connections 2024-05-10T04:06:57.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.491+0000 7ff455868700 1 --2- 172.21.15.150:0/1049463451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff450107880 0x7ff450109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.491+0000 7ff455868700 1 --2- 172.21.15.150:0/1049463451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450069090 0x7ff450104720 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.491+0000 7ff455868700 1 --2- 172.21.15.150:0/1049463451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff450104d10 0x7ff4501071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.491+0000 7ff455868700 1 -- 172.21.15.150:0/1049463451 >> 172.21.15.150:0/1049463451 conn(0x7ff4500fc0d0 msgr2=0x7ff4500fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.491+0000 7ff455868700 1 -- 172.21.15.150:0/1049463451 shutdown_connections 2024-05-10T04:06:57.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.491+0000 7ff455868700 1 -- 172.21.15.150:0/1049463451 wait complete. 2024-05-10T04:06:57.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.492+0000 7ff455868700 1 Processor -- start 2024-05-10T04:06:57.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.492+0000 7ff455868700 1 -- start start 2024-05-10T04:06:57.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.492+0000 7ff455868700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450069090 0x7ff450198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff455868700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff450104d10 0x7ff450198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff455868700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff450107880 0x7ff45019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff455868700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45019fa90 con 0x7ff450069090 2024-05-10T04:06:57.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff455868700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45019fc00 con 0x7ff450104d10 2024-05-10T04:06:57.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff455868700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45019fdc0 con 0x7ff450107880 2024-05-10T04:06:57.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff44effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450069090 0x7ff450198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff44effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450069090 0x7ff450198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52460/0 (socket says 172.21.15.150:52460) 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff44effd700 1 -- 172.21.15.150:0/1054703384 learned_addr learned my addr 172.21.15.150:0/1054703384 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff44f7fe700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff450107880 0x7ff45019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff44e7fc700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff450104d10 0x7ff450198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff44effd700 1 -- 172.21.15.150:0/1054703384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff450107880 msgr2=0x7ff45019f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff44effd700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff450107880 0x7ff45019f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff44effd700 1 -- 172.21.15.150:0/1054703384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff450104d10 msgr2=0x7ff450198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff44effd700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff450104d10 0x7ff450198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff44effd700 1 -- 172.21.15.150:0/1054703384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4400095f0 con 0x7ff450069090 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff44e7fc700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff450104d10 0x7ff450198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff44effd700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450069090 0x7ff450198000 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff440005b40 tx=0x7ff44000f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff454866700 1 -- 172.21.15.150:0/1054703384 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44001d070 con 0x7ff450069090 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff454866700 1 -- 172.21.15.150:0/1054703384 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff44000fcb0 con 0x7ff450069090 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff455868700 1 -- 172.21.15.150:0/1054703384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4501a0040 con 0x7ff450069090 2024-05-10T04:06:57.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff454866700 1 -- 172.21.15.150:0/1054703384 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440004ac0 con 0x7ff450069090 2024-05-10T04:06:57.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.493+0000 7ff455868700 1 -- 172.21.15.150:0/1054703384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4501a04f0 con 0x7ff450069090 2024-05-10T04:06:57.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.495+0000 7ff455868700 1 -- 172.21.15.150:0/1054703384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4500fdc80 con 0x7ff450069090 2024-05-10T04:06:57.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.496+0000 7ff454866700 1 -- 172.21.15.150:0/1054703384 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff440004c60 con 0x7ff450069090 2024-05-10T04:06:57.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.496+0000 7ff454866700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff43c0a1120 0x7ff43c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.496+0000 7ff44e7fc700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff43c0a1120 0x7ff43c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.497+0000 7ff454866700 1 -- 172.21.15.150:0/1054703384 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff4400cdae0 con 0x7ff450069090 2024-05-10T04:06:57.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.497+0000 7ff44e7fc700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff43c0a1120 0x7ff43c0a3630 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff438005fd0 tx=0x7ff438005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.501+0000 7ff454866700 1 -- 172.21.15.150:0/1054703384 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff440096ff0 con 0x7ff450069090 2024-05-10T04:06:57.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.753+0000 7ff455868700 1 -- 172.21.15.150:0/1054703384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4500694d0 con 0x7ff450069090 2024-05-10T04:06:57.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.754+0000 7ff454866700 1 -- 172.21.15.150:0/1054703384 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10886 (secure 0 0 0) 0x7ff44009a610 con 0x7ff450069090 2024-05-10T04:06:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:06:57.756 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:04:04.157819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:06:57.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.756+0000 7ff455868700 1 -- 172.21.15.150:0/1054703384 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff43c0a1120 msgr2=0x7ff43c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.756+0000 7ff455868700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff43c0a1120 0x7ff43c0a3630 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff438005fd0 tx=0x7ff438005f00 comp rx=0 tx=0).stop 2024-05-10T04:06:57.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.756+0000 7ff455868700 1 -- 172.21.15.150:0/1054703384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450069090 msgr2=0x7ff450198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.756+0000 7ff455868700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450069090 0x7ff450198000 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff440005b40 tx=0x7ff44000f7c0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.756+0000 7ff455868700 1 -- 172.21.15.150:0/1054703384 shutdown_connections 2024-05-10T04:06:57.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.756+0000 7ff455868700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff43c0a1120 0x7ff43c0a3630 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.756+0000 7ff455868700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff450107880 0x7ff45019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.756+0000 7ff455868700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450069090 0x7ff450198000 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.756+0000 7ff455868700 1 --2- 172.21.15.150:0/1054703384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff450104d10 0x7ff450198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.756+0000 7ff455868700 1 -- 172.21.15.150:0/1054703384 >> 172.21.15.150:0/1054703384 conn(0x7ff4500fc0d0 msgr2=0x7ff450104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.756+0000 7ff455868700 1 -- 172.21.15.150:0/1054703384 shutdown_connections 2024-05-10T04:06:57.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.756+0000 7ff455868700 1 -- 172.21.15.150:0/1054703384 wait complete. 2024-05-10T04:06:57.761 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 25 2024-05-10T04:06:57.771 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T04:06:57.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.905+0000 7f5b58b7a700 1 -- 172.21.15.150:0/575185874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b54102590 msgr2=0x7f5b54108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.905+0000 7f5b58b7a700 1 --2- 172.21.15.150:0/575185874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b54102590 0x7f5b54108e40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00b0a0 tx=0x7f5b4c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.905+0000 7f5b58b7a700 1 -- 172.21.15.150:0/575185874 shutdown_connections 2024-05-10T04:06:57.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.905+0000 7f5b58b7a700 1 --2- 172.21.15.150:0/575185874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b54102590 0x7f5b54108e40 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.905+0000 7f5b58b7a700 1 --2- 172.21.15.150:0/575185874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b54100ad0 0x7f5b54100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.905+0000 7f5b58b7a700 1 --2- 172.21.15.150:0/575185874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b54101bf0 0x7f5b54102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.905+0000 7f5b58b7a700 1 -- 172.21.15.150:0/575185874 >> 172.21.15.150:0/575185874 conn(0x7f5b540fc0e0 msgr2=0x7f5b540fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.905+0000 7f5b58b7a700 1 -- 172.21.15.150:0/575185874 shutdown_connections 2024-05-10T04:06:57.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.905+0000 7f5b58b7a700 1 -- 172.21.15.150:0/575185874 wait complete. 2024-05-10T04:06:57.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.906+0000 7f5b58b7a700 1 Processor -- start 2024-05-10T04:06:57.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.906+0000 7f5b58b7a700 1 -- start start 2024-05-10T04:06:57.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.906+0000 7f5b58b7a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b54100ad0 0x7f5b54196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.906+0000 7f5b58b7a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b54101bf0 0x7f5b541974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.906+0000 7f5b58b7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b54102590 0x7f5b5419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.906+0000 7f5b58b7a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419ea30 con 0x7f5b54101bf0 2024-05-10T04:06:57.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.906+0000 7f5b58b7a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419eb70 con 0x7f5b54100ad0 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.906+0000 7f5b58b7a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419ed00 con 0x7f5b54102590 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.906+0000 7f5b51d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b54101bf0 0x7f5b541974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.906+0000 7f5b51d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b54101bf0 0x7f5b541974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52482/0 (socket says 172.21.15.150:52482) 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.906+0000 7f5b51d9b700 1 -- 172.21.15.150:0/2436515931 learned_addr learned my addr 172.21.15.150:0/2436515931 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.906+0000 7f5b5259c700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b54100ad0 0x7f5b54196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.906+0000 7f5b52d9d700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b54102590 0x7f5b5419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.907+0000 7f5b51d9b700 1 -- 172.21.15.150:0/2436515931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b54102590 msgr2=0x7f5b5419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.907+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b54102590 0x7f5b5419e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.907+0000 7f5b51d9b700 1 -- 172.21.15.150:0/2436515931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b54100ad0 msgr2=0x7f5b54196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.907+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b54100ad0 0x7f5b54196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.907+0000 7f5b51d9b700 1 -- 172.21.15.150:0/2436515931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c009d00 con 0x7f5b54101bf0 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.907+0000 7f5b52d9d700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b54102590 0x7f5b5419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.907+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b54101bf0 0x7f5b541974d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400e9d0 tx=0x7f5b4400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.907+0000 7f5b4b7fe700 1 -- 172.21.15.150:0/2436515931 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44009790 con 0x7f5b54101bf0 2024-05-10T04:06:57.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.907+0000 7f5b4b7fe700 1 -- 172.21.15.150:0/2436515931 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b4400cbd0 con 0x7f5b54101bf0 2024-05-10T04:06:57.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.907+0000 7f5b4b7fe700 1 -- 172.21.15.150:0/2436515931 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44018680 con 0x7f5b54101bf0 2024-05-10T04:06:57.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.907+0000 7f5b58b7a700 1 -- 172.21.15.150:0/2436515931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5419efe0 con 0x7f5b54101bf0 2024-05-10T04:06:57.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.907+0000 7f5b58b7a700 1 -- 172.21.15.150:0/2436515931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5419f3b0 con 0x7f5b54101bf0 2024-05-10T04:06:57.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.909+0000 7f5b4b7fe700 1 -- 172.21.15.150:0/2436515931 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5b440187e0 con 0x7f5b54101bf0 2024-05-10T04:06:57.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.910+0000 7f5b4b7fe700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5b400a11c0 0x7f5b400a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.910+0000 7f5b4b7fe700 1 -- 172.21.15.150:0/2436515931 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5b44014070 con 0x7f5b54101bf0 2024-05-10T04:06:57.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.910+0000 7f5b5259c700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5b400a11c0 0x7f5b400a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.911+0000 7f5b5259c700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5b400a11c0 0x7f5b400a36d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5b54101af0 tx=0x7f5b3c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.911+0000 7f5b68ebeb80 1 Processor -- start 2024-05-10T04:06:57.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.911+0000 7f5b68ebeb80 1 -- start start 2024-05-10T04:06:57.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.911+0000 7f5b68ebeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc46a0 0x5643b5bc4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.911+0000 7f5b68ebeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5570 0x5643b5bc5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.911+0000 7f5b68ebeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5ec0 0x5643b5bcc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.911+0000 7f5b68ebeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b5aeb150 con 0x5643b5bc5ec0 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.911+0000 7f5b68ebeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b5ae8b10 con 0x5643b5bc46a0 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.911+0000 7f5b68ebeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b5ae4400 con 0x5643b5bc5570 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.911+0000 7f5b52d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5ec0 0x5643b5bcc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.911+0000 7f5b52d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5ec0 0x5643b5bcc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52496/0 (socket says 172.21.15.150:52496) 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.911+0000 7f5b52d9d700 1 -- 172.21.15.150:0/3176490954 learned_addr learned my addr 172.21.15.150:0/3176490954 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.912+0000 7f5b5259c700 1 --2- 172.21.15.150:0/3176490954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc46a0 0x5643b5bc4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.912+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/3176490954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5570 0x5643b5bc5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.912+0000 7f5b52d9d700 1 -- 172.21.15.150:0/3176490954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5570 msgr2=0x5643b5bc5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.912+0000 7f5b52d9d700 1 --2- 172.21.15.150:0/3176490954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5570 0x5643b5bc5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.912+0000 7f5b52d9d700 1 -- 172.21.15.150:0/3176490954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc46a0 msgr2=0x5643b5bc4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.912+0000 7f5b52d9d700 1 --2- 172.21.15.150:0/3176490954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc46a0 0x5643b5bc4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.912+0000 7f5b52d9d700 1 -- 172.21.15.150:0/3176490954 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643b59653b0 con 0x5643b5bc5ec0 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.912+0000 7f5b5259c700 1 --2- 172.21.15.150:0/3176490954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc46a0 0x5643b5bc4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.912+0000 7f5b52d9d700 1 --2- 172.21.15.150:0/3176490954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5ec0 0x5643b5bcc7c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00b070 tx=0x7f5b4c0097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffac76f128c1730 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.912+0000 7f5b48ff9700 1 -- 172.21.15.150:0/3176490954 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c00e050 con 0x5643b5bc5ec0 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.912+0000 7f5b48ff9700 1 -- 172.21.15.150:0/3176490954 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b4c009460 con 0x5643b5bc5ec0 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.912+0000 7f5b48ff9700 1 -- 172.21.15.150:0/3176490954 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c01b450 con 0x5643b5bc5ec0 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.913+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/3176490954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5ec0 msgr2=0x5643b5bcc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.913+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/3176490954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5ec0 0x5643b5bcc7c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00b070 tx=0x7f5b4c0097c0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.913+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/3176490954 shutdown_connections 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.913+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/3176490954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5ec0 0x5643b5bcc7c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.913+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/3176490954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc46a0 0x5643b5bc4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.913+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/3176490954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5570 0x5643b5bc5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.913+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/3176490954 >> 172.21.15.150:0/3176490954 conn(0x5643b5b26a60 msgr2=0x5643b5bbefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.913+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/3176490954 shutdown_connections 2024-05-10T04:06:57.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.913+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/3176490954 wait complete. 2024-05-10T04:06:57.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.913+0000 7f5b68ebeb80 1 -- start start 2024-05-10T04:06:57.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.913+0000 7f5b68ebeb80 1 Processor -- start 2024-05-10T04:06:57.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b68ebeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 0x5643b5bd0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b68ebeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5570 0x5643b5bd0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b68ebeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5ec0 0x5643b5bd7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b68ebeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b5a80560 con 0x5643b5bc46a0 2024-05-10T04:06:57.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b68ebeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b5a80200 con 0x5643b5bc5ec0 2024-05-10T04:06:57.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b68ebeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b59676a0 con 0x5643b5bc5570 2024-05-10T04:06:57.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b68ebeb80 10 client.0 _ll_register_callbacks cb 0x5643b5a89b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:57.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b5259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 0x5643b5bd0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b5259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 0x5643b5bd0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52498/0 (socket says 172.21.15.150:52498) 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b5259c700 1 -- 172.21.15.150:0/3558580807 learned_addr learned my addr 172.21.15.150:0/3558580807 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5570 0x5643b5bd0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b52d9d700 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5ec0 0x5643b5bd7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b5259c700 1 -- 172.21.15.150:0/3558580807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5570 msgr2=0x5643b5bd0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b5259c700 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5570 0x5643b5bd0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b5259c700 1 -- 172.21.15.150:0/3558580807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5ec0 msgr2=0x5643b5bd7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b5259c700 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5ec0 0x5643b5bd7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b5259c700 1 -- 172.21.15.150:0/3558580807 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b541098d0 con 0x5643b5bc46a0 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5570 0x5643b5bd0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b5259c700 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 0x5643b5bd0850 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c01db40 tx=0x7f5b3c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f036d00d7ca14e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b39ffb700 1 -- 172.21.15.150:0/3558580807 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c0109c0 con 0x5643b5bc46a0 2024-05-10T04:06:57.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/3558580807 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643b5a3d270 con 0x5643b5bc46a0 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.914+0000 7f5b39ffb700 1 -- 172.21.15.150:0/3558580807 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5b3c010b20 con 0x5643b5bc46a0 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.915+0000 7f5b39ffb700 1 -- 172.21.15.150:0/3558580807 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b3c010ce0 con 0x5643b5bc46a0 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.915+0000 7f5b68ebeb80 10 client.16326 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.915+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/3558580807 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5643b5911b60 con 0x5643b5bc46a0 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.915+0000 7f5b39ffb700 1 -- 172.21.15.150:0/3558580807 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5643b5a3d270 con 0x5643b5bc46a0 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.915+0000 7f5b39ffb700 1 -- 172.21.15.150:0/3558580807 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5b3c01b420 con 0x5643b5bc46a0 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.915+0000 7f5b68ebeb80 10 client.16326 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.915+0000 7f5b68ebeb80 10 client.16326 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.915+0000 7f5b68ebeb80 20 client.16326 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.915+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5643b5bdc5d0 0x5643b5bdea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.915+0000 7f5b68ebeb80 4 client.16326 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.915+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/3558580807 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643b5b93780 con 0x5643b5bdc5d0 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.915+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5643b5bdc5d0 0x5643b5bdea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.916+0000 7f5b39ffb700 10 client.16326 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:06:57.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.916+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5643b5bdc5d0 0x5643b5bdea20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74abf0bab5e506fa server_cookie=4e4bde3774a94ef in_seq=0 out_seq=0 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.916+0000 7f5b39ffb700 1 -- 172.21.15.150:0/3558580807 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5643b5b93780 con 0x5643b5bdc5d0 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.916+0000 7f5b39ffb700 10 client.16326 handle_command_reply: tid=0 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.917+0000 7f5b68ebeb80 1 client.16326 shutdown 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.917+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/3558580807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 msgr2=0x5643b5bd0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.917+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 0x5643b5bd0850 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c01db40 tx=0x7f5b3c0047c0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.917+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/3558580807 shutdown_connections 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.917+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5ec0 0x5643b5bd7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.917+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 0x5643b5bd0850 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.917+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5643b5bdc5d0 0x5643b5bdea20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.917+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/3558580807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5570 0x5643b5bd0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.917+0000 7f5b39ffb700 0 client.16326 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.917+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/3558580807 >> 172.21.15.150:0/3558580807 conn(0x5643b5b26a60 msgr2=0x5643b5bbefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.917+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/3558580807 shutdown_connections 2024-05-10T04:06:57.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.917+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/3558580807 wait complete. 2024-05-10T04:06:57.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.917+0000 7f5b68ebeb80 20 client.16326 trim_cache size 0 max 16384 2024-05-10T04:06:57.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.930+0000 7f5b68ebeb80 1 Processor -- start 2024-05-10T04:06:57.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.930+0000 7f5b68ebeb80 1 -- start start 2024-05-10T04:06:57.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.930+0000 7f5b68ebeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc46a0 0x5643b5bd82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.930+0000 7f5b68ebeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5570 0x5643b5bd7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.930+0000 7f5b68ebeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5ec0 0x5643b5bcdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.930+0000 7f5b68ebeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b5bd18f0 con 0x5643b5bc5570 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.930+0000 7f5b68ebeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b5bc0770 con 0x5643b5bc5ec0 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.930+0000 7f5b68ebeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b5bc5340 con 0x5643b5bc46a0 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.930+0000 7f5b51d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5570 0x5643b5bd7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.930+0000 7f5b51d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5570 0x5643b5bd7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52510/0 (socket says 172.21.15.150:52510) 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.930+0000 7f5b51d9b700 1 -- 172.21.15.150:0/350109609 learned_addr learned my addr 172.21.15.150:0/350109609 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.930+0000 7f5b52d9d700 1 --2- 172.21.15.150:0/350109609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5ec0 0x5643b5bcdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.930+0000 7f5b5259c700 1 --2- 172.21.15.150:0/350109609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc46a0 0x5643b5bd82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.931+0000 7f5b51d9b700 1 -- 172.21.15.150:0/350109609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc46a0 msgr2=0x5643b5bd82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.931+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/350109609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc46a0 0x5643b5bd82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.931+0000 7f5b51d9b700 1 -- 172.21.15.150:0/350109609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5ec0 msgr2=0x5643b5bcdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.931+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/350109609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5ec0 0x5643b5bcdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.931+0000 7f5b51d9b700 1 -- 172.21.15.150:0/350109609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643b5aaa560 con 0x5643b5bc5570 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.931+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/350109609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5570 0x5643b5bd7cb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5b44099ec0 tx=0x7f5b44010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49524f2dd0507d75 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.931+0000 7f5b497fa700 1 -- 172.21.15.150:0/350109609 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4409dd60 con 0x5643b5bc5570 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.931+0000 7f5b497fa700 1 -- 172.21.15.150:0/350109609 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b440a23b0 con 0x5643b5bc5570 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.931+0000 7f5b497fa700 1 -- 172.21.15.150:0/350109609 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44099420 con 0x5643b5bc5570 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/350109609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5570 msgr2=0x5643b5bd7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/350109609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5570 0x5643b5bd7cb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5b44099ec0 tx=0x7f5b44010b10 comp rx=0 tx=0).stop 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/350109609 shutdown_connections 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/350109609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5ec0 0x5643b5bcdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/350109609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc46a0 0x5643b5bd82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/350109609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc5570 0x5643b5bd7cb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/350109609 >> 172.21.15.150:0/350109609 conn(0x5643b5b26a60 msgr2=0x5643b5bbb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/350109609 shutdown_connections 2024-05-10T04:06:57.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/350109609 wait complete. 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 -- start start 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 Processor -- start 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 0x5643b5bd50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5570 0x5643b5bd5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5ec0 0x5643b5bd6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b5bd6a80 con 0x5643b5bc46a0 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b5bd6bf0 con 0x5643b5bc5570 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b5bd6db0 con 0x5643b5bc5ec0 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b68ebeb80 10 client.0 _ll_register_callbacks cb 0x5643b5a89b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b5259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 0x5643b5bd50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b5259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 0x5643b5bd50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52524/0 (socket says 172.21.15.150:52524) 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b5259c700 1 -- 172.21.15.150:0/2911921005 learned_addr learned my addr 172.21.15.150:0/2911921005 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b52d9d700 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5ec0 0x5643b5bd6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.932+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5570 0x5643b5bd5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b5259c700 1 -- 172.21.15.150:0/2911921005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5ec0 msgr2=0x5643b5bd6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b5259c700 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5ec0 0x5643b5bd6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b5259c700 1 -- 172.21.15.150:0/2911921005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5570 msgr2=0x5643b5bd5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b5259c700 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5570 0x5643b5bd5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b5259c700 1 -- 172.21.15.150:0/2911921005 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5419f3b0 con 0x5643b5bc46a0 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5570 0x5643b5bd5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b52d9d700 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5ec0 0x5643b5bd6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b5259c700 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 0x5643b5bd50d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c000c00 tx=0x7f5b3c01b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164126584574b790 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b39ffb700 1 -- 172.21.15.150:0/2911921005 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c0109c0 con 0x5643b5bc46a0 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b39ffb700 1 -- 172.21.15.150:0/2911921005 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5b3c010b20 con 0x5643b5bc46a0 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/2911921005 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b4c009460 con 0x5643b5bc46a0 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b39ffb700 1 -- 172.21.15.150:0/2911921005 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b3c010ce0 con 0x5643b5bc46a0 2024-05-10T04:06:57.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b68ebeb80 10 client.16332 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/2911921005 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5643b5969cd0 con 0x5643b5bc46a0 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b39ffb700 1 -- 172.21.15.150:0/2911921005 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b4c009460 con 0x5643b5bc46a0 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b39ffb700 1 -- 172.21.15.150:0/2911921005 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5b3c0273f0 con 0x5643b5bc46a0 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.933+0000 7f5b68ebeb80 10 client.16332 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 10 client.16332 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 20 client.16332 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5643b5c33050 0x5643b5c35470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 4 client.16332 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/2911921005 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5643b5adcec0 con 0x5643b5c33050 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5643b5c33050 0x5643b5c35470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b39ffb700 10 client.16332 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b51d9b700 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5643b5c33050 0x5643b5c35470 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96547b52af8984bd server_cookie=d728ba4b891de983 in_seq=0 out_seq=0 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b39ffb700 1 -- 172.21.15.150:0/2911921005 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5643b5adcec0 con 0x5643b5c33050 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b39ffb700 10 client.16332 handle_command_reply: tid=0 2024-05-10T04:06:57.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 1 client.16332 shutdown 2024-05-10T04:06:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/2911921005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 msgr2=0x5643b5bd50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 0x5643b5bd50d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c000c00 tx=0x7f5b3c01b970 comp rx=0 tx=0).stop 2024-05-10T04:06:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/2911921005 shutdown_connections 2024-05-10T04:06:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5643b5c33050 0x5643b5c35470 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643b5bc5ec0 0x5643b5bd6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b5bc46a0 0x5643b5bd50d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 1 --2- 172.21.15.150:0/2911921005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5643b5bc5570 0x5643b5bd5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b39ffb700 0 client.16332 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:06:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.934+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/2911921005 >> 172.21.15.150:0/2911921005 conn(0x5643b5b26a60 msgr2=0x5643b5bbb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/2911921005 shutdown_connections 2024-05-10T04:06:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b68ebeb80 1 -- 172.21.15.150:0/2911921005 wait complete. 2024-05-10T04:06:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b68ebeb80 20 client.16332 trim_cache size 0 max 16384 2024-05-10T04:06:57.944 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:06:57.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b58b7a700 1 -- 172.21.15.150:0/2436515931 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5b400a11c0 msgr2=0x7f5b400a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b58b7a700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5b400a11c0 0x7f5b400a36d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5b54101af0 tx=0x7f5b3c00a3b0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b58b7a700 1 -- 172.21.15.150:0/2436515931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b54101bf0 msgr2=0x7f5b541974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:57.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b58b7a700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b54101bf0 0x7f5b541974d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400e9d0 tx=0x7f5b4400ece0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b58b7a700 1 -- 172.21.15.150:0/2436515931 shutdown_connections 2024-05-10T04:06:57.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b58b7a700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5b400a11c0 0x7f5b400a36d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b58b7a700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b54102590 0x7f5b5419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b58b7a700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b54100ad0 0x7f5b54196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b58b7a700 1 --2- 172.21.15.150:0/2436515931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b54101bf0 0x7f5b541974d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:57.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b58b7a700 1 -- 172.21.15.150:0/2436515931 >> 172.21.15.150:0/2436515931 conn(0x7f5b540fc0e0 msgr2=0x7f5b54104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:57.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b58b7a700 1 -- 172.21.15.150:0/2436515931 shutdown_connections 2024-05-10T04:06:57.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:57.935+0000 7f5b58b7a700 1 -- 172.21.15.150:0/2436515931 wait complete. 2024-05-10T04:06:57.950 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T04:06:58.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.082+0000 7f7b305c7700 1 -- 172.21.15.150:0/132565530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b28101be0 msgr2=0x7f7b28101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.082+0000 7f7b305c7700 1 --2- 172.21.15.150:0/132565530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b28101be0 0x7f7b28101ff0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009990 tx=0x7f7b18009ca0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.082+0000 7f7b305c7700 1 -- 172.21.15.150:0/132565530 shutdown_connections 2024-05-10T04:06:58.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.082+0000 7f7b305c7700 1 --2- 172.21.15.150:0/132565530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b28102580 0x7f7b28108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.082+0000 7f7b305c7700 1 --2- 172.21.15.150:0/132565530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b28100ac0 0x7f7b28100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.082+0000 7f7b305c7700 1 --2- 172.21.15.150:0/132565530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b28101be0 0x7f7b28101ff0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.082+0000 7f7b305c7700 1 -- 172.21.15.150:0/132565530 >> 172.21.15.150:0/132565530 conn(0x7f7b280fc0d0 msgr2=0x7f7b280fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:58.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.082+0000 7f7b305c7700 1 -- 172.21.15.150:0/132565530 shutdown_connections 2024-05-10T04:06:58.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.082+0000 7f7b305c7700 1 -- 172.21.15.150:0/132565530 wait complete. 2024-05-10T04:06:58.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.083+0000 7f7b305c7700 1 Processor -- start 2024-05-10T04:06:58.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.083+0000 7f7b305c7700 1 -- start start 2024-05-10T04:06:58.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.083+0000 7f7b305c7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b28100ac0 0x7f7b28068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.083+0000 7f7b305c7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b28101be0 0x7f7b2819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.083+0000 7f7b305c7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b28102580 0x7f7b2819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.083+0000 7f7b305c7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2819fa20 con 0x7f7b28100ac0 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.083+0000 7f7b305c7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2819fb90 con 0x7f7b28101be0 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.083+0000 7f7b305c7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2819fd50 con 0x7f7b28102580 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.083+0000 7f7b2e363700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b28100ac0 0x7f7b28068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.083+0000 7f7b2e363700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b28100ac0 0x7f7b28068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52534/0 (socket says 172.21.15.150:52534) 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.083+0000 7f7b2e363700 1 -- 172.21.15.150:0/1065999503 learned_addr learned my addr 172.21.15.150:0/1065999503 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.084+0000 7f7b2eb64700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b28102580 0x7f7b2819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.084+0000 7f7b2db62700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b28101be0 0x7f7b2819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.084+0000 7f7b2e363700 1 -- 172.21.15.150:0/1065999503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b28102580 msgr2=0x7f7b2819f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.084+0000 7f7b2e363700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b28102580 0x7f7b2819f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.084+0000 7f7b2e363700 1 -- 172.21.15.150:0/1065999503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b28101be0 msgr2=0x7f7b2819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.084+0000 7f7b2e363700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b28101be0 0x7f7b2819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.084+0000 7f7b2e363700 1 -- 172.21.15.150:0/1065999503 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b180095f0 con 0x7f7b28100ac0 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.084+0000 7f7b2e363700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b28100ac0 0x7f7b28068e50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400e940 tx=0x7f7b2400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.084+0000 7f7b1f7fe700 1 -- 172.21.15.150:0/1065999503 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2400cbd0 con 0x7f7b28100ac0 2024-05-10T04:06:58.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.084+0000 7f7b1f7fe700 1 -- 172.21.15.150:0/1065999503 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b240044e0 con 0x7f7b28100ac0 2024-05-10T04:06:58.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.084+0000 7f7b1f7fe700 1 -- 172.21.15.150:0/1065999503 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24003dc0 con 0x7f7b28100ac0 2024-05-10T04:06:58.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.084+0000 7f7b305c7700 1 -- 172.21.15.150:0/1065999503 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2819ffd0 con 0x7f7b28100ac0 2024-05-10T04:06:58.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.084+0000 7f7b305c7700 1 -- 172.21.15.150:0/1065999503 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b281a0400 con 0x7f7b28100ac0 2024-05-10T04:06:58.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.086+0000 7f7b1f7fe700 1 -- 172.21.15.150:0/1065999503 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7b24003f20 con 0x7f7b28100ac0 2024-05-10T04:06:58.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.087+0000 7f7b1f7fe700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b140a1240 0x7f7b140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.087+0000 7f7b1f7fe700 1 -- 172.21.15.150:0/1065999503 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7b24014070 con 0x7f7b28100ac0 2024-05-10T04:06:58.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.087+0000 7f7b2db62700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b140a1240 0x7f7b140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.087+0000 7f7b2db62700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b140a1240 0x7f7b140a3750 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7b18006010 tx=0x7f7b1801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:58.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b4090bb80 1 Processor -- start 2024-05-10T04:06:58.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b4090bb80 1 -- start start 2024-05-10T04:06:58.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b4090bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 0x5620756fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b4090bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fc570 0x5620756fc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b4090bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fcec0 0x5620757037c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b4090bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562075622150 con 0x5620756fb6a0 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b4090bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207561fb10 con 0x5620756fcec0 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b4090bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207561b400 con 0x5620756fc570 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b2e363700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 0x5620756fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b2e363700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 0x5620756fbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52548/0 (socket says 172.21.15.150:52548) 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b2e363700 1 -- 172.21.15.150:0/2626086442 learned_addr learned my addr 172.21.15.150:0/2626086442 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b2db62700 1 --2- 172.21.15.150:0/2626086442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fc570 0x5620756fc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b2eb64700 1 --2- 172.21.15.150:0/2626086442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fcec0 0x5620757037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b2e363700 1 -- 172.21.15.150:0/2626086442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fc570 msgr2=0x5620756fc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b2e363700 1 --2- 172.21.15.150:0/2626086442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fc570 0x5620756fc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b2e363700 1 -- 172.21.15.150:0/2626086442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fcec0 msgr2=0x5620757037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b2e363700 1 --2- 172.21.15.150:0/2626086442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fcec0 0x5620757037c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b2e363700 1 -- 172.21.15.150:0/2626086442 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56207549c3b0 con 0x5620756fb6a0 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.088+0000 7f7b2db62700 1 --2- 172.21.15.150:0/2626086442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fc570 0x5620756fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:06:58.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.089+0000 7f7b2e363700 1 --2- 172.21.15.150:0/2626086442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 0x5620756fbaf0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7b240a1f70 tx=0x7f7b24003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212e59954f3dc967 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.089+0000 7f7b1cff9700 1 -- 172.21.15.150:0/2626086442 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2400cdc0 con 0x5620756fb6a0 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.089+0000 7f7b1cff9700 1 -- 172.21.15.150:0/2626086442 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b2409a720 con 0x5620756fb6a0 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.089+0000 7f7b1cff9700 1 -- 172.21.15.150:0/2626086442 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24096730 con 0x5620756fb6a0 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.090+0000 7f7b4090bb80 1 -- 172.21.15.150:0/2626086442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 msgr2=0x5620756fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.090+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/2626086442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 0x5620756fbaf0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7b240a1f70 tx=0x7f7b24003680 comp rx=0 tx=0).stop 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.090+0000 7f7b4090bb80 1 -- 172.21.15.150:0/2626086442 shutdown_connections 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.090+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/2626086442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fcec0 0x5620757037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.090+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/2626086442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 0x5620756fbaf0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.090+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/2626086442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fc570 0x5620756fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.090+0000 7f7b4090bb80 1 -- 172.21.15.150:0/2626086442 >> 172.21.15.150:0/2626086442 conn(0x56207565da60 msgr2=0x5620756f5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.090+0000 7f7b4090bb80 1 -- 172.21.15.150:0/2626086442 shutdown_connections 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.090+0000 7f7b4090bb80 1 -- 172.21.15.150:0/2626086442 wait complete. 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.090+0000 7f7b4090bb80 1 -- start start 2024-05-10T04:06:58.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.090+0000 7f7b4090bb80 1 Processor -- start 2024-05-10T04:06:58.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b4090bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 0x562075705750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b4090bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fc570 0x562075705c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b4090bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fcec0 0x56207570cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b4090bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620755b7560 con 0x5620756fb6a0 2024-05-10T04:06:58.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b4090bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620755b7200 con 0x5620756fc570 2024-05-10T04:06:58.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b4090bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207549e6a0 con 0x5620756fcec0 2024-05-10T04:06:58.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b4090bb80 10 client.0 _ll_register_callbacks cb 0x5620755c0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:58.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b2e363700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 0x562075705750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b2e363700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 0x562075705750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52552/0 (socket says 172.21.15.150:52552) 2024-05-10T04:06:58.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b2e363700 1 -- 172.21.15.150:0/1734651168 learned_addr learned my addr 172.21.15.150:0/1734651168 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:58.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b2db62700 1 --2- 172.21.15.150:0/1734651168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fc570 0x562075705c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b2eb64700 1 --2- 172.21.15.150:0/1734651168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fcec0 0x56207570cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b2e363700 1 -- 172.21.15.150:0/1734651168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fcec0 msgr2=0x56207570cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b2e363700 1 --2- 172.21.15.150:0/1734651168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fcec0 0x56207570cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b2e363700 1 -- 172.21.15.150:0/1734651168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fc570 msgr2=0x562075705c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b2e363700 1 --2- 172.21.15.150:0/1734651168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fc570 0x562075705c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b2e363700 1 -- 172.21.15.150:0/1734651168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b281a0400 con 0x5620756fb6a0 2024-05-10T04:06:58.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b2e363700 1 --2- 172.21.15.150:0/1734651168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 0x562075705750 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7b240037d0 tx=0x7f7b2400efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d47a3aabf764087 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:58.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:58.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b11ffb700 1 -- 172.21.15.150:0/1734651168 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b240b83f0 con 0x5620756fb6a0 2024-05-10T04:06:58.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b11ffb700 1 -- 172.21.15.150:0/1734651168 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7b2409de50 con 0x5620756fb6a0 2024-05-10T04:06:58.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.091+0000 7f7b4090bb80 1 -- 172.21.15.150:0/1734651168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562075574270 con 0x5620756fb6a0 2024-05-10T04:06:58.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.092+0000 7f7b11ffb700 1 -- 172.21.15.150:0/1734651168 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b24083400 con 0x5620756fb6a0 2024-05-10T04:06:58.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.092+0000 7f7b4090bb80 10 client.16344 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:58.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.092+0000 7f7b4090bb80 1 -- 172.21.15.150:0/1734651168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562075448b60 con 0x5620756fb6a0 2024-05-10T04:06:58.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.092+0000 7f7b11ffb700 1 -- 172.21.15.150:0/1734651168 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b240a6080 con 0x5620756fb6a0 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.092+0000 7f7b11ffb700 1 -- 172.21.15.150:0/1734651168 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7b24099510 con 0x5620756fb6a0 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.092+0000 7f7b4090bb80 10 client.16344 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.092+0000 7f7b4090bb80 10 client.16344 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.092+0000 7f7b4090bb80 20 client.16344 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.092+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/1734651168 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x562075710fa0 0x5620757133c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.092+0000 7f7b4090bb80 4 client.16344 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.092+0000 7f7b4090bb80 1 -- 172.21.15.150:0/1734651168 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620756ca780 con 0x562075710fa0 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.092+0000 7f7b2db62700 1 --2- 172.21.15.150:0/1734651168 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x562075710fa0 0x5620757133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.093+0000 7f7b2db62700 1 --2- 172.21.15.150:0/1734651168 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x562075710fa0 0x5620757133c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=113bc36f10489e6a server_cookie=158cd3ca4b162fcb in_seq=0 out_seq=0 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.093+0000 7f7b11ffb700 10 client.16344 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b11ffb700 1 -- 172.21.15.150:0/1734651168 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5620756ca780 con 0x562075710fa0 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b11ffb700 10 client.16344 handle_command_reply: tid=0 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b4090bb80 1 client.16344 shutdown 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b4090bb80 1 -- 172.21.15.150:0/1734651168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 msgr2=0x562075705750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/1734651168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 0x562075705750 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7b240037d0 tx=0x7f7b2400efb0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b4090bb80 1 -- 172.21.15.150:0/1734651168 shutdown_connections 2024-05-10T04:06:58.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/1734651168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fcec0 0x56207570cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/1734651168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fb6a0 0x562075705750 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/1734651168 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x562075710fa0 0x5620757133c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/1734651168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fc570 0x562075705c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b11ffb700 0 client.16344 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:06:58.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b4090bb80 1 -- 172.21.15.150:0/1734651168 >> 172.21.15.150:0/1734651168 conn(0x56207565da60 msgr2=0x5620756f5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:58.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b4090bb80 1 -- 172.21.15.150:0/1734651168 shutdown_connections 2024-05-10T04:06:58.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b4090bb80 1 -- 172.21.15.150:0/1734651168 wait complete. 2024-05-10T04:06:58.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.094+0000 7f7b4090bb80 20 client.16344 trim_cache size 0 max 16384 2024-05-10T04:06:58.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.107+0000 7f7b4090bb80 1 Processor -- start 2024-05-10T04:06:58.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.107+0000 7f7b4090bb80 1 -- start start 2024-05-10T04:06:58.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.107+0000 7f7b4090bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fb6a0 0x56207570d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.107+0000 7f7b4090bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fc570 0x56207570cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.107+0000 7f7b4090bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 0x562075703a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.107+0000 7f7b4090bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207570dca0 con 0x5620756fcec0 2024-05-10T04:06:58.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.107+0000 7f7b4090bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620756fd310 con 0x5620756fc570 2024-05-10T04:06:58.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.107+0000 7f7b4090bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562075718560 con 0x5620756fb6a0 2024-05-10T04:06:58.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b2eb64700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 0x562075703a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b2eb64700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 0x562075703a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52558/0 (socket says 172.21.15.150:52558) 2024-05-10T04:06:58.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b2eb64700 1 -- 172.21.15.150:0/341447277 learned_addr learned my addr 172.21.15.150:0/341447277 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:58.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b2e363700 1 --2- 172.21.15.150:0/341447277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fb6a0 0x56207570d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b2db62700 1 --2- 172.21.15.150:0/341447277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fc570 0x56207570cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b2eb64700 1 -- 172.21.15.150:0/341447277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fb6a0 msgr2=0x56207570d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b2eb64700 1 --2- 172.21.15.150:0/341447277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fb6a0 0x56207570d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b2eb64700 1 -- 172.21.15.150:0/341447277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fc570 msgr2=0x56207570cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b2eb64700 1 --2- 172.21.15.150:0/341447277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fc570 0x56207570cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b2eb64700 1 -- 172.21.15.150:0/341447277 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620755e1560 con 0x5620756fcec0 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b2eb64700 1 --2- 172.21.15.150:0/341447277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 0x562075703a40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009460 tx=0x7f7b2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3040c18f95c516c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b1d7fa700 1 -- 172.21.15.150:0/341447277 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000e030 con 0x5620756fcec0 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b1d7fa700 1 -- 172.21.15.150:0/341447277 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b20016b20 con 0x5620756fcec0 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.108+0000 7f7b1d7fa700 1 -- 172.21.15.150:0/341447277 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20015a40 con 0x5620756fcec0 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.109+0000 7f7b4090bb80 1 -- 172.21.15.150:0/341447277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 msgr2=0x562075703a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.109+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/341447277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 0x562075703a40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009460 tx=0x7f7b2000b040 comp rx=0 tx=0).stop 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.109+0000 7f7b4090bb80 1 -- 172.21.15.150:0/341447277 shutdown_connections 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.109+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/341447277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 0x562075703a40 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.109+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/341447277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fb6a0 0x56207570d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.109+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/341447277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fc570 0x56207570cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.109+0000 7f7b4090bb80 1 -- 172.21.15.150:0/341447277 >> 172.21.15.150:0/341447277 conn(0x56207565da60 msgr2=0x5620756ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.109+0000 7f7b4090bb80 1 -- 172.21.15.150:0/341447277 shutdown_connections 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.109+0000 7f7b4090bb80 1 -- 172.21.15.150:0/341447277 wait complete. 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.109+0000 7f7b4090bb80 1 -- start start 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.109+0000 7f7b4090bb80 1 Processor -- start 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b4090bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fb6a0 0x56207570a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b4090bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fc570 0x56207570abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b4090bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 0x56207570bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b4090bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207570c050 con 0x5620756fcec0 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b4090bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207570c1c0 con 0x5620756fb6a0 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b4090bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56207570c380 con 0x5620756fc570 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b4090bb80 10 client.0 _ll_register_callbacks cb 0x5620755c0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b2eb64700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 0x56207570bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b2eb64700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 0x56207570bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52570/0 (socket says 172.21.15.150:52570) 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b2eb64700 1 -- 172.21.15.150:0/4201402451 learned_addr learned my addr 172.21.15.150:0/4201402451 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:58.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b2db62700 1 --2- 172.21.15.150:0/4201402451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fc570 0x56207570abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b2e363700 1 --2- 172.21.15.150:0/4201402451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fb6a0 0x56207570a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b2eb64700 1 -- 172.21.15.150:0/4201402451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fc570 msgr2=0x56207570abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b2eb64700 1 --2- 172.21.15.150:0/4201402451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fc570 0x56207570abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b2eb64700 1 -- 172.21.15.150:0/4201402451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fb6a0 msgr2=0x56207570a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b2eb64700 1 --2- 172.21.15.150:0/4201402451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fb6a0 0x56207570a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b2eb64700 1 -- 172.21.15.150:0/4201402451 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b20009110 con 0x5620756fcec0 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b2eb64700 1 --2- 172.21.15.150:0/4201402451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 0x56207570bb10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7b20000f80 tx=0x7f7b20009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79127b87748fd01a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.110+0000 7f7b11ffb700 1 -- 172.21.15.150:0/4201402451 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000e040 con 0x5620756fcec0 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b11ffb700 1 -- 172.21.15.150:0/4201402451 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7b2000f070 con 0x5620756fcec0 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b4090bb80 1 -- 172.21.15.150:0/4201402451 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b2409a720 con 0x5620756fcec0 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b11ffb700 1 -- 172.21.15.150:0/4201402451 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b20003c00 con 0x5620756fcec0 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b4090bb80 10 client.16350 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:58.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b4090bb80 1 -- 172.21.15.150:0/4201402451 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5620754a0cd0 con 0x5620756fcec0 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b11ffb700 1 -- 172.21.15.150:0/4201402451 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b2409a720 con 0x5620756fcec0 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b11ffb700 1 -- 172.21.15.150:0/4201402451 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7b20014040 con 0x5620756fcec0 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b4090bb80 10 client.16350 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b4090bb80 10 client.16350 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b4090bb80 20 client.16350 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/4201402451 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x562075767f80 0x56207576a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b4090bb80 4 client.16350 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b4090bb80 1 -- 172.21.15.150:0/4201402451 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562075613ec0 con 0x562075767f80 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.111+0000 7f7b2e363700 1 --2- 172.21.15.150:0/4201402451 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x562075767f80 0x56207576a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b2e363700 1 --2- 172.21.15.150:0/4201402451 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x562075767f80 0x56207576a3a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5858fb10123323d8 server_cookie=f91176546e9d438b in_seq=0 out_seq=0 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b11ffb700 10 client.16350 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b11ffb700 1 -- 172.21.15.150:0/4201402451 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562075613ec0 con 0x562075767f80 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b11ffb700 10 client.16350 handle_command_reply: tid=0 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b4090bb80 1 client.16350 shutdown 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b4090bb80 1 -- 172.21.15.150:0/4201402451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 msgr2=0x56207570bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/4201402451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 0x56207570bb10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7b20000f80 tx=0x7f7b20009750 comp rx=0 tx=0).stop 2024-05-10T04:06:58.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b4090bb80 1 -- 172.21.15.150:0/4201402451 shutdown_connections 2024-05-10T04:06:58.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/4201402451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620756fcec0 0x56207570bb10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/4201402451 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x562075767f80 0x56207576a3a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/4201402451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620756fb6a0 0x56207570a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b4090bb80 1 --2- 172.21.15.150:0/4201402451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620756fc570 0x56207570abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b11ffb700 0 client.16350 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:06:58.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b4090bb80 1 -- 172.21.15.150:0/4201402451 >> 172.21.15.150:0/4201402451 conn(0x56207565da60 msgr2=0x5620756ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:58.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b4090bb80 1 -- 172.21.15.150:0/4201402451 shutdown_connections 2024-05-10T04:06:58.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.112+0000 7f7b4090bb80 1 -- 172.21.15.150:0/4201402451 wait complete. 2024-05-10T04:06:58.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.113+0000 7f7b4090bb80 20 client.16350 trim_cache size 0 max 16384 2024-05-10T04:06:58.120 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:06:58.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.113+0000 7f7b305c7700 1 -- 172.21.15.150:0/1065999503 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b140a1240 msgr2=0x7f7b140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.113+0000 7f7b305c7700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b140a1240 0x7f7b140a3750 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7b18006010 tx=0x7f7b1801a040 comp rx=0 tx=0).stop 2024-05-10T04:06:58.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.113+0000 7f7b305c7700 1 -- 172.21.15.150:0/1065999503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b28100ac0 msgr2=0x7f7b28068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.113+0000 7f7b305c7700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b28100ac0 0x7f7b28068e50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400e940 tx=0x7f7b2400ed00 comp rx=0 tx=0).stop 2024-05-10T04:06:58.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.113+0000 7f7b305c7700 1 -- 172.21.15.150:0/1065999503 shutdown_connections 2024-05-10T04:06:58.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.113+0000 7f7b305c7700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f7b140a1240 0x7f7b140a3750 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.113+0000 7f7b305c7700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b28102580 0x7f7b2819f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.113+0000 7f7b305c7700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b28100ac0 0x7f7b28068e50 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.113+0000 7f7b305c7700 1 --2- 172.21.15.150:0/1065999503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b28101be0 0x7f7b2819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.113+0000 7f7b305c7700 1 -- 172.21.15.150:0/1065999503 >> 172.21.15.150:0/1065999503 conn(0x7f7b280fc0d0 msgr2=0x7f7b28104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:58.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.113+0000 7f7b305c7700 1 -- 172.21.15.150:0/1065999503 shutdown_connections 2024-05-10T04:06:58.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.113+0000 7f7b305c7700 1 -- 172.21.15.150:0/1065999503 wait complete. 2024-05-10T04:06:58.128 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T04:06:58.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.261+0000 7f2d0ebcb700 1 -- 172.21.15.150:0/132780410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d081050c0 msgr2=0x7f2d081074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.261+0000 7f2d0ebcb700 1 --2- 172.21.15.150:0/132780410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d081050c0 0x7f2d081074e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009990 tx=0x7f2cf4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.261+0000 7f2d0ebcb700 1 -- 172.21.15.150:0/132780410 shutdown_connections 2024-05-10T04:06:58.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.261+0000 7f2d0ebcb700 1 --2- 172.21.15.150:0/132780410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d08107a20 0x7f2d08109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.261+0000 7f2d0ebcb700 1 --2- 172.21.15.150:0/132780410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d08102760 0x7f2d08104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.261+0000 7f2d0ebcb700 1 --2- 172.21.15.150:0/132780410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d081050c0 0x7f2d081074e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.261+0000 7f2d0ebcb700 1 -- 172.21.15.150:0/132780410 >> 172.21.15.150:0/132780410 conn(0x7f2d080fc180 msgr2=0x7f2d080fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:58.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.261+0000 7f2d0ebcb700 1 -- 172.21.15.150:0/132780410 shutdown_connections 2024-05-10T04:06:58.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.261+0000 7f2d0ebcb700 1 -- 172.21.15.150:0/132780410 wait complete. 2024-05-10T04:06:58.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.261+0000 7f2d0ebcb700 1 Processor -- start 2024-05-10T04:06:58.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.261+0000 7f2d0ebcb700 1 -- start start 2024-05-10T04:06:58.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0ebcb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d08102760 0x7f2d08197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0ebcb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d081050c0 0x7f2d08198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0ebcb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d08107a20 0x7f2d0819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0ebcb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0819f9b0 con 0x7f2d08107a20 2024-05-10T04:06:58.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0ebcb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0819fb20 con 0x7f2d08102760 2024-05-10T04:06:58.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0ebcb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0819fce0 con 0x7f2d081050c0 2024-05-10T04:06:58.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0d168700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d08107a20 0x7f2d0819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0c967700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d08102760 0x7f2d08197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0d168700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d08107a20 0x7f2d0819f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52594/0 (socket says 172.21.15.150:52594) 2024-05-10T04:06:58.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0c967700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d08102760 0x7f2d08197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:40140/0 (socket says 172.21.15.150:40140) 2024-05-10T04:06:58.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0d168700 1 -- 172.21.15.150:0/327750456 learned_addr learned my addr 172.21.15.150:0/327750456 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:58.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d07fff700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d081050c0 0x7f2d08198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0d168700 1 -- 172.21.15.150:0/327750456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d081050c0 msgr2=0x7f2d08198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0d168700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d081050c0 0x7f2d08198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0d168700 1 -- 172.21.15.150:0/327750456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d08102760 msgr2=0x7f2d08197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0d168700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d08102760 0x7f2d08197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0d168700 1 -- 172.21.15.150:0/327750456 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40095f0 con 0x7f2d08107a20 2024-05-10T04:06:58.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0c967700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d08102760 0x7f2d08197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:06:58.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d0d168700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d08107a20 0x7f2d0819f3a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000ef90 tx=0x7f2d0000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:58.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.262+0000 7f2d05ffb700 1 -- 172.21.15.150:0/327750456 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00018070 con 0x7f2d08107a20 2024-05-10T04:06:58.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.263+0000 7f2d05ffb700 1 -- 172.21.15.150:0/327750456 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d0000f040 con 0x7f2d08107a20 2024-05-10T04:06:58.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.263+0000 7f2d05ffb700 1 -- 172.21.15.150:0/327750456 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0000cda0 con 0x7f2d08107a20 2024-05-10T04:06:58.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.263+0000 7f2d0ebcb700 1 -- 172.21.15.150:0/327750456 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0819ff60 con 0x7f2d08107a20 2024-05-10T04:06:58.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.263+0000 7f2d0ebcb700 1 -- 172.21.15.150:0/327750456 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d081a0390 con 0x7f2d08107a20 2024-05-10T04:06:58.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.265+0000 7f2d05ffb700 1 -- 172.21.15.150:0/327750456 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2d00003a00 con 0x7f2d08107a20 2024-05-10T04:06:58.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.265+0000 7f2d05ffb700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2cf80a1240 0x7f2cf80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.266+0000 7f2d05ffb700 1 -- 172.21.15.150:0/327750456 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2d000cc6d0 con 0x7f2d08107a20 2024-05-10T04:06:58.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.266+0000 7f2d0c967700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2cf80a1240 0x7f2cf80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.266+0000 7f2d0c967700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2cf80a1240 0x7f2cf80a3750 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2d0806a1c0 tx=0x7f2cfc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:58.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d1ef0fb80 1 Processor -- start 2024-05-10T04:06:58.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d1ef0fb80 1 -- start start 2024-05-10T04:06:58.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d1ef0fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f96aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d1ef0fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f96b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d1ef0fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f9727c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d1ef0fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f891150 con 0x564a7f96a6a0 2024-05-10T04:06:58.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d1ef0fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f88eb10 con 0x564a7f96bec0 2024-05-10T04:06:58.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d1ef0fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f88a400 con 0x564a7f96b570 2024-05-10T04:06:58.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d0c967700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f96aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d0c967700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f96aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52598/0 (socket says 172.21.15.150:52598) 2024-05-10T04:06:58.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d0c967700 1 -- 172.21.15.150:0/1535599446 learned_addr learned my addr 172.21.15.150:0/1535599446 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:58.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d07fff700 1 --2- 172.21.15.150:0/1535599446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f96b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d0d168700 1 --2- 172.21.15.150:0/1535599446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f9727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d0c967700 1 -- 172.21.15.150:0/1535599446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 msgr2=0x564a7f96b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d0c967700 1 --2- 172.21.15.150:0/1535599446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f96b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d0c967700 1 -- 172.21.15.150:0/1535599446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 msgr2=0x564a7f9727c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d0c967700 1 --2- 172.21.15.150:0/1535599446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f9727c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d0c967700 1 -- 172.21.15.150:0/1535599446 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a7f70b3b0 con 0x564a7f96a6a0 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d0d168700 1 --2- 172.21.15.150:0/1535599446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f9727c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.267+0000 7f2d0c967700 1 --2- 172.21.15.150:0/1535599446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f96aaf0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0043c0 tx=0x7f2cfc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b531fdbaf5589bf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.268+0000 7f2cf2ffd700 1 -- 172.21.15.150:0/1535599446 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc003d10 con 0x564a7f96a6a0 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.268+0000 7f2cf2ffd700 1 -- 172.21.15.150:0/1535599446 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cfc003e70 con 0x564a7f96a6a0 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.268+0000 7f2cf2ffd700 1 -- 172.21.15.150:0/1535599446 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc01e7d0 con 0x564a7f96a6a0 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.268+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/1535599446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 msgr2=0x564a7f96aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.268+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/1535599446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f96aaf0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0043c0 tx=0x7f2cfc014000 comp rx=0 tx=0).stop 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.268+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/1535599446 shutdown_connections 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.268+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/1535599446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f9727c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.268+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/1535599446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f96aaf0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.268+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/1535599446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f96b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.268+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/1535599446 >> 172.21.15.150:0/1535599446 conn(0x564a7f8cca60 msgr2=0x564a7f964fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.269+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/1535599446 shutdown_connections 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.269+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/1535599446 wait complete. 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.269+0000 7f2d1ef0fb80 1 -- start start 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.269+0000 7f2d1ef0fb80 1 Processor -- start 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.269+0000 7f2d1ef0fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f974740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.269+0000 7f2d1ef0fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f974c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.269+0000 7f2d1ef0fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f97bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.269+0000 7f2d1ef0fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f826560 con 0x564a7f96a6a0 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.269+0000 7f2d1ef0fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f826200 con 0x564a7f96bec0 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.269+0000 7f2d1ef0fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f70d6a0 con 0x564a7f96b570 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.269+0000 7f2d1ef0fb80 10 client.0 _ll_register_callbacks cb 0x564a7f82fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.269+0000 7f2d0c967700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f974740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2d0c967700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f974740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52612/0 (socket says 172.21.15.150:52612) 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2d0c967700 1 -- 172.21.15.150:0/1339412603 learned_addr learned my addr 172.21.15.150:0/1339412603 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:58.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2d07fff700 1 --2- 172.21.15.150:0/1339412603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f974c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2d0d168700 1 --2- 172.21.15.150:0/1339412603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f97bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2d0c967700 1 -- 172.21.15.150:0/1339412603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 msgr2=0x564a7f974c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2d0c967700 1 --2- 172.21.15.150:0/1339412603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f974c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2d0c967700 1 -- 172.21.15.150:0/1339412603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 msgr2=0x564a7f97bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2d0c967700 1 --2- 172.21.15.150:0/1339412603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f97bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2d0c967700 1 -- 172.21.15.150:0/1339412603 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cfc017070 con 0x564a7f96a6a0 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2d0c967700 1 --2- 172.21.15.150:0/1339412603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f974740 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc026660 tx=0x7f2cfc026690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0875591e7eb8688 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2ce7fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2ce7fff700 1 -- 172.21.15.150:0/1339412603 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc02b070 con 0x564a7f96a6a0 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2ce7fff700 1 -- 172.21.15.150:0/1339412603 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2cfc026bb0 con 0x564a7f96a6a0 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/1339412603 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a7f7e3270 con 0x564a7f96a6a0 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2ce7fff700 1 -- 172.21.15.150:0/1339412603 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cfc005050 con 0x564a7f96a6a0 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2d1ef0fb80 10 client.16362 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/1339412603 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564a7f6b7b60 con 0x564a7f96a6a0 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.270+0000 7f2ce7fff700 1 -- 172.21.15.150:0/1339412603 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a7f7e3270 con 0x564a7f96a6a0 2024-05-10T04:06:58.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.271+0000 7f2ce7fff700 1 -- 172.21.15.150:0/1339412603 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2cfc024020 con 0x564a7f96a6a0 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.271+0000 7f2d1ef0fb80 10 client.16362 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.271+0000 7f2d1ef0fb80 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.271+0000 7f2d1ef0fb80 20 client.16362 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.271+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/1339412603 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564a7f980110 0x564a7f982560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.271+0000 7f2d1ef0fb80 4 client.16362 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.271+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/1339412603 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a7f939780 con 0x564a7f980110 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.271+0000 7f2d07fff700 1 --2- 172.21.15.150:0/1339412603 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564a7f980110 0x564a7f982560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.271+0000 7f2ce7fff700 10 client.16362 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.271+0000 7f2d07fff700 1 --2- 172.21.15.150:0/1339412603 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564a7f980110 0x564a7f982560 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5df1f6d679428477 server_cookie=e5343016f5b568e4 in_seq=0 out_seq=0 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2ce7fff700 1 -- 172.21.15.150:0/1339412603 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a7f939780 con 0x564a7f980110 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2ce7fff700 10 client.16362 handle_command_reply: tid=0 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2d1ef0fb80 1 client.16362 shutdown 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/1339412603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 msgr2=0x564a7f974740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/1339412603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f974740 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc026660 tx=0x7f2cfc026690 comp rx=0 tx=0).stop 2024-05-10T04:06:58.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/1339412603 shutdown_connections 2024-05-10T04:06:58.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/1339412603 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564a7f980110 0x564a7f982560 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/1339412603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f97bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/1339412603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f974740 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/1339412603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f974c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2ce7fff700 0 client.16362 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:58.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/1339412603 >> 172.21.15.150:0/1339412603 conn(0x564a7f8cca60 msgr2=0x564a7f964fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:58.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/1339412603 shutdown_connections 2024-05-10T04:06:58.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/1339412603 wait complete. 2024-05-10T04:06:58.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.272+0000 7f2d1ef0fb80 20 client.16362 trim_cache size 0 max 16384 2024-05-10T04:06:58.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d1ef0fb80 1 Processor -- start 2024-05-10T04:06:58.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d1ef0fb80 1 -- start start 2024-05-10T04:06:58.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d1ef0fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f97c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d1ef0fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f97bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d1ef0fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f96e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d1ef0fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f97cce0 con 0x564a7f96a6a0 2024-05-10T04:06:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d1ef0fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f97fe30 con 0x564a7f96bec0 2024-05-10T04:06:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d1ef0fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f982530 con 0x564a7f96b570 2024-05-10T04:06:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d0c967700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f97c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d0c967700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f97c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52614/0 (socket says 172.21.15.150:52614) 2024-05-10T04:06:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d0c967700 1 -- 172.21.15.150:0/126162813 learned_addr learned my addr 172.21.15.150:0/126162813 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d07fff700 1 --2- 172.21.15.150:0/126162813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f97bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d0d168700 1 --2- 172.21.15.150:0/126162813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f96e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d0c967700 1 -- 172.21.15.150:0/126162813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 msgr2=0x564a7f97bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d0c967700 1 --2- 172.21.15.150:0/126162813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f97bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d0c967700 1 -- 172.21.15.150:0/126162813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 msgr2=0x564a7f96e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d0c967700 1 --2- 172.21.15.150:0/126162813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f96e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d0c967700 1 -- 172.21.15.150:0/126162813 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a7f850560 con 0x564a7f96a6a0 2024-05-10T04:06:58.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.286+0000 7f2d0d168700 1 --2- 172.21.15.150:0/126162813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f96e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:06:58.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.287+0000 7f2d0c967700 1 --2- 172.21.15.150:0/126162813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f97c170 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc011bb0 tx=0x7f2cfc0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e99db52d3b7afc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:58.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.287+0000 7f2cf37fe700 1 -- 172.21.15.150:0/126162813 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc02b070 con 0x564a7f96a6a0 2024-05-10T04:06:58.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.287+0000 7f2cf37fe700 1 -- 172.21.15.150:0/126162813 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cfc003960 con 0x564a7f96a6a0 2024-05-10T04:06:58.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.287+0000 7f2cf37fe700 1 -- 172.21.15.150:0/126162813 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc005580 con 0x564a7f96a6a0 2024-05-10T04:06:58.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.288+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/126162813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 msgr2=0x564a7f97c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.288+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/126162813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f97c170 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc011bb0 tx=0x7f2cfc0092d0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.288+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/126162813 shutdown_connections 2024-05-10T04:06:58.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.288+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/126162813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96bec0 0x564a7f96e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.288+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/126162813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96a6a0 0x564a7f97c170 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.288+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/126162813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f97bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.288+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/126162813 >> 172.21.15.150:0/126162813 conn(0x564a7f8cca60 msgr2=0x564a7f964f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:58.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.288+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/126162813 shutdown_connections 2024-05-10T04:06:58.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.288+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/126162813 wait complete. 2024-05-10T04:06:58.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.288+0000 7f2d1ef0fb80 1 -- start start 2024-05-10T04:06:58.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.288+0000 7f2d1ef0fb80 1 Processor -- start 2024-05-10T04:06:58.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d1ef0fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96a6a0 0x564a7f978d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d1ef0fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f9792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d1ef0fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96bec0 0x564a7f97a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d1ef0fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f97a720 con 0x564a7f96bec0 2024-05-10T04:06:58.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d1ef0fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f97a890 con 0x564a7f96a6a0 2024-05-10T04:06:58.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d1ef0fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f97aa50 con 0x564a7f96b570 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d1ef0fb80 10 client.0 _ll_register_callbacks cb 0x564a7f82fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d0d168700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96bec0 0x564a7f97a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d0c967700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96a6a0 0x564a7f978d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d0d168700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96bec0 0x564a7f97a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52622/0 (socket says 172.21.15.150:52622) 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d0d168700 1 -- 172.21.15.150:0/928281581 learned_addr learned my addr 172.21.15.150:0/928281581 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d07fff700 1 --2- 172.21.15.150:0/928281581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f9792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d0d168700 1 -- 172.21.15.150:0/928281581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 msgr2=0x564a7f9792b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d0d168700 1 --2- 172.21.15.150:0/928281581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f9792b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d0d168700 1 -- 172.21.15.150:0/928281581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96a6a0 msgr2=0x564a7f978d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d0d168700 1 --2- 172.21.15.150:0/928281581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96a6a0 0x564a7f978d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d0d168700 1 -- 172.21.15.150:0/928281581 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a7f6b7b60 con 0x564a7f96bec0 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d0c967700 1 --2- 172.21.15.150:0/928281581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96a6a0 0x564a7f978d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2d0d168700 1 --2- 172.21.15.150:0/928281581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96bec0 0x564a7f97a1e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2d00095d40 tx=0x7f2d00099290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332d0534d4ae581d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.289+0000 7f2ce7fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:58.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2ce7fff700 1 -- 172.21.15.150:0/928281581 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d000b1070 con 0x564a7f96bec0 2024-05-10T04:06:58.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2ce7fff700 1 -- 172.21.15.150:0/928281581 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2d00099840 con 0x564a7f96bec0 2024-05-10T04:06:58.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/928281581 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cfc003e70 con 0x564a7f96bec0 2024-05-10T04:06:58.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2ce7fff700 1 -- 172.21.15.150:0/928281581 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d0009d670 con 0x564a7f96bec0 2024-05-10T04:06:58.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2d1ef0fb80 10 client.16368 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:58.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/928281581 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564a7f70fcd0 con 0x564a7f96bec0 2024-05-10T04:06:58.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2ce7fff700 1 -- 172.21.15.150:0/928281581 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cfc003e70 con 0x564a7f96bec0 2024-05-10T04:06:58.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2ce7fff700 1 -- 172.21.15.150:0/928281581 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2d00098d30 con 0x564a7f96bec0 2024-05-10T04:06:58.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2d1ef0fb80 10 client.16368 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2d1ef0fb80 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2d1ef0fb80 20 client.16368 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/928281581 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564a7f9d6660 0x564a7f9d8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2d1ef0fb80 4 client.16368 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/928281581 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564a7f882ec0 con 0x564a7f9d6660 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.291+0000 7f2d0c967700 1 --2- 172.21.15.150:0/928281581 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564a7f9d6660 0x564a7f9d8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2ce7fff700 10 client.16368 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2d0c967700 1 --2- 172.21.15.150:0/928281581 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564a7f9d6660 0x564a7f9d8a80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bec14eac0f4665a server_cookie=f4760e159358df9b in_seq=0 out_seq=0 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2ce7fff700 1 -- 172.21.15.150:0/928281581 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564a7f882ec0 con 0x564a7f9d6660 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2ce7fff700 10 client.16368 handle_command_reply: tid=0 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2d1ef0fb80 1 client.16368 shutdown 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/928281581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96bec0 msgr2=0x564a7f97a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/928281581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96bec0 0x564a7f97a1e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2d00095d40 tx=0x7f2d00099290 comp rx=0 tx=0).stop 2024-05-10T04:06:58.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/928281581 shutdown_connections 2024-05-10T04:06:58.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/928281581 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x564a7f9d6660 0x564a7f9d8a80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/928281581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564a7f96bec0 0x564a7f97a1e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/928281581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a7f96a6a0 0x564a7f978d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2ce7fff700 0 client.16368 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:58.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2d1ef0fb80 1 --2- 172.21.15.150:0/928281581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a7f96b570 0x564a7f9792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/928281581 >> 172.21.15.150:0/928281581 conn(0x564a7f8cca60 msgr2=0x564a7f96f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:58.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/928281581 shutdown_connections 2024-05-10T04:06:58.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.292+0000 7f2d1ef0fb80 1 -- 172.21.15.150:0/928281581 wait complete. 2024-05-10T04:06:58.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.293+0000 7f2d1ef0fb80 20 client.16368 trim_cache size 0 max 16384 2024-05-10T04:06:58.300 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:06:58.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.293+0000 7f2d0ebcb700 1 -- 172.21.15.150:0/327750456 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2cf80a1240 msgr2=0x7f2cf80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.293+0000 7f2d0ebcb700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2cf80a1240 0x7f2cf80a3750 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2d0806a1c0 tx=0x7f2cfc009260 comp rx=0 tx=0).stop 2024-05-10T04:06:58.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.293+0000 7f2d0ebcb700 1 -- 172.21.15.150:0/327750456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d08107a20 msgr2=0x7f2d0819f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:58.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.293+0000 7f2d0ebcb700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d08107a20 0x7f2d0819f3a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000ef90 tx=0x7f2d0000c3a0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.293+0000 7f2d0ebcb700 1 -- 172.21.15.150:0/327750456 shutdown_connections 2024-05-10T04:06:58.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.293+0000 7f2d0ebcb700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2cf80a1240 0x7f2cf80a3750 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.293+0000 7f2d0ebcb700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d08107a20 0x7f2d0819f3a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.293+0000 7f2d0ebcb700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d08102760 0x7f2d08197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.293+0000 7f2d0ebcb700 1 --2- 172.21.15.150:0/327750456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d081050c0 0x7f2d08198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:58.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.293+0000 7f2d0ebcb700 1 -- 172.21.15.150:0/327750456 >> 172.21.15.150:0/327750456 conn(0x7f2d080fc180 msgr2=0x7f2d081031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:58.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.293+0000 7f2d0ebcb700 1 -- 172.21.15.150:0/327750456 shutdown_connections 2024-05-10T04:06:58.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:58.293+0000 7f2d0ebcb700 1 -- 172.21.15.150:0/327750456 wait complete. 2024-05-10T04:06:58.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:58 smithi150 ceph-mon[42477]: pgmap v361: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-05-10T04:06:58.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:06:58 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1054703384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:06:58.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:58 smithi181 ceph-mon[46602]: pgmap v361: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-05-10T04:06:58.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:06:58 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1054703384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:06:58.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:58 smithi161 ceph-mon[46918]: pgmap v361: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-05-10T04:06:58.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:06:58 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1054703384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:06:59.309 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T04:06:59.309 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T04:06:59.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.458+0000 7f6cce0b9700 1 -- 172.21.15.150:0/2080518914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8101a70 msgr2=0x7f6cc8101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.458+0000 7f6cce0b9700 1 --2- 172.21.15.150:0/2080518914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8101a70 0x7f6cc8101f40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009990 tx=0x7f6cb8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.458+0000 7f6cce0b9700 1 -- 172.21.15.150:0/2080518914 shutdown_connections 2024-05-10T04:06:59.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.458+0000 7f6cce0b9700 1 --2- 172.21.15.150:0/2080518914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cc81025d0 0x7f6cc8109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.458+0000 7f6cce0b9700 1 --2- 172.21.15.150:0/2080518914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cc8101090 0x7f6cc81014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.458+0000 7f6cce0b9700 1 --2- 172.21.15.150:0/2080518914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8101a70 0x7f6cc8101f40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.458+0000 7f6cce0b9700 1 -- 172.21.15.150:0/2080518914 >> 172.21.15.150:0/2080518914 conn(0x7f6cc80fb8f0 msgr2=0x7f6cc80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:59.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.458+0000 7f6cce0b9700 1 -- 172.21.15.150:0/2080518914 shutdown_connections 2024-05-10T04:06:59.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.458+0000 7f6cce0b9700 1 -- 172.21.15.150:0/2080518914 wait complete. 2024-05-10T04:06:59.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.459+0000 7f6cce0b9700 1 Processor -- start 2024-05-10T04:06:59.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.459+0000 7f6cce0b9700 1 -- start start 2024-05-10T04:06:59.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.459+0000 7f6cce0b9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8101090 0x7f6cc8196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.459+0000 7f6cce0b9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cc8101a70 0x7f6cc81973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.459+0000 7f6cce0b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cc81025d0 0x7f6cc819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.459+0000 7f6cce0b9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc819e990 con 0x7f6cc8101090 2024-05-10T04:06:59.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.459+0000 7f6cce0b9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc819eb00 con 0x7f6cc8101a70 2024-05-10T04:06:59.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.459+0000 7f6cce0b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc819ecc0 con 0x7f6cc81025d0 2024-05-10T04:06:59.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.459+0000 7f6cc77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8101090 0x7f6cc8196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.459+0000 7f6cc77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8101090 0x7f6cc8196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37952/0 (socket says 172.21.15.150:37952) 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.459+0000 7f6cc77fe700 1 -- 172.21.15.150:0/3939816424 learned_addr learned my addr 172.21.15.150:0/3939816424 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.459+0000 7f6cc6ffd700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cc8101a70 0x7f6cc81973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.459+0000 7f6cc7fff700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cc81025d0 0x7f6cc819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.460+0000 7f6cc77fe700 1 -- 172.21.15.150:0/3939816424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cc81025d0 msgr2=0x7f6cc819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.460+0000 7f6cc77fe700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cc81025d0 0x7f6cc819e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.460+0000 7f6cc77fe700 1 -- 172.21.15.150:0/3939816424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cc8101a70 msgr2=0x7f6cc81973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.460+0000 7f6cc77fe700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cc8101a70 0x7f6cc81973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.460+0000 7f6cc77fe700 1 -- 172.21.15.150:0/3939816424 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb80095f0 con 0x7f6cc8101090 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.460+0000 7f6cc7fff700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cc81025d0 0x7f6cc819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.460+0000 7f6cc6ffd700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cc8101a70 0x7f6cc81973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.460+0000 7f6cc77fe700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8101090 0x7f6cc8196eb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00b4d0 tx=0x7f6cbc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.460+0000 7f6cc4ff9700 1 -- 172.21.15.150:0/3939816424 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc011830 con 0x7f6cc8101090 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.460+0000 7f6cc4ff9700 1 -- 172.21.15.150:0/3939816424 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cbc011e50 con 0x7f6cc8101090 2024-05-10T04:06:59.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.460+0000 7f6cce0b9700 1 -- 172.21.15.150:0/3939816424 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc819efa0 con 0x7f6cc8101090 2024-05-10T04:06:59.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.460+0000 7f6cc4ff9700 1 -- 172.21.15.150:0/3939816424 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc010e50 con 0x7f6cc8101090 2024-05-10T04:06:59.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.460+0000 7f6cce0b9700 1 -- 172.21.15.150:0/3939816424 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc819f450 con 0x7f6cc8101090 2024-05-10T04:06:59.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.462+0000 7f6cc4ff9700 1 -- 172.21.15.150:0/3939816424 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f6cbc01a400 con 0x7f6cc8101090 2024-05-10T04:06:59.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.463+0000 7f6cc4ff9700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6cb00a11c0 0x7f6cb00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.463+0000 7f6cc6ffd700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6cb00a11c0 0x7f6cb00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.463+0000 7f6cc4ff9700 1 -- 172.21.15.150:0/3939816424 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6cbc0cc7f0 con 0x7f6cc8101090 2024-05-10T04:06:59.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.463+0000 7f6cc6ffd700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6cb00a11c0 0x7f6cb00a36d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80038c0 tx=0x7f6cb801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:59.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cde3fdb80 1 Processor -- start 2024-05-10T04:06:59.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cde3fdb80 1 -- start start 2024-05-10T04:06:59.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cde3fdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 0x56286a484340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cde3fdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a486060 0x56286a484880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cde3fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 0x56286a490990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cde3fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286a3ab1d0 con 0x56286a484e60 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cde3fdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286a3a8b90 con 0x56286a485bb0 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cde3fdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286a3a4480 con 0x56286a486060 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cc7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 0x56286a490990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cc7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 0x56286a490990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37968/0 (socket says 172.21.15.150:37968) 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cc7fff700 1 -- 172.21.15.150:0/2423810788 learned_addr learned my addr 172.21.15.150:0/2423810788 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cc77fe700 1 --2- 172.21.15.150:0/2423810788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 0x56286a484340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cc6ffd700 1 --2- 172.21.15.150:0/2423810788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a486060 0x56286a484880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cc7fff700 1 -- 172.21.15.150:0/2423810788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a486060 msgr2=0x56286a484880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cc7fff700 1 --2- 172.21.15.150:0/2423810788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a486060 0x56286a484880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cc7fff700 1 -- 172.21.15.150:0/2423810788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 msgr2=0x56286a484340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cc7fff700 1 --2- 172.21.15.150:0/2423810788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 0x56286a484340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.464+0000 7f6cc7fff700 1 -- 172.21.15.150:0/2423810788 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56286a225430 con 0x56286a484e60 2024-05-10T04:06:59.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.465+0000 7f6cc7fff700 1 --2- 172.21.15.150:0/2423810788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 0x56286a490990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009cb0 tx=0x7f6cc0007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff25a6a8054af6b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:59.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.465+0000 7f6cadffb700 1 -- 172.21.15.150:0/2423810788 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0014400 con 0x56286a484e60 2024-05-10T04:06:59.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.465+0000 7f6cadffb700 1 -- 172.21.15.150:0/2423810788 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc0014a20 con 0x56286a484e60 2024-05-10T04:06:59.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.465+0000 7f6cadffb700 1 -- 172.21.15.150:0/2423810788 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0013a00 con 0x56286a484e60 2024-05-10T04:06:59.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.466+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/2423810788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 msgr2=0x56286a490990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.466+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/2423810788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 0x56286a490990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009cb0 tx=0x7f6cc0007500 comp rx=0 tx=0).stop 2024-05-10T04:06:59.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.466+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/2423810788 shutdown_connections 2024-05-10T04:06:59.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.466+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/2423810788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 0x56286a490990 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.466+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/2423810788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 0x56286a484340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.466+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/2423810788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a486060 0x56286a484880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.466+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/2423810788 >> 172.21.15.150:0/2423810788 conn(0x56286a3e6ae0 msgr2=0x56286a47f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:59.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.466+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/2423810788 shutdown_connections 2024-05-10T04:06:59.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.466+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/2423810788 wait complete. 2024-05-10T04:06:59.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.466+0000 7f6cde3fdb80 1 -- start start 2024-05-10T04:06:59.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.466+0000 7f6cde3fdb80 1 Processor -- start 2024-05-10T04:06:59.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cde3fdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a484e60 0x56286a492b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cde3fdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 0x56286a4930d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cde3fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a486060 0x56286a49a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cde3fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286a3405e0 con 0x56286a486060 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cde3fdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286a340280 con 0x56286a485bb0 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cde3fdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286a227720 con 0x56286a484e60 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cde3fdb80 10 client.0 _ll_register_callbacks cb 0x56286a349bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cc7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a486060 0x56286a49a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cc7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a486060 0x56286a49a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37970/0 (socket says 172.21.15.150:37970) 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cc7fff700 1 -- 172.21.15.150:0/2351234390 learned_addr learned my addr 172.21.15.150:0/2351234390 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cc6ffd700 1 --2- 172.21.15.150:0/2351234390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 0x56286a4930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cc77fe700 1 --2- 172.21.15.150:0/2351234390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a484e60 0x56286a492b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cc7fff700 1 -- 172.21.15.150:0/2351234390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a484e60 msgr2=0x56286a492b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cc7fff700 1 --2- 172.21.15.150:0/2351234390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a484e60 0x56286a492b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cc7fff700 1 -- 172.21.15.150:0/2351234390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 msgr2=0x56286a4930d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cc7fff700 1 --2- 172.21.15.150:0/2351234390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 0x56286a4930d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cc7fff700 1 -- 172.21.15.150:0/2351234390 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc00098e0 con 0x56286a486060 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cc7fff700 1 --2- 172.21.15.150:0/2351234390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a486060 0x56286a49a030 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0008fb0 tx=0x7f6cc0004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aecc25ccfe7de48 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6ca6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6ca6ffd700 1 -- 172.21.15.150:0/2351234390 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0014600 con 0x56286a486060 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6ca6ffd700 1 -- 172.21.15.150:0/2351234390 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6cc0014760 con 0x56286a486060 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.467+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/2351234390 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56286a2fd2f0 con 0x56286a486060 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.468+0000 7f6ca6ffd700 1 -- 172.21.15.150:0/2351234390 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cc0013740 con 0x56286a486060 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.468+0000 7f6cde3fdb80 10 client.16380 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.468+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/2351234390 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56286a1d1be0 con 0x56286a486060 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.468+0000 7f6ca6ffd700 1 -- 172.21.15.150:0/2351234390 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56286a2fd2f0 con 0x56286a486060 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.468+0000 7f6ca6ffd700 1 -- 172.21.15.150:0/2351234390 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6cc0005410 con 0x56286a486060 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.468+0000 7f6cde3fdb80 10 client.16380 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.468+0000 7f6cde3fdb80 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:59.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.468+0000 7f6cde3fdb80 20 client.16380 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.468+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/2351234390 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56286a49e6d0 0x56286a4a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.468+0000 7f6cde3fdb80 4 client.16380 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.468+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/2351234390 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56286a453800 con 0x56286a49e6d0 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.468+0000 7f6cc77fe700 1 --2- 172.21.15.150:0/2351234390 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56286a49e6d0 0x56286a4a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.469+0000 7f6ca6ffd700 10 client.16380 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.469+0000 7f6cc77fe700 1 --2- 172.21.15.150:0/2351234390 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56286a49e6d0 0x56286a4a0b20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e6306a1950648a8 server_cookie=e97121d85f0953ac in_seq=0 out_seq=0 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.469+0000 7f6ca6ffd700 1 -- 172.21.15.150:0/2351234390 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56286a453800 con 0x56286a49e6d0 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.469+0000 7f6ca6ffd700 10 client.16380 handle_command_reply: tid=0 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.469+0000 7f6cde3fdb80 1 client.16380 shutdown 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.470+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/2351234390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a486060 msgr2=0x56286a49a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.470+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/2351234390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a486060 0x56286a49a030 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0008fb0 tx=0x7f6cc0004a30 comp rx=0 tx=0).stop 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.470+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/2351234390 shutdown_connections 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.470+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/2351234390 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56286a49e6d0 0x56286a4a0b20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.470+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/2351234390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a486060 0x56286a49a030 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.470+0000 7f6ca6ffd700 0 client.16380 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:59.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.470+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/2351234390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a484e60 0x56286a492b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.470+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/2351234390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 0x56286a4930d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.470+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/2351234390 >> 172.21.15.150:0/2351234390 conn(0x56286a3e6ae0 msgr2=0x56286a47f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:59.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.470+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/2351234390 shutdown_connections 2024-05-10T04:06:59.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.470+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/2351234390 wait complete. 2024-05-10T04:06:59.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.470+0000 7f6cde3fdb80 20 client.16380 trim_cache size 0 max 16384 2024-05-10T04:06:59.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.487+0000 7f6cde3fdb80 1 Processor -- start 2024-05-10T04:06:59.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.487+0000 7f6cde3fdb80 1 -- start start 2024-05-10T04:06:59.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.487+0000 7f6cde3fdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a484e60 0x56286a4917e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.487+0000 7f6cde3fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a485bb0 0x56286a499ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.487+0000 7f6cde3fdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a486060 0x56286a492b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.487+0000 7f6cde3fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286a49b130 con 0x56286a485bb0 2024-05-10T04:06:59.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.487+0000 7f6cde3fdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286a485440 con 0x56286a486060 2024-05-10T04:06:59.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.487+0000 7f6cde3fdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286a47fb50 con 0x56286a484e60 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.487+0000 7f6cc6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a485bb0 0x56286a499ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cc6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a485bb0 0x56286a499ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37984/0 (socket says 172.21.15.150:37984) 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cc6ffd700 1 -- 172.21.15.150:0/1170757505 learned_addr learned my addr 172.21.15.150:0/1170757505 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cc7fff700 1 --2- 172.21.15.150:0/1170757505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a486060 0x56286a492b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cc77fe700 1 --2- 172.21.15.150:0/1170757505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a484e60 0x56286a4917e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cc6ffd700 1 -- 172.21.15.150:0/1170757505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a484e60 msgr2=0x56286a4917e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cc6ffd700 1 --2- 172.21.15.150:0/1170757505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a484e60 0x56286a4917e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cc6ffd700 1 -- 172.21.15.150:0/1170757505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a486060 msgr2=0x56286a492b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cc6ffd700 1 --2- 172.21.15.150:0/1170757505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a486060 0x56286a492b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cc6ffd700 1 -- 172.21.15.150:0/1170757505 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56286a36a5e0 con 0x56286a485bb0 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cc77fe700 1 --2- 172.21.15.150:0/1170757505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a484e60 0x56286a4917e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cc7fff700 1 --2- 172.21.15.150:0/1170757505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a486060 0x56286a492b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cc6ffd700 1 --2- 172.21.15.150:0/1170757505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a485bb0 0x56286a499ff0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009990 tx=0x7f6cb801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820b20ecc912dacc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cae7fc700 1 -- 172.21.15.150:0/1170757505 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb802a050 con 0x56286a485bb0 2024-05-10T04:06:59.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cae7fc700 1 -- 172.21.15.150:0/1170757505 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cb802b040 con 0x56286a485bb0 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.488+0000 7f6cae7fc700 1 -- 172.21.15.150:0/1170757505 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb801d040 con 0x56286a485bb0 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.489+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/1170757505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a485bb0 msgr2=0x56286a499ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.489+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/1170757505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a485bb0 0x56286a499ff0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009990 tx=0x7f6cb801e000 comp rx=0 tx=0).stop 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.489+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/1170757505 shutdown_connections 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.489+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/1170757505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a486060 0x56286a492b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.489+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/1170757505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a484e60 0x56286a4917e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.489+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/1170757505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a485bb0 0x56286a499ff0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.489+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/1170757505 >> 172.21.15.150:0/1170757505 conn(0x56286a3e6ae0 msgr2=0x56286a4910d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.489+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/1170757505 shutdown_connections 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.489+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/1170757505 wait complete. 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.489+0000 7f6cde3fdb80 1 -- start start 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cde3fdb80 1 Processor -- start 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cde3fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 0x56286a498b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cde3fdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 0x56286a499070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cde3fdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a486060 0x56286a4f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cde3fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286a4997d0 con 0x56286a484e60 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cde3fdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286a499940 con 0x56286a485bb0 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cde3fdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286a4f50e0 con 0x56286a486060 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cde3fdb80 10 client.0 _ll_register_callbacks cb 0x56286a349bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cc77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 0x56286a498b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cc77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 0x56286a498b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37990/0 (socket says 172.21.15.150:37990) 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cc77fe700 1 -- 172.21.15.150:0/473920931 learned_addr learned my addr 172.21.15.150:0/473920931 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cc6ffd700 1 --2- 172.21.15.150:0/473920931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 0x56286a499070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cc7fff700 1 --2- 172.21.15.150:0/473920931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a486060 0x56286a4f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cc77fe700 1 -- 172.21.15.150:0/473920931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a486060 msgr2=0x56286a4f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cc77fe700 1 --2- 172.21.15.150:0/473920931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a486060 0x56286a4f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cc77fe700 1 -- 172.21.15.150:0/473920931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 msgr2=0x56286a499070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cc77fe700 1 --2- 172.21.15.150:0/473920931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 0x56286a499070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cc77fe700 1 -- 172.21.15.150:0/473920931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc8109940 con 0x56286a484e60 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6cc77fe700 1 --2- 172.21.15.150:0/473920931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 0x56286a498b30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00bc90 tx=0x7f6cbc085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552068f3b1149073 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.490+0000 7f6ca6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6ca6ffd700 1 -- 172.21.15.150:0/473920931 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc0661a0 con 0x56286a484e60 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6ca6ffd700 1 -- 172.21.15.150:0/473920931 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6cbc0a2c10 con 0x56286a484e60 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/473920931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cc0014a20 con 0x56286a484e60 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6ca6ffd700 1 -- 172.21.15.150:0/473920931 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cbc0104f0 con 0x56286a484e60 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6cde3fdb80 10 client.16386 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/473920931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56286a229d50 con 0x56286a484e60 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6ca6ffd700 1 -- 172.21.15.150:0/473920931 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc0014a20 con 0x56286a484e60 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6ca6ffd700 1 -- 172.21.15.150:0/473920931 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6cbc0b4020 con 0x56286a484e60 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6cde3fdb80 10 client.16386 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6cde3fdb80 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6cde3fdb80 20 client.16386 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:59.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/473920931 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56286a4f6a30 0x56286a4f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6cde3fdb80 4 client.16386 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T04:06:59.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.491+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/473920931 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56286a39cf40 con 0x56286a4f6a30 2024-05-10T04:06:59.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.492+0000 7f6cc6ffd700 1 --2- 172.21.15.150:0/473920931 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56286a4f6a30 0x56286a4f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.492+0000 7f6ca6ffd700 10 client.16386 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:59.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.492+0000 7f6cc6ffd700 1 --2- 172.21.15.150:0/473920931 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56286a4f6a30 0x56286a4f8e50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d7817d0d90253a server_cookie=1a85aab610fed777 in_seq=0 out_seq=0 2024-05-10T04:06:59.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.493+0000 7f6ca6ffd700 1 -- 172.21.15.150:0/473920931 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56286a39cf40 con 0x56286a4f6a30 2024-05-10T04:06:59.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.493+0000 7f6ca6ffd700 10 client.16386 handle_command_reply: tid=0 2024-05-10T04:06:59.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.493+0000 7f6cde3fdb80 1 client.16386 shutdown 2024-05-10T04:06:59.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.493+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/473920931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 msgr2=0x56286a498b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.493+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/473920931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 0x56286a498b30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00bc90 tx=0x7f6cbc085040 comp rx=0 tx=0).stop 2024-05-10T04:06:59.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.493+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/473920931 shutdown_connections 2024-05-10T04:06:59.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.493+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/473920931 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x56286a4f6a30 0x56286a4f8e50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.493+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/473920931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56286a486060 0x56286a4f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.493+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/473920931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56286a484e60 0x56286a498b30 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.493+0000 7f6ca6ffd700 0 client.16386 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:59.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.493+0000 7f6cde3fdb80 1 --2- 172.21.15.150:0/473920931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56286a485bb0 0x56286a499070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.493+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/473920931 >> 172.21.15.150:0/473920931 conn(0x56286a3e6ae0 msgr2=0x56286a492b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:59.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/473920931 shutdown_connections 2024-05-10T04:06:59.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cde3fdb80 1 -- 172.21.15.150:0/473920931 wait complete. 2024-05-10T04:06:59.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cde3fdb80 20 client.16386 trim_cache size 0 max 16384 2024-05-10T04:06:59.501 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:06:59.501 INFO:teuthology.orchestra.run.smithi150.stdout: "return_code": 0, 2024-05-10T04:06:59.501 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_tag": "5fcedd15-813d-4c71-baf9-e9280e1a12d4", 2024-05-10T04:06:59.502 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": "asynchronous" 2024-05-10T04:06:59.502 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:06:59.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cce0b9700 1 -- 172.21.15.150:0/3939816424 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6cb00a11c0 msgr2=0x7f6cb00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cce0b9700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6cb00a11c0 0x7f6cb00a36d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80038c0 tx=0x7f6cb801a040 comp rx=0 tx=0).stop 2024-05-10T04:06:59.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cce0b9700 1 -- 172.21.15.150:0/3939816424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8101090 msgr2=0x7f6cc8196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cce0b9700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8101090 0x7f6cc8196eb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00b4d0 tx=0x7f6cbc00b890 comp rx=0 tx=0).stop 2024-05-10T04:06:59.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cce0b9700 1 -- 172.21.15.150:0/3939816424 shutdown_connections 2024-05-10T04:06:59.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cce0b9700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f6cb00a11c0 0x7f6cb00a36d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cce0b9700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cc81025d0 0x7f6cc819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cce0b9700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8101090 0x7f6cc8196eb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cce0b9700 1 --2- 172.21.15.150:0/3939816424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cc8101a70 0x7f6cc81973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cce0b9700 1 -- 172.21.15.150:0/3939816424 >> 172.21.15.150:0/3939816424 conn(0x7f6cc80fb8f0 msgr2=0x7f6cc8105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:59.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cce0b9700 1 -- 172.21.15.150:0/3939816424 shutdown_connections 2024-05-10T04:06:59.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.494+0000 7f6cce0b9700 1 -- 172.21.15.150:0/3939816424 wait complete. 2024-05-10T04:06:59.509 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:06:59.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.647+0000 7fc8c02a7700 1 -- 172.21.15.150:0/4079016102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b80750e0 msgr2=0x7fc8b80735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.647+0000 7fc8c02a7700 1 --2- 172.21.15.150:0/4079016102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b80750e0 0x7fc8b80735a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4009990 tx=0x7fc8b4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.647+0000 7fc8c02a7700 1 -- 172.21.15.150:0/4079016102 shutdown_connections 2024-05-10T04:06:59.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.647+0000 7fc8c02a7700 1 --2- 172.21.15.150:0/4079016102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b8109ad0 0x7fc8b810bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.647+0000 7fc8c02a7700 1 --2- 172.21.15.150:0/4079016102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b80750e0 0x7fc8b80735a0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.647+0000 7fc8c02a7700 1 --2- 172.21.15.150:0/4079016102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b8073ae0 0x7fc8b8073f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.647+0000 7fc8c02a7700 1 -- 172.21.15.150:0/4079016102 >> 172.21.15.150:0/4079016102 conn(0x7fc8b80fd1e0 msgr2=0x7fc8b80ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:59.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.647+0000 7fc8c02a7700 1 -- 172.21.15.150:0/4079016102 shutdown_connections 2024-05-10T04:06:59.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.647+0000 7fc8c02a7700 1 -- 172.21.15.150:0/4079016102 wait complete. 2024-05-10T04:06:59.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8c02a7700 1 Processor -- start 2024-05-10T04:06:59.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8c02a7700 1 -- start start 2024-05-10T04:06:59.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8c02a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b8073ae0 0x7fc8b819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8c02a7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b80750e0 0x7fc8b819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8c02a7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b8109ad0 0x7fc8b81a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8c02a7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b81a1d00 con 0x7fc8b80750e0 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8c02a7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b81a1e70 con 0x7fc8b8109ad0 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8c02a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b81a2030 con 0x7fc8b8073ae0 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8bd842700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b80750e0 0x7fc8b819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8bd842700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b80750e0 0x7fc8b819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38008/0 (socket says 172.21.15.150:38008) 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8bd842700 1 -- 172.21.15.150:0/1069488225 learned_addr learned my addr 172.21.15.150:0/1069488225 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8be844700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b8109ad0 0x7fc8b81a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8be043700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b8073ae0 0x7fc8b819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8bd842700 1 -- 172.21.15.150:0/1069488225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b8073ae0 msgr2=0x7fc8b819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8bd842700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b8073ae0 0x7fc8b819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8bd842700 1 -- 172.21.15.150:0/1069488225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b8109ad0 msgr2=0x7fc8b81a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.648+0000 7fc8bd842700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b8109ad0 0x7fc8b81a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.649+0000 7fc8bd842700 1 -- 172.21.15.150:0/1069488225 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b40095f0 con 0x7fc8b80750e0 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.649+0000 7fc8be043700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b8073ae0 0x7fc8b819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:06:59.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.649+0000 7fc8bd842700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b80750e0 0x7fc8b819a760 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800e970 tx=0x7fc8a800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:59.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.649+0000 7fc8af7fe700 1 -- 172.21.15.150:0/1069488225 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8009790 con 0x7fc8b80750e0 2024-05-10T04:06:59.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.649+0000 7fc8af7fe700 1 -- 172.21.15.150:0/1069488225 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8a8004cf0 con 0x7fc8b80750e0 2024-05-10T04:06:59.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.649+0000 7fc8af7fe700 1 -- 172.21.15.150:0/1069488225 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8005680 con 0x7fc8b80750e0 2024-05-10T04:06:59.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.649+0000 7fc8c02a7700 1 -- 172.21.15.150:0/1069488225 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b81a2310 con 0x7fc8b80750e0 2024-05-10T04:06:59.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.649+0000 7fc8c02a7700 1 -- 172.21.15.150:0/1069488225 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b81a27d0 con 0x7fc8b80750e0 2024-05-10T04:06:59.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.654+0000 7fc8d05ebb80 1 Processor -- start 2024-05-10T04:06:59.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.654+0000 7fc8d05ebb80 1 -- start start 2024-05-10T04:06:59.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.654+0000 7fc8d05ebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b2f6a0 0x557137b2faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.654+0000 7fc8d05ebb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30570 0x557137b30980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.654+0000 7fc8d05ebb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30ec0 0x557137b377c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.654+0000 7fc8d05ebb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557137a56150 con 0x557137b30570 2024-05-10T04:06:59.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.654+0000 7fc8d05ebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557137a53b10 con 0x557137b2f6a0 2024-05-10T04:06:59.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.654+0000 7fc8d05ebb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557137a4f400 con 0x557137b30ec0 2024-05-10T04:06:59.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.654+0000 7fc8be844700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b2f6a0 0x557137b2faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.654+0000 7fc8be844700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b2f6a0 0x557137b2faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:56600/0 (socket says 172.21.15.150:56600) 2024-05-10T04:06:59.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.654+0000 7fc8be844700 1 -- 172.21.15.150:0/185059164 learned_addr learned my addr 172.21.15.150:0/185059164 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:59.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.655+0000 7fc8be043700 1 --2- 172.21.15.150:0/185059164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30570 0x557137b30980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.655+0000 7fc8be844700 1 -- 172.21.15.150:0/185059164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30ec0 msgr2=0x557137b377c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:06:59.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.655+0000 7fc8be844700 1 --2- 172.21.15.150:0/185059164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30ec0 0x557137b377c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.655+0000 7fc8be844700 1 -- 172.21.15.150:0/185059164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30570 msgr2=0x557137b30980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.655+0000 7fc8be844700 1 --2- 172.21.15.150:0/185059164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30570 0x557137b30980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.655+0000 7fc8be844700 1 -- 172.21.15.150:0/185059164 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571378d03b0 con 0x557137b2f6a0 2024-05-10T04:06:59.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.655+0000 7fc8af7fe700 1 -- 172.21.15.150:0/1069488225 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc8a8004750 con 0x7fc8b80750e0 2024-05-10T04:06:59.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.655+0000 7fc8be844700 1 --2- 172.21.15.150:0/185059164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b2f6a0 0x557137b2faf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009d10 tx=0x7fc8b00077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb2429e539e7ac0e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:59.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.655+0000 7fc8acd7f700 1 -- 172.21.15.150:0/185059164 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0018070 con 0x557137b2f6a0 2024-05-10T04:06:59.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.655+0000 7fc8acd7f700 1 -- 172.21.15.150:0/185059164 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8b000f050 con 0x557137b2f6a0 2024-05-10T04:06:59.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.655+0000 7fc8acd7f700 1 -- 172.21.15.150:0/185059164 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0004720 con 0x557137b2f6a0 2024-05-10T04:06:59.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.656+0000 7fc8af7fe700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc8a40a11c0 0x7fc8a40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.656+0000 7fc8be043700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc8a40a11c0 0x7fc8a40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.656+0000 7fc8af7fe700 1 -- 172.21.15.150:0/1069488225 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc8a8014070 con 0x7fc8b80750e0 2024-05-10T04:06:59.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.656+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/185059164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b2f6a0 msgr2=0x557137b2faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.656+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/185059164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b2f6a0 0x557137b2faf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009d10 tx=0x7fc8b00077e0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.656+0000 7fc8be043700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc8a40a11c0 0x7fc8a40a36d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4006010 tx=0x7fc8b400b770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:59.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.657+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/185059164 shutdown_connections 2024-05-10T04:06:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.657+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/185059164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30ec0 0x557137b377c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.657+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/185059164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b2f6a0 0x557137b2faf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.657+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/185059164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30570 0x557137b30980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.657+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/185059164 >> 172.21.15.150:0/185059164 conn(0x557137a91a60 msgr2=0x557137b29fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.657+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/185059164 shutdown_connections 2024-05-10T04:06:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.657+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/185059164 wait complete. 2024-05-10T04:06:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.657+0000 7fc8d05ebb80 1 -- start start 2024-05-10T04:06:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.657+0000 7fc8d05ebb80 1 Processor -- start 2024-05-10T04:06:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8d05ebb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b2f6a0 0x557137b41ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8d05ebb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30570 0x557137b42430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8d05ebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b30ec0 0x557137b49390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8d05ebb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571379eb560 con 0x557137b2f6a0 2024-05-10T04:06:59.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8d05ebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571379eb200 con 0x557137b30ec0 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8d05ebb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571378d26a0 con 0x557137b30570 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8d05ebb80 10 client.0 _ll_register_callbacks cb 0x5571379f4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8bd842700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30570 0x557137b42430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8bd842700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30570 0x557137b42430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:41370/0 (socket says 172.21.15.150:41370) 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8bd842700 1 -- 172.21.15.150:0/3513522635 learned_addr learned my addr 172.21.15.150:0/3513522635 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8be844700 1 --2- 172.21.15.150:0/3513522635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b30ec0 0x557137b49390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8be844700 1 -- 172.21.15.150:0/3513522635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30570 msgr2=0x557137b42430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8be844700 1 --2- 172.21.15.150:0/3513522635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30570 0x557137b42430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8be844700 1 -- 172.21.15.150:0/3513522635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b2f6a0 msgr2=0x557137b41ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8be844700 1 --2- 172.21.15.150:0/3513522635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b2f6a0 0x557137b41ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8be844700 1 -- 172.21.15.150:0/3513522635 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b0009940 con 0x557137b30ec0 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8be844700 1 --2- 172.21.15.150:0/3513522635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b30ec0 0x557137b49390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0007b30 tx=0x7fc8b0003cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe8ccde9a242781 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.658+0000 7fc8a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8a1ffb700 1 -- 172.21.15.150:0/3513522635 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0018070 con 0x557137b30ec0 2024-05-10T04:06:59.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/3513522635 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571379a8270 con 0x557137b30ec0 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8a1ffb700 1 -- 172.21.15.150:0/3513522635 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc8b001e450 con 0x557137b30ec0 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8a1ffb700 1 -- 172.21.15.150:0/3513522635 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8b00040c0 con 0x557137b30ec0 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8d05ebb80 10 client.26024 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/3513522635 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55713787cb60 con 0x557137b30ec0 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8a1ffb700 1 -- 172.21.15.150:0/3513522635 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5571379a8270 con 0x557137b30ec0 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8a1ffb700 1 -- 172.21.15.150:0/3513522635 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8b00052b0 con 0x557137b30ec0 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8d05ebb80 10 client.26024 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8d05ebb80 10 client.26024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8d05ebb80 20 client.26024 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/3513522635 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557137b4db00 0x557137b4ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8d05ebb80 4 client.26024 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.659+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/3513522635 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557137afe780 con 0x557137b4db00 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.660+0000 7fc8be043700 1 --2- 172.21.15.150:0/3513522635 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557137b4db00 0x557137b4ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.660+0000 7fc8a1ffb700 10 client.26024 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:59.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.660+0000 7fc8be043700 1 --2- 172.21.15.150:0/3513522635 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557137b4db00 0x557137b4ff50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=713a47f60dad6b7f server_cookie=968021a19241c31a in_seq=0 out_seq=0 2024-05-10T04:06:59.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8a1ffb700 1 -- 172.21.15.150:0/3513522635 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557137afe780 con 0x557137b4db00 2024-05-10T04:06:59.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8a1ffb700 10 client.26024 handle_command_reply: tid=0 2024-05-10T04:06:59.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8d05ebb80 1 client.26024 shutdown 2024-05-10T04:06:59.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/3513522635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b30ec0 msgr2=0x557137b49390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/3513522635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b30ec0 0x557137b49390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0007b30 tx=0x7fc8b0003cb0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/3513522635 shutdown_connections 2024-05-10T04:06:59.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/3513522635 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557137b4db00 0x557137b4ff50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/3513522635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b30ec0 0x557137b49390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/3513522635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b2f6a0 0x557137b41ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/3513522635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30570 0x557137b42430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/3513522635 >> 172.21.15.150:0/3513522635 conn(0x557137a91a60 msgr2=0x557137b29fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:59.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8a1ffb700 0 client.26024 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:59.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/3513522635 shutdown_connections 2024-05-10T04:06:59.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/3513522635 wait complete. 2024-05-10T04:06:59.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.661+0000 7fc8d05ebb80 20 client.26024 trim_cache size 0 max 16384 2024-05-10T04:06:59.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.678+0000 7fc8d05ebb80 1 Processor -- start 2024-05-10T04:06:59.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8d05ebb80 1 -- start start 2024-05-10T04:06:59.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8d05ebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b2f6a0 0x557137b41460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8d05ebb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30570 0x557137b49350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8d05ebb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 0x557137b337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8d05ebb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557137b312d0 con 0x557137b30ec0 2024-05-10T04:06:59.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8d05ebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557137b42db0 con 0x557137b2f6a0 2024-05-10T04:06:59.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8d05ebb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557137b42f90 con 0x557137b30570 2024-05-10T04:06:59.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8be844700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 0x557137b337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8be844700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 0x557137b337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38022/0 (socket says 172.21.15.150:38022) 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8be844700 1 -- 172.21.15.150:0/3797912678 learned_addr learned my addr 172.21.15.150:0/3797912678 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8be043700 1 --2- 172.21.15.150:0/3797912678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b2f6a0 0x557137b41460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8bd842700 1 --2- 172.21.15.150:0/3797912678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30570 0x557137b49350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8be844700 1 -- 172.21.15.150:0/3797912678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30570 msgr2=0x557137b49350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8be844700 1 --2- 172.21.15.150:0/3797912678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30570 0x557137b49350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8be844700 1 -- 172.21.15.150:0/3797912678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b2f6a0 msgr2=0x557137b41460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8be844700 1 --2- 172.21.15.150:0/3797912678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b2f6a0 0x557137b41460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8be844700 1 -- 172.21.15.150:0/3797912678 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557137a15560 con 0x557137b30ec0 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.679+0000 7fc8be043700 1 --2- 172.21.15.150:0/3797912678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b2f6a0 0x557137b41460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.680+0000 7fc8be844700 1 --2- 172.21.15.150:0/3797912678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 0x557137b337a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009c30 tx=0x7fc8b0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e386135660aa5460 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.680+0000 7fc8ad580700 1 -- 172.21.15.150:0/3797912678 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0018070 con 0x557137b30ec0 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.680+0000 7fc8ad580700 1 -- 172.21.15.150:0/3797912678 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8b0019040 con 0x557137b30ec0 2024-05-10T04:06:59.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.680+0000 7fc8ad580700 1 -- 172.21.15.150:0/3797912678 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00040e0 con 0x557137b30ec0 2024-05-10T04:06:59.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.681+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/3797912678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 msgr2=0x557137b337a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.681+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/3797912678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 0x557137b337a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009c30 tx=0x7fc8b0000f40 comp rx=0 tx=0).stop 2024-05-10T04:06:59.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.681+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/3797912678 shutdown_connections 2024-05-10T04:06:59.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.681+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/3797912678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 0x557137b337a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.681+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/3797912678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b2f6a0 0x557137b41460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.681+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/3797912678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b30570 0x557137b49350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.681+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/3797912678 >> 172.21.15.150:0/3797912678 conn(0x557137a91a60 msgr2=0x557137b2ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.681+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/3797912678 shutdown_connections 2024-05-10T04:06:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.681+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/3797912678 wait complete. 2024-05-10T04:06:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.681+0000 7fc8d05ebb80 1 -- start start 2024-05-10T04:06:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.681+0000 7fc8d05ebb80 1 Processor -- start 2024-05-10T04:06:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8d05ebb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b2f6a0 0x557137b46740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8d05ebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b30570 0x557137b46c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8d05ebb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 0x557137b47bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8d05ebb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557137b480f0 con 0x557137b30ec0 2024-05-10T04:06:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8d05ebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557137b48260 con 0x557137b30570 2024-05-10T04:06:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8d05ebb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557137b48420 con 0x557137b2f6a0 2024-05-10T04:06:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8d05ebb80 10 client.0 _ll_register_callbacks cb 0x5571379f4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:06:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8be844700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 0x557137b47bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8bd842700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b30570 0x557137b46c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8be844700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 0x557137b47bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38024/0 (socket says 172.21.15.150:38024) 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8be844700 1 -- 172.21.15.150:0/992084874 learned_addr learned my addr 172.21.15.150:0/992084874 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8bd842700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b30570 0x557137b46c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:56624/0 (socket says 172.21.15.150:56624) 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8be043700 1 --2- 172.21.15.150:0/992084874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b2f6a0 0x557137b46740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8be844700 1 -- 172.21.15.150:0/992084874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b2f6a0 msgr2=0x557137b46740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8be844700 1 --2- 172.21.15.150:0/992084874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b2f6a0 0x557137b46740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8be844700 1 -- 172.21.15.150:0/992084874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b30570 msgr2=0x557137b46c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8be844700 1 --2- 172.21.15.150:0/992084874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b30570 0x557137b46c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8be844700 1 -- 172.21.15.150:0/992084874 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b810cbc0 con 0x557137b30ec0 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8be844700 1 --2- 172.21.15.150:0/992084874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 0x557137b47bb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0007b30 tx=0x7fc8b0009010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95318de07bef48f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8a1ffb700 1 -- 172.21.15.150:0/992084874 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0018040 con 0x557137b30ec0 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8a1ffb700 1 -- 172.21.15.150:0/992084874 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc8b0004050 con 0x557137b30ec0 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.682+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/992084874 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8b000f050 con 0x557137b30ec0 2024-05-10T04:06:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.683+0000 7fc8a1ffb700 1 -- 172.21.15.150:0/992084874 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8b0004210 con 0x557137b30ec0 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.683+0000 7fc8d05ebb80 10 client.16401 fetch_fsmap learned FSMap version 25 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.683+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/992084874 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55713787cb60 con 0x557137b30ec0 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.683+0000 7fc8a1ffb700 1 -- 172.21.15.150:0/992084874 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8b000f050 con 0x557137b30ec0 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.683+0000 7fc8a1ffb700 1 -- 172.21.15.150:0/992084874 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8b001e8e0 con 0x557137b30ec0 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.683+0000 7fc8d05ebb80 10 client.16401 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.683+0000 7fc8d05ebb80 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.683+0000 7fc8d05ebb80 20 client.16401 populate_metadata read hostname 'smithi150' 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.683+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/992084874 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557137ba3fd0 0x557137ba63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.683+0000 7fc8d05ebb80 4 client.16401 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.683+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/992084874 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557137a47ec0 con 0x557137ba3fd0 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.683+0000 7fc8be043700 1 --2- 172.21.15.150:0/992084874 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557137ba3fd0 0x557137ba63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8a1ffb700 10 client.16401 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8be043700 1 --2- 172.21.15.150:0/992084874 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557137ba3fd0 0x557137ba63f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd6fa71f84ced0d2 server_cookie=99e18f76679f8524 in_seq=0 out_seq=0 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8a1ffb700 1 -- 172.21.15.150:0/992084874 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557137a47ec0 con 0x557137ba3fd0 2024-05-10T04:06:59.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8a1ffb700 10 client.16401 handle_command_reply: tid=0 2024-05-10T04:06:59.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8d05ebb80 1 client.16401 shutdown 2024-05-10T04:06:59.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/992084874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 msgr2=0x557137b47bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/992084874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 0x557137b47bb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0007b30 tx=0x7fc8b0009010 comp rx=0 tx=0).stop 2024-05-10T04:06:59.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/992084874 shutdown_connections 2024-05-10T04:06:59.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/992084874 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x557137ba3fd0 0x557137ba63f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/992084874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557137b30ec0 0x557137b47bb0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/992084874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557137b2f6a0 0x557137b46740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8d05ebb80 1 --2- 172.21.15.150:0/992084874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557137b30570 0x557137b46c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8a1ffb700 0 client.16401 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:06:59.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/992084874 >> 172.21.15.150:0/992084874 conn(0x557137a91a60 msgr2=0x557137b2ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:59.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.684+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/992084874 shutdown_connections 2024-05-10T04:06:59.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8d05ebb80 1 -- 172.21.15.150:0/992084874 wait complete. 2024-05-10T04:06:59.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8d05ebb80 20 client.16401 trim_cache size 0 max 16384 2024-05-10T04:06:59.694 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:06:59.694 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T04:06:59.694 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": { 2024-05-10T04:06:59.694 INFO:teuthology.orchestra.run.smithi150.stdout: "5fcedd15-813d-4c71-baf9-e9280e1a12d4": { 2024-05-10T04:06:59.694 INFO:teuthology.orchestra.run.smithi150.stdout: "path": "/", 2024-05-10T04:06:59.694 INFO:teuthology.orchestra.run.smithi150.stdout: "tag": "5fcedd15-813d-4c71-baf9-e9280e1a12d4", 2024-05-10T04:06:59.694 INFO:teuthology.orchestra.run.smithi150.stdout: "options": "recursive,force" 2024-05-10T04:06:59.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:06:59.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:06:59.694 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:06:59.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8c02a7700 1 -- 172.21.15.150:0/1069488225 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc8a40a11c0 msgr2=0x7fc8a40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8c02a7700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc8a40a11c0 0x7fc8a40a36d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4006010 tx=0x7fc8b400b770 comp rx=0 tx=0).stop 2024-05-10T04:06:59.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8c02a7700 1 -- 172.21.15.150:0/1069488225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b80750e0 msgr2=0x7fc8b819a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:06:59.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8c02a7700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b80750e0 0x7fc8b819a760 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800e970 tx=0x7fc8a800ed30 comp rx=0 tx=0).stop 2024-05-10T04:06:59.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8c02a7700 1 -- 172.21.15.150:0/1069488225 shutdown_connections 2024-05-10T04:06:59.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8c02a7700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc8a40a11c0 0x7fc8a40a36d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8c02a7700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b8109ad0 0x7fc8b81a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8c02a7700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b8073ae0 0x7fc8b819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8c02a7700 1 --2- 172.21.15.150:0/1069488225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b80750e0 0x7fc8b819a760 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:06:59.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8c02a7700 1 -- 172.21.15.150:0/1069488225 >> 172.21.15.150:0/1069488225 conn(0x7fc8b80fd1e0 msgr2=0x7fc8b8105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:06:59.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8c02a7700 1 -- 172.21.15.150:0/1069488225 shutdown_connections 2024-05-10T04:06:59.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:06:59.685+0000 7fc8c02a7700 1 -- 172.21.15.150:0/1069488225 wait complete. 2024-05-10T04:06:59.700 INFO:tasks.cephfs.filesystem:scrub status for tag:5fcedd15-813d-4c71-baf9-e9280e1a12d4 - {'path': '/', 'tag': '5fcedd15-813d-4c71-baf9-e9280e1a12d4', 'options': 'recursive,force'} 2024-05-10T04:07:00.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:00 smithi150 ceph-mon[42477]: pgmap v362: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-10T04:07:00.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:00 smithi150 ceph-mon[42477]: scrub queued for path: / 2024-05-10T04:07:00.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:00 smithi150 ceph-mon[42477]: scrub summary: idle+waiting paths [/] 2024-05-10T04:07:00.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:00 smithi150 ceph-mon[42477]: scrub summary: active paths [/] 2024-05-10T04:07:00.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:00 smithi150 ceph-mon[42477]: scrub summary: idle+waiting paths [/] 2024-05-10T04:07:00.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:00 smithi181 ceph-mon[46602]: pgmap v362: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-10T04:07:00.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:00 smithi181 ceph-mon[46602]: scrub queued for path: / 2024-05-10T04:07:00.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:00 smithi181 ceph-mon[46602]: scrub summary: idle+waiting paths [/] 2024-05-10T04:07:00.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:00 smithi181 ceph-mon[46602]: scrub summary: active paths [/] 2024-05-10T04:07:00.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:00 smithi181 ceph-mon[46602]: scrub summary: idle+waiting paths [/] 2024-05-10T04:07:00.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:00 smithi161 ceph-mon[46918]: pgmap v362: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-10T04:07:00.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:00 smithi161 ceph-mon[46918]: scrub queued for path: / 2024-05-10T04:07:00.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:00 smithi161 ceph-mon[46918]: scrub summary: idle+waiting paths [/] 2024-05-10T04:07:00.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:00 smithi161 ceph-mon[46918]: scrub summary: active paths [/] 2024-05-10T04:07:00.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:00 smithi161 ceph-mon[46918]: scrub summary: idle+waiting paths [/] 2024-05-10T04:07:01.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:01 smithi150 ceph-mon[42477]: scrub summary: idle 2024-05-10T04:07:01.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:01 smithi181 ceph-mon[46602]: scrub summary: idle 2024-05-10T04:07:01.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:01 smithi161 ceph-mon[46918]: scrub summary: idle 2024-05-10T04:07:02.171 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:07:02.171 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:07:02.171 INFO:tasks.workunit.client.0.smithi150.stdout: 2024-05-10T04:07:02.171 INFO:tasks.workunit.client.0.smithi150.stdout:iozone test complete. 2024-05-10T04:07:02.172 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-10T04:07:02.172 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-10T04:07:02.209 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-05-10T04:07:02.209 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-10T04:07:02.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:02 smithi150 ceph-mon[42477]: pgmap v363: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-10T04:07:02.537 DEBUG:teuthology.parallel:result is None 2024-05-10T04:07:02.537 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-10T04:07:02.571 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-10T04:07:02.571 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-10T04:07:02.583 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:07:02.620 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-10T04:07:02.632 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-10T04:07:02.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:02 smithi181 ceph-mon[46602]: pgmap v363: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-10T04:07:02.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:02 smithi161 ceph-mon[46918]: pgmap v363: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-10T04:07:03.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:03 smithi150 ceph-mon[42477]: pgmap v364: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-10T04:07:03.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:03 smithi181 ceph-mon[46602]: pgmap v364: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-10T04:07:03.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:03 smithi161 ceph-mon[46918]: pgmap v364: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-10T04:07:06.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:06 smithi150 ceph-mon[42477]: pgmap v365: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-05-10T04:07:06.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:06 smithi181 ceph-mon[46602]: pgmap v365: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-05-10T04:07:06.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:06 smithi161 ceph-mon[46918]: pgmap v365: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-05-10T04:07:08.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:08 smithi150 ceph-mon[42477]: pgmap v366: 129 pgs: 129 active+clean; 6.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 76 op/s 2024-05-10T04:07:08.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:08 smithi181 ceph-mon[46602]: pgmap v366: 129 pgs: 129 active+clean; 6.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 76 op/s 2024-05-10T04:07:08.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:08 smithi161 ceph-mon[46918]: pgmap v366: 129 pgs: 129 active+clean; 6.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 76 op/s 2024-05-10T04:07:10.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:10 smithi150 ceph-mon[42477]: pgmap v367: 129 pgs: 129 active+clean; 157 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 217 op/s 2024-05-10T04:07:10.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:10 smithi181 ceph-mon[46602]: pgmap v367: 129 pgs: 129 active+clean; 157 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 217 op/s 2024-05-10T04:07:10.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:10 smithi161 ceph-mon[46918]: pgmap v367: 129 pgs: 129 active+clean; 157 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 217 op/s 2024-05-10T04:07:12.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:12 smithi150 ceph-mon[42477]: pgmap v368: 129 pgs: 129 active+clean; 21 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 221 op/s 2024-05-10T04:07:12.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:12 smithi181 ceph-mon[46602]: pgmap v368: 129 pgs: 129 active+clean; 21 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 221 op/s 2024-05-10T04:07:12.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:12 smithi161 ceph-mon[46918]: pgmap v368: 129 pgs: 129 active+clean; 21 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 221 op/s 2024-05-10T04:07:14.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:14 smithi150 ceph-mon[42477]: pgmap v369: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 222 op/s 2024-05-10T04:07:14.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:14 smithi181 ceph-mon[46602]: pgmap v369: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 222 op/s 2024-05-10T04:07:14.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:14 smithi161 ceph-mon[46918]: pgmap v369: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 222 op/s 2024-05-10T04:07:16.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:16 smithi150 ceph-mon[42477]: pgmap v370: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.2 KiB/s wr, 222 op/s 2024-05-10T04:07:16.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:16 smithi181 ceph-mon[46602]: pgmap v370: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.2 KiB/s wr, 222 op/s 2024-05-10T04:07:16.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:16 smithi161 ceph-mon[46918]: pgmap v370: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.2 KiB/s wr, 222 op/s 2024-05-10T04:07:18.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:18 smithi150 ceph-mon[42477]: pgmap v371: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 221 op/s 2024-05-10T04:07:18.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:18 smithi181 ceph-mon[46602]: pgmap v371: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 221 op/s 2024-05-10T04:07:18.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:18 smithi161 ceph-mon[46918]: pgmap v371: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 221 op/s 2024-05-10T04:07:20.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:20 smithi150 ceph-mon[42477]: pgmap v372: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 152 op/s 2024-05-10T04:07:20.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:20 smithi181 ceph-mon[46602]: pgmap v372: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 152 op/s 2024-05-10T04:07:20.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:20 smithi161 ceph-mon[46918]: pgmap v372: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 152 op/s 2024-05-10T04:07:22.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:22 smithi150 ceph-mon[42477]: pgmap v373: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-10T04:07:22.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:22 smithi181 ceph-mon[46602]: pgmap v373: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-10T04:07:22.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:22 smithi161 ceph-mon[46918]: pgmap v373: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-10T04:07:24.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:24 smithi150 ceph-mon[42477]: pgmap v374: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T04:07:24.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:24 smithi181 ceph-mon[46602]: pgmap v374: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T04:07:24.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:24 smithi161 ceph-mon[46918]: pgmap v374: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T04:07:26.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:26 smithi150 ceph-mon[42477]: pgmap v375: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-10T04:07:26.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:26 smithi181 ceph-mon[46602]: pgmap v375: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-10T04:07:26.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:26 smithi161 ceph-mon[46918]: pgmap v375: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-10T04:07:28.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:28 smithi150 ceph-mon[42477]: pgmap v376: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:28.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:28 smithi181 ceph-mon[46602]: pgmap v376: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:28.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:28 smithi161 ceph-mon[46918]: pgmap v376: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:29.702 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:07:29.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.848+0000 7fba1fbfc700 1 -- 172.21.15.150:0/3846381715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba18101be0 msgr2=0x7fba18101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:29.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.848+0000 7fba1fbfc700 1 --2- 172.21.15.150:0/3846381715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba18101be0 0x7fba18101ff0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fba14009990 tx=0x7fba14009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.848+0000 7fba1fbfc700 1 -- 172.21.15.150:0/3846381715 shutdown_connections 2024-05-10T04:07:29.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.848+0000 7fba1fbfc700 1 --2- 172.21.15.150:0/3846381715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba18102580 0x7fba18108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.848+0000 7fba1fbfc700 1 --2- 172.21.15.150:0/3846381715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18100ac0 0x7fba18100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.848+0000 7fba1fbfc700 1 --2- 172.21.15.150:0/3846381715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba18101be0 0x7fba18101ff0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.848+0000 7fba1fbfc700 1 -- 172.21.15.150:0/3846381715 >> 172.21.15.150:0/3846381715 conn(0x7fba180fc0d0 msgr2=0x7fba180fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:29.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.848+0000 7fba1fbfc700 1 -- 172.21.15.150:0/3846381715 shutdown_connections 2024-05-10T04:07:29.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.848+0000 7fba1fbfc700 1 -- 172.21.15.150:0/3846381715 wait complete. 2024-05-10T04:07:29.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.849+0000 7fba1fbfc700 1 Processor -- start 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.849+0000 7fba1fbfc700 1 -- start start 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.849+0000 7fba1fbfc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba18100ac0 0x7fba18197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.849+0000 7fba1fbfc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba18101be0 0x7fba18198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.849+0000 7fba1fbfc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18102580 0x7fba1819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.849+0000 7fba1fbfc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1819f9a0 con 0x7fba18100ac0 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.849+0000 7fba1fbfc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1819fb10 con 0x7fba18102580 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.849+0000 7fba1fbfc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1819fcd0 con 0x7fba18101be0 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.849+0000 7fba1d998700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba18100ac0 0x7fba18197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.849+0000 7fba1d998700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba18100ac0 0x7fba18197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48410/0 (socket says 172.21.15.150:48410) 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.849+0000 7fba1d998700 1 -- 172.21.15.150:0/2057234175 learned_addr learned my addr 172.21.15.150:0/2057234175 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba1d197700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba18101be0 0x7fba18198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba1e199700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18102580 0x7fba1819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba1d998700 1 -- 172.21.15.150:0/2057234175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba18101be0 msgr2=0x7fba18198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:29.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba1d998700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba18101be0 0x7fba18198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba1d998700 1 -- 172.21.15.150:0/2057234175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18102580 msgr2=0x7fba1819f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:29.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba1d998700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18102580 0x7fba1819f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba1d998700 1 -- 172.21.15.150:0/2057234175 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba140095f0 con 0x7fba18100ac0 2024-05-10T04:07:29.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba1e199700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18102580 0x7fba1819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:07:29.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba1d197700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba18101be0 0x7fba18198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:07:29.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba1d998700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba18100ac0 0x7fba18197ec0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fba0800e940 tx=0x7fba0800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:29.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba0effd700 1 -- 172.21.15.150:0/2057234175 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0800cbd0 con 0x7fba18100ac0 2024-05-10T04:07:29.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba0effd700 1 -- 172.21.15.150:0/2057234175 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba080044e0 con 0x7fba18100ac0 2024-05-10T04:07:29.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba1fbfc700 1 -- 172.21.15.150:0/2057234175 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1819ffb0 con 0x7fba18100ac0 2024-05-10T04:07:29.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba0effd700 1 -- 172.21.15.150:0/2057234175 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba08003dc0 con 0x7fba18100ac0 2024-05-10T04:07:29.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.850+0000 7fba1fbfc700 1 -- 172.21.15.150:0/2057234175 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba181a03e0 con 0x7fba18100ac0 2024-05-10T04:07:29.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.852+0000 7fba0effd700 1 -- 172.21.15.150:0/2057234175 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fba08003f20 con 0x7fba18100ac0 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.853+0000 7fba0effd700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fba040a11c0 0x7fba040a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.853+0000 7fba1d197700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fba040a11c0 0x7fba040a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.853+0000 7fba0effd700 1 -- 172.21.15.150:0/2057234175 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fba08014070 con 0x7fba18100ac0 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.853+0000 7fba1d197700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fba040a11c0 0x7fba040a36d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fba1400b2e0 tx=0x7fba14005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.854+0000 7fba2ff40b80 1 Processor -- start 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.854+0000 7fba2ff40b80 1 -- start start 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.854+0000 7fba2ff40b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c66a0 0x5602ea0c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.854+0000 7fba2ff40b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c7570 0x5602ea0c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.854+0000 7fba2ff40b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.854+0000 7fba2ff40b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e9fed150 con 0x5602ea0c7ec0 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.854+0000 7fba2ff40b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e9feab10 con 0x5602ea0c7570 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.854+0000 7fba2ff40b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e9fe6400 con 0x5602ea0c66a0 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.854+0000 7fba1e199700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.854+0000 7fba1e199700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48424/0 (socket says 172.21.15.150:48424) 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.854+0000 7fba1e199700 1 -- 172.21.15.150:0/2947198210 learned_addr learned my addr 172.21.15.150:0/2947198210 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.854+0000 7fba1d998700 1 --2- 172.21.15.150:0/2947198210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c66a0 0x5602ea0c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.855+0000 7fba1d197700 1 --2- 172.21.15.150:0/2947198210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c7570 0x5602ea0c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.855+0000 7fba1e199700 1 -- 172.21.15.150:0/2947198210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c66a0 msgr2=0x5602ea0c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.855+0000 7fba1e199700 1 --2- 172.21.15.150:0/2947198210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c66a0 0x5602ea0c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.855+0000 7fba1e199700 1 -- 172.21.15.150:0/2947198210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c7570 msgr2=0x5602ea0c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.855+0000 7fba1e199700 1 --2- 172.21.15.150:0/2947198210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c7570 0x5602ea0c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.855+0000 7fba1e199700 1 -- 172.21.15.150:0/2947198210 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602e9e673b0 con 0x5602ea0c7ec0 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.855+0000 7fba1e199700 1 --2- 172.21.15.150:0/2947198210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0ce7c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fba10009cb0 tx=0x7fba10007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac227198e218defc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.855+0000 7fba03fff700 1 -- 172.21.15.150:0/2947198210 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10014400 con 0x5602ea0c7ec0 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.855+0000 7fba03fff700 1 -- 172.21.15.150:0/2947198210 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba10014a20 con 0x5602ea0c7ec0 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.855+0000 7fba03fff700 1 -- 172.21.15.150:0/2947198210 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10013a00 con 0x5602ea0c7ec0 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.856+0000 7fba2ff40b80 1 -- 172.21.15.150:0/2947198210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 msgr2=0x5602ea0ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.856+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/2947198210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0ce7c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fba10009cb0 tx=0x7fba10007500 comp rx=0 tx=0).stop 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.856+0000 7fba2ff40b80 1 -- 172.21.15.150:0/2947198210 shutdown_connections 2024-05-10T04:07:29.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.856+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/2947198210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0ce7c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.856+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/2947198210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c66a0 0x5602ea0c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.856+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/2947198210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c7570 0x5602ea0c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.856+0000 7fba2ff40b80 1 -- 172.21.15.150:0/2947198210 >> 172.21.15.150:0/2947198210 conn(0x5602ea028a60 msgr2=0x5602ea0c0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.856+0000 7fba2ff40b80 1 -- 172.21.15.150:0/2947198210 shutdown_connections 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.856+0000 7fba2ff40b80 1 -- 172.21.15.150:0/2947198210 wait complete. 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.856+0000 7fba2ff40b80 1 -- start start 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.857+0000 7fba2ff40b80 1 Processor -- start 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.857+0000 7fba2ff40b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c66a0 0x5602ea0d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.857+0000 7fba2ff40b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c7570 0x5602ea0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.857+0000 7fba2ff40b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.857+0000 7fba2ff40b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e9f82560 con 0x5602ea0c7ec0 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.857+0000 7fba2ff40b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e9f82200 con 0x5602ea0c66a0 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.857+0000 7fba2ff40b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e9e696a0 con 0x5602ea0c7570 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.857+0000 7fba2ff40b80 10 client.0 _ll_register_callbacks cb 0x5602e9f8bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.857+0000 7fba1e199700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.857+0000 7fba1e199700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0d7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48436/0 (socket says 172.21.15.150:48436) 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.857+0000 7fba1e199700 1 -- 172.21.15.150:0/1134869000 learned_addr learned my addr 172.21.15.150:0/1134869000 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.857+0000 7fba1d998700 1 --2- 172.21.15.150:0/1134869000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c66a0 0x5602ea0d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.857+0000 7fba1d197700 1 --2- 172.21.15.150:0/1134869000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c7570 0x5602ea0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba1e199700 1 -- 172.21.15.150:0/1134869000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c7570 msgr2=0x5602ea0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba1e199700 1 --2- 172.21.15.150:0/1134869000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c7570 0x5602ea0d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba1e199700 1 -- 172.21.15.150:0/1134869000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c66a0 msgr2=0x5602ea0d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba1e199700 1 --2- 172.21.15.150:0/1134869000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c66a0 0x5602ea0d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba1e199700 1 -- 172.21.15.150:0/1134869000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba100098e0 con 0x5602ea0c7ec0 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba1e199700 1 --2- 172.21.15.150:0/1134869000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0d7b40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fba10008fd0 tx=0x7fba10004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1477662b03cbe013 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba017fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba017fa700 1 -- 172.21.15.150:0/1134869000 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10014600 con 0x5602ea0c7ec0 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba017fa700 1 -- 172.21.15.150:0/1134869000 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fba10014760 con 0x5602ea0c7ec0 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba2ff40b80 1 -- 172.21.15.150:0/1134869000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602e9f3f270 con 0x5602ea0c7ec0 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba017fa700 1 -- 172.21.15.150:0/1134869000 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba10013740 con 0x5602ea0c7ec0 2024-05-10T04:07:29.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba2ff40b80 10 client.16413 fetch_fsmap learned FSMap version 25 2024-05-10T04:07:29.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba2ff40b80 1 -- 172.21.15.150:0/1134869000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5602e9e13b60 con 0x5602ea0c7ec0 2024-05-10T04:07:29.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba017fa700 1 -- 172.21.15.150:0/1134869000 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5602e9f3f270 con 0x5602ea0c7ec0 2024-05-10T04:07:29.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.858+0000 7fba017fa700 1 -- 172.21.15.150:0/1134869000 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fba10005460 con 0x5602ea0c7ec0 2024-05-10T04:07:29.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.859+0000 7fba2ff40b80 10 client.16413 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:07:29.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.859+0000 7fba2ff40b80 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:07:29.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.859+0000 7fba2ff40b80 20 client.16413 populate_metadata read hostname 'smithi150' 2024-05-10T04:07:29.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.859+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/1134869000 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602ea0dc210 0x5602ea0de660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.859+0000 7fba2ff40b80 4 client.16413 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:07:29.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.859+0000 7fba2ff40b80 1 -- 172.21.15.150:0/1134869000 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602ea095780 con 0x5602ea0dc210 2024-05-10T04:07:29.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.859+0000 7fba1d998700 1 --2- 172.21.15.150:0/1134869000 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602ea0dc210 0x5602ea0de660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.859+0000 7fba017fa700 10 client.16413 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:07:29.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.859+0000 7fba1d998700 1 --2- 172.21.15.150:0/1134869000 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602ea0dc210 0x5602ea0de660 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60f23eaefc1b15a0 server_cookie=8114924956cff40c in_seq=0 out_seq=0 2024-05-10T04:07:29.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba017fa700 1 -- 172.21.15.150:0/1134869000 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5602ea095780 con 0x5602ea0dc210 2024-05-10T04:07:29.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba017fa700 10 client.16413 handle_command_reply: tid=0 2024-05-10T04:07:29.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba2ff40b80 1 client.16413 shutdown 2024-05-10T04:07:29.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba2ff40b80 1 -- 172.21.15.150:0/1134869000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 msgr2=0x5602ea0d7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:29.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/1134869000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0d7b40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fba10008fd0 tx=0x7fba10004ab0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba2ff40b80 1 -- 172.21.15.150:0/1134869000 shutdown_connections 2024-05-10T04:07:29.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/1134869000 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602ea0dc210 0x5602ea0de660 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/1134869000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0d7b40 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba017fa700 0 client.16413 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:07:29.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/1134869000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c66a0 0x5602ea0d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/1134869000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c7570 0x5602ea0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba2ff40b80 1 -- 172.21.15.150:0/1134869000 >> 172.21.15.150:0/1134869000 conn(0x5602ea028a60 msgr2=0x5602ea0c0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:29.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba2ff40b80 1 -- 172.21.15.150:0/1134869000 shutdown_connections 2024-05-10T04:07:29.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.860+0000 7fba2ff40b80 1 -- 172.21.15.150:0/1134869000 wait complete. 2024-05-10T04:07:29.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.861+0000 7fba2ff40b80 20 client.16413 trim_cache size 0 max 16384 2024-05-10T04:07:29.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba2ff40b80 1 Processor -- start 2024-05-10T04:07:29.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba2ff40b80 1 -- start start 2024-05-10T04:07:29.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba2ff40b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c66a0 0x5602ea0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba2ff40b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c7570 0x5602ea0d7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba2ff40b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba2ff40b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602ea0d1740 con 0x5602ea0c7ec0 2024-05-10T04:07:29.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba2ff40b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602ea0c2770 con 0x5602ea0c66a0 2024-05-10T04:07:29.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba2ff40b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602ea0bb7c0 con 0x5602ea0c7570 2024-05-10T04:07:29.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba1e199700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba1e199700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48448/0 (socket says 172.21.15.150:48448) 2024-05-10T04:07:29.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba1e199700 1 -- 172.21.15.150:0/1226827296 learned_addr learned my addr 172.21.15.150:0/1226827296 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:29.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba1d998700 1 --2- 172.21.15.150:0/1226827296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c66a0 0x5602ea0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba1d197700 1 --2- 172.21.15.150:0/1226827296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c7570 0x5602ea0d7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba1e199700 1 -- 172.21.15.150:0/1226827296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c7570 msgr2=0x5602ea0d7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba1e199700 1 --2- 172.21.15.150:0/1226827296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c7570 0x5602ea0d7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba1e199700 1 -- 172.21.15.150:0/1226827296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c66a0 msgr2=0x5602ea0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba1e199700 1 --2- 172.21.15.150:0/1226827296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c66a0 0x5602ea0cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba1e199700 1 -- 172.21.15.150:0/1226827296 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602e9fac560 con 0x5602ea0c7ec0 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba1d197700 1 --2- 172.21.15.150:0/1226827296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c7570 0x5602ea0d7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba1d998700 1 --2- 172.21.15.150:0/1226827296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c66a0 0x5602ea0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.878+0000 7fba1e199700 1 --2- 172.21.15.150:0/1226827296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0cb350 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fba10006130 tx=0x7fba10013410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dd3e8396f1af76 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.879+0000 7fba0cd7f700 1 -- 172.21.15.150:0/1226827296 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10014600 con 0x5602ea0c7ec0 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.879+0000 7fba0cd7f700 1 -- 172.21.15.150:0/1226827296 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba10004bd0 con 0x5602ea0c7ec0 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.879+0000 7fba0cd7f700 1 -- 172.21.15.150:0/1226827296 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10013880 con 0x5602ea0c7ec0 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.879+0000 7fba2ff40b80 1 -- 172.21.15.150:0/1226827296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 msgr2=0x5602ea0cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.879+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/1226827296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0cb350 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fba10006130 tx=0x7fba10013410 comp rx=0 tx=0).stop 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.879+0000 7fba2ff40b80 1 -- 172.21.15.150:0/1226827296 shutdown_connections 2024-05-10T04:07:29.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.879+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/1226827296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0cb350 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.879+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/1226827296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c66a0 0x5602ea0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.879+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/1226827296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c7570 0x5602ea0d7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.879+0000 7fba2ff40b80 1 -- 172.21.15.150:0/1226827296 >> 172.21.15.150:0/1226827296 conn(0x5602ea028a60 msgr2=0x5602ea0c6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.879+0000 7fba2ff40b80 1 -- 172.21.15.150:0/1226827296 shutdown_connections 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.879+0000 7fba2ff40b80 1 -- 172.21.15.150:0/1226827296 wait complete. 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba2ff40b80 1 -- start start 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba2ff40b80 1 Processor -- start 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba2ff40b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c66a0 0x5602ea0d4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba2ff40b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c7570 0x5602ea0d5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba2ff40b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0d6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba2ff40b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602ea0d6580 con 0x5602ea0c7ec0 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba2ff40b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602ea0d66f0 con 0x5602ea0c7570 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba2ff40b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602ea0d68b0 con 0x5602ea0c66a0 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba2ff40b80 10 client.0 _ll_register_callbacks cb 0x5602e9f8bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:07:29.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba1e199700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0d6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba1d197700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c7570 0x5602ea0d5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba1e199700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0d6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48456/0 (socket says 172.21.15.150:48456) 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba1d197700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c7570 0x5602ea0d5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:39962/0 (socket says 172.21.15.150:39962) 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.880+0000 7fba1e199700 1 -- 172.21.15.150:0/2109911588 learned_addr learned my addr 172.21.15.150:0/2109911588 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba1d998700 1 --2- 172.21.15.150:0/2109911588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c66a0 0x5602ea0d4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba1e199700 1 -- 172.21.15.150:0/2109911588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c66a0 msgr2=0x5602ea0d4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba1e199700 1 --2- 172.21.15.150:0/2109911588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c66a0 0x5602ea0d4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba1e199700 1 -- 172.21.15.150:0/2109911588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c7570 msgr2=0x5602ea0d5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba1e199700 1 --2- 172.21.15.150:0/2109911588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c7570 0x5602ea0d5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba1e199700 1 -- 172.21.15.150:0/2109911588 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602e9e13b60 con 0x5602ea0c7ec0 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba1d197700 1 --2- 172.21.15.150:0/2109911588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c7570 0x5602ea0d5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba1e199700 1 --2- 172.21.15.150:0/2109911588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0d6040 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fba100057f0 tx=0x7fba100146b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ede8bfd0df7f391 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba017fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba017fa700 1 -- 172.21.15.150:0/2109911588 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba100053a0 con 0x5602ea0c7ec0 2024-05-10T04:07:29.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba017fa700 1 -- 172.21.15.150:0/2109911588 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fba10013600 con 0x5602ea0c7ec0 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba2ff40b80 1 -- 172.21.15.150:0/2109911588 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba10014a20 con 0x5602ea0c7ec0 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba017fa700 1 -- 172.21.15.150:0/2109911588 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba100148a0 con 0x5602ea0c7ec0 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba2ff40b80 10 client.16419 fetch_fsmap learned FSMap version 25 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba2ff40b80 1 -- 172.21.15.150:0/2109911588 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5602e9e6bcd0 con 0x5602ea0c7ec0 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.881+0000 7fba017fa700 1 -- 172.21.15.150:0/2109911588 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba10014a20 con 0x5602ea0c7ec0 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba017fa700 1 -- 172.21.15.150:0/2109911588 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fba1001d740 con 0x5602ea0c7ec0 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba2ff40b80 10 client.16419 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba2ff40b80 10 client.16419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba2ff40b80 20 client.16419 populate_metadata read hostname 'smithi150' 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/2109911588 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602ea1326c0 0x5602ea134ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba2ff40b80 4 client.16419 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba2ff40b80 1 -- 172.21.15.150:0/2109911588 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5602e9fdeec0 con 0x5602ea1326c0 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba1d998700 1 --2- 172.21.15.150:0/2109911588 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602ea1326c0 0x5602ea134ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba017fa700 10 client.16419 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:07:29.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba1d998700 1 --2- 172.21.15.150:0/2109911588 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602ea1326c0 0x5602ea134ae0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e93ddc46939d8143 server_cookie=62f390473d3db744 in_seq=0 out_seq=0 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba017fa700 1 -- 172.21.15.150:0/2109911588 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5602e9fdeec0 con 0x5602ea1326c0 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba017fa700 10 client.16419 handle_command_reply: tid=0 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba2ff40b80 1 client.16419 shutdown 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba2ff40b80 1 -- 172.21.15.150:0/2109911588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 msgr2=0x5602ea0d6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.882+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/2109911588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0d6040 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fba100057f0 tx=0x7fba100146b0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba2ff40b80 1 -- 172.21.15.150:0/2109911588 shutdown_connections 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/2109911588 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x5602ea1326c0 0x5602ea134ae0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/2109911588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602ea0c7ec0 0x5602ea0d6040 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba017fa700 0 client.16419 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/2109911588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602ea0c66a0 0x5602ea0d4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba2ff40b80 1 --2- 172.21.15.150:0/2109911588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ea0c7570 0x5602ea0d5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba2ff40b80 1 -- 172.21.15.150:0/2109911588 >> 172.21.15.150:0/2109911588 conn(0x5602ea028a60 msgr2=0x5602ea0c6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba2ff40b80 1 -- 172.21.15.150:0/2109911588 shutdown_connections 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba2ff40b80 1 -- 172.21.15.150:0/2109911588 wait complete. 2024-05-10T04:07:29.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba2ff40b80 20 client.16419 trim_cache size 0 max 16384 2024-05-10T04:07:29.892 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:07:29.892 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "no active scrubs running", 2024-05-10T04:07:29.892 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": {} 2024-05-10T04:07:29.892 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:07:29.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba1fbfc700 1 -- 172.21.15.150:0/2057234175 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fba040a11c0 msgr2=0x7fba040a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:29.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba1fbfc700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fba040a11c0 0x7fba040a36d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fba1400b2e0 tx=0x7fba14005fd0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba1fbfc700 1 -- 172.21.15.150:0/2057234175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba18100ac0 msgr2=0x7fba18197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:29.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba1fbfc700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba18100ac0 0x7fba18197ec0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fba0800e940 tx=0x7fba0800ed00 comp rx=0 tx=0).stop 2024-05-10T04:07:29.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba1fbfc700 1 -- 172.21.15.150:0/2057234175 shutdown_connections 2024-05-10T04:07:29.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba1fbfc700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fba040a11c0 0x7fba040a36d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba1fbfc700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba18102580 0x7fba1819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba1fbfc700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba18100ac0 0x7fba18197ec0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba1fbfc700 1 --2- 172.21.15.150:0/2057234175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba18101be0 0x7fba18198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:29.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba1fbfc700 1 -- 172.21.15.150:0/2057234175 >> 172.21.15.150:0/2057234175 conn(0x7fba180fc0d0 msgr2=0x7fba180fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:29.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba1fbfc700 1 -- 172.21.15.150:0/2057234175 shutdown_connections 2024-05-10T04:07:29.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:29.883+0000 7fba1fbfc700 1 -- 172.21.15.150:0/2057234175 wait complete. 2024-05-10T04:07:29.898 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T04:07:29.898 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:07:30.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.029+0000 7fce61ee9700 1 -- 172.21.15.150:0/2549402180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c102580 msgr2=0x7fce5c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.029+0000 7fce61ee9700 1 --2- 172.21.15.150:0/2549402180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c102580 0x7fce5c108e30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fce5400b0a0 tx=0x7fce5400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.029+0000 7fce61ee9700 1 -- 172.21.15.150:0/2549402180 shutdown_connections 2024-05-10T04:07:30.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.029+0000 7fce61ee9700 1 --2- 172.21.15.150:0/2549402180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c102580 0x7fce5c108e30 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.029+0000 7fce61ee9700 1 --2- 172.21.15.150:0/2549402180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce5c100ac0 0x7fce5c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.029+0000 7fce61ee9700 1 --2- 172.21.15.150:0/2549402180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5c101be0 0x7fce5c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.029+0000 7fce61ee9700 1 -- 172.21.15.150:0/2549402180 >> 172.21.15.150:0/2549402180 conn(0x7fce5c0fc0d0 msgr2=0x7fce5c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.029+0000 7fce61ee9700 1 -- 172.21.15.150:0/2549402180 shutdown_connections 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.029+0000 7fce61ee9700 1 -- 172.21.15.150:0/2549402180 wait complete. 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.030+0000 7fce61ee9700 1 Processor -- start 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.030+0000 7fce61ee9700 1 -- start start 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.030+0000 7fce61ee9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce5c100ac0 0x7fce5c068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.030+0000 7fce61ee9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5c101be0 0x7fce5c19f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.030+0000 7fce61ee9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c102580 0x7fce5c1a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.030+0000 7fce61ee9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c1a1d40 con 0x7fce5c102580 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.030+0000 7fce61ee9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c1a1eb0 con 0x7fce5c100ac0 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.030+0000 7fce61ee9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c1a2070 con 0x7fce5c101be0 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce5bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c102580 0x7fce5c1a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce5bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c102580 0x7fce5c1a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48468/0 (socket says 172.21.15.150:48468) 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce5bfff700 1 -- 172.21.15.150:0/2388881109 learned_addr learned my addr 172.21.15.150:0/2388881109 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce5b7fe700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce5c100ac0 0x7fce5c068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce5affd700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5c101be0 0x7fce5c19f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce5bfff700 1 -- 172.21.15.150:0/2388881109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5c101be0 msgr2=0x7fce5c19f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce5bfff700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5c101be0 0x7fce5c19f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce5bfff700 1 -- 172.21.15.150:0/2388881109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce5c100ac0 msgr2=0x7fce5c068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce5bfff700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce5c100ac0 0x7fce5c068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce5bfff700 1 -- 172.21.15.150:0/2388881109 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce54009d00 con 0x7fce5c102580 2024-05-10T04:07:30.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce5affd700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5c101be0 0x7fce5c19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:07:30.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce5bfff700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c102580 0x7fce5c1a1770 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fce54006070 tx=0x7fce54009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce58ff9700 1 -- 172.21.15.150:0/2388881109 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5400e040 con 0x7fce5c102580 2024-05-10T04:07:30.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce58ff9700 1 -- 172.21.15.150:0/2388881109 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce54004d60 con 0x7fce5c102580 2024-05-10T04:07:30.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce61ee9700 1 -- 172.21.15.150:0/2388881109 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5c1a22f0 con 0x7fce5c102580 2024-05-10T04:07:30.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce58ff9700 1 -- 172.21.15.150:0/2388881109 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5401d480 con 0x7fce5c102580 2024-05-10T04:07:30.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.031+0000 7fce61ee9700 1 -- 172.21.15.150:0/2388881109 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c1a2770 con 0x7fce5c102580 2024-05-10T04:07:30.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.032+0000 7fce61ee9700 1 -- 172.21.15.150:0/2388881109 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5c107cd0 con 0x7fce5c102580 2024-05-10T04:07:30.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.033+0000 7fce58ff9700 1 -- 172.21.15.150:0/2388881109 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fce54003a00 con 0x7fce5c102580 2024-05-10T04:07:30.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.034+0000 7fce58ff9700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fce440a1120 0x7fce440a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.034+0000 7fce58ff9700 1 -- 172.21.15.150:0/2388881109 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fce540ce630 con 0x7fce5c102580 2024-05-10T04:07:30.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.034+0000 7fce5b7fe700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fce440a1120 0x7fce440a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.034+0000 7fce5b7fe700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fce440a1120 0x7fce440a3630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fce5c101ae0 tx=0x7fce4c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.038+0000 7fce58ff9700 1 -- 172.21.15.150:0/2388881109 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fce54097b40 con 0x7fce5c102580 2024-05-10T04:07:30.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.280+0000 7fce61ee9700 1 -- 172.21.15.150:0/2388881109 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce5c0694d0 con 0x7fce5c102580 2024-05-10T04:07:30.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.281+0000 7fce58ff9700 1 -- 172.21.15.150:0/2388881109 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10886 (secure 0 0 0) 0x7fce5409b160 con 0x7fce5c102580 2024-05-10T04:07:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:07:30.284 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:04:04.157819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:07:30.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.283+0000 7fce61ee9700 1 -- 172.21.15.150:0/2388881109 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fce440a1120 msgr2=0x7fce440a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.283+0000 7fce61ee9700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fce440a1120 0x7fce440a3630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fce5c101ae0 tx=0x7fce4c00b410 comp rx=0 tx=0).stop 2024-05-10T04:07:30.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.284+0000 7fce61ee9700 1 -- 172.21.15.150:0/2388881109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c102580 msgr2=0x7fce5c1a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.284+0000 7fce61ee9700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c102580 0x7fce5c1a1770 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fce54006070 tx=0x7fce54009440 comp rx=0 tx=0).stop 2024-05-10T04:07:30.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.284+0000 7fce61ee9700 1 -- 172.21.15.150:0/2388881109 shutdown_connections 2024-05-10T04:07:30.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.284+0000 7fce61ee9700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fce440a1120 0x7fce440a3630 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.284+0000 7fce61ee9700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c102580 0x7fce5c1a1770 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.284+0000 7fce61ee9700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce5c100ac0 0x7fce5c068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.284+0000 7fce61ee9700 1 --2- 172.21.15.150:0/2388881109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5c101be0 0x7fce5c19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.284+0000 7fce61ee9700 1 -- 172.21.15.150:0/2388881109 >> 172.21.15.150:0/2388881109 conn(0x7fce5c0fc0d0 msgr2=0x7fce5c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.284+0000 7fce61ee9700 1 -- 172.21.15.150:0/2388881109 shutdown_connections 2024-05-10T04:07:30.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.284+0000 7fce61ee9700 1 -- 172.21.15.150:0/2388881109 wait complete. 2024-05-10T04:07:30.287 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 25 2024-05-10T04:07:30.299 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T04:07:30.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.431+0000 7f7570a0b700 1 -- 172.21.15.150:0/2452976628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f756c100ad0 msgr2=0x7f756c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.431+0000 7f7570a0b700 1 --2- 172.21.15.150:0/2452976628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f756c100ad0 0x7f756c100ee0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7560009990 tx=0x7f7560009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.432+0000 7f7570a0b700 1 -- 172.21.15.150:0/2452976628 shutdown_connections 2024-05-10T04:07:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.432+0000 7f7570a0b700 1 --2- 172.21.15.150:0/2452976628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c102590 0x7f756c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.432+0000 7f7570a0b700 1 --2- 172.21.15.150:0/2452976628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f756c100ad0 0x7f756c100ee0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.432+0000 7f7570a0b700 1 --2- 172.21.15.150:0/2452976628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f756c101bf0 0x7f756c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.432+0000 7f7570a0b700 1 -- 172.21.15.150:0/2452976628 >> 172.21.15.150:0/2452976628 conn(0x7f756c0fc0e0 msgr2=0x7f756c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.432+0000 7f7570a0b700 1 -- 172.21.15.150:0/2452976628 shutdown_connections 2024-05-10T04:07:30.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.432+0000 7f7570a0b700 1 -- 172.21.15.150:0/2452976628 wait complete. 2024-05-10T04:07:30.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.432+0000 7f7570a0b700 1 Processor -- start 2024-05-10T04:07:30.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.432+0000 7f7570a0b700 1 -- start start 2024-05-10T04:07:30.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7570a0b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c100ad0 0x7f756c196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7570a0b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f756c101bf0 0x7f756c197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7570a0b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f756c102590 0x7f756c19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7570a0b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c19e9d0 con 0x7f756c101bf0 2024-05-10T04:07:30.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7570a0b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c19eb40 con 0x7f756c102590 2024-05-10T04:07:30.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7570a0b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c19ed00 con 0x7f756c100ad0 2024-05-10T04:07:30.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7569d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f756c101bf0 0x7f756c197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7569d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f756c101bf0 0x7f756c197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48484/0 (socket says 172.21.15.150:48484) 2024-05-10T04:07:30.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7569d9b700 1 -- 172.21.15.150:0/2153381359 learned_addr learned my addr 172.21.15.150:0/2153381359 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f756a59c700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c100ad0 0x7f756c196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f756ad9d700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f756c102590 0x7f756c19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7569d9b700 1 -- 172.21.15.150:0/2153381359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c100ad0 msgr2=0x7f756c196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7569d9b700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c100ad0 0x7f756c196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7569d9b700 1 -- 172.21.15.150:0/2153381359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f756c102590 msgr2=0x7f756c19e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7569d9b700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f756c102590 0x7f756c19e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7569d9b700 1 -- 172.21.15.150:0/2153381359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75600095f0 con 0x7f756c101bf0 2024-05-10T04:07:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f7569d9b700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f756c101bf0 0x7f756c197430 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f755c00c920 tx=0x7f755c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.433+0000 7f755b7fe700 1 -- 172.21.15.150:0/2153381359 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c004a30 con 0x7f756c101bf0 2024-05-10T04:07:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.434+0000 7f755b7fe700 1 -- 172.21.15.150:0/2153381359 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f755c0054e0 con 0x7f756c101bf0 2024-05-10T04:07:30.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.434+0000 7f7570a0b700 1 -- 172.21.15.150:0/2153381359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756c19efe0 con 0x7f756c101bf0 2024-05-10T04:07:30.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.434+0000 7f755b7fe700 1 -- 172.21.15.150:0/2153381359 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c007bd0 con 0x7f756c101bf0 2024-05-10T04:07:30.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.434+0000 7f7570a0b700 1 -- 172.21.15.150:0/2153381359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c19f400 con 0x7f756c101bf0 2024-05-10T04:07:30.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.436+0000 7f755b7fe700 1 -- 172.21.15.150:0/2153381359 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f755c00f450 con 0x7f756c101bf0 2024-05-10T04:07:30.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.437+0000 7f755b7fe700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f75540a11c0 0x7f75540a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.437+0000 7f755b7fe700 1 -- 172.21.15.150:0/2153381359 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f755c0cc7d0 con 0x7f756c101bf0 2024-05-10T04:07:30.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.437+0000 7f756a59c700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f75540a11c0 0x7f75540a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f756a59c700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f75540a11c0 0x7f75540a36d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7560006010 tx=0x7f756001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f7580d4fb80 1 Processor -- start 2024-05-10T04:07:30.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f7580d4fb80 1 -- start start 2024-05-10T04:07:30.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f7580d4fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9b30 0x5557604d82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f7580d4fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9fe0 0x5557604d8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f7580d4fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d8de0 0x5557604e4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f7580d4fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557603ff150 con 0x5557604d9b30 2024-05-10T04:07:30.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f7580d4fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557603fcb10 con 0x5557604d8de0 2024-05-10T04:07:30.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f7580d4fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557603f8400 con 0x5557604d9fe0 2024-05-10T04:07:30.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f756a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9b30 0x5557604d82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f756a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9b30 0x5557604d82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48498/0 (socket says 172.21.15.150:48498) 2024-05-10T04:07:30.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f756a59c700 1 -- 172.21.15.150:0/1543302407 learned_addr learned my addr 172.21.15.150:0/1543302407 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f756ad9d700 1 --2- 172.21.15.150:0/1543302407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d8de0 0x5557604e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f7569d9b700 1 --2- 172.21.15.150:0/1543302407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9fe0 0x5557604d8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.438+0000 7f756a59c700 1 -- 172.21.15.150:0/1543302407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9fe0 msgr2=0x5557604d8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.439+0000 7f756a59c700 1 --2- 172.21.15.150:0/1543302407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9fe0 0x5557604d8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.439+0000 7f756a59c700 1 -- 172.21.15.150:0/1543302407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d8de0 msgr2=0x5557604e4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.439+0000 7f756a59c700 1 --2- 172.21.15.150:0/1543302407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d8de0 0x5557604e4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.439+0000 7f756a59c700 1 -- 172.21.15.150:0/1543302407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557602793b0 con 0x5557604d9b30 2024-05-10T04:07:30.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.439+0000 7f756a59c700 1 --2- 172.21.15.150:0/1543302407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9b30 0x5557604d82c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7560015000 tx=0x7f756001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1422543ae306e05c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.439+0000 7f7558ff9700 1 -- 172.21.15.150:0/1543302407 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560005070 con 0x5557604d9b30 2024-05-10T04:07:30.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.439+0000 7f7558ff9700 1 -- 172.21.15.150:0/1543302407 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f756001e040 con 0x5557604d9b30 2024-05-10T04:07:30.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.439+0000 7f7558ff9700 1 -- 172.21.15.150:0/1543302407 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756000fac0 con 0x5557604d9b30 2024-05-10T04:07:30.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.440+0000 7f7580d4fb80 1 -- 172.21.15.150:0/1543302407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9b30 msgr2=0x5557604d82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.440+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/1543302407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9b30 0x5557604d82c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7560015000 tx=0x7f756001f000 comp rx=0 tx=0).stop 2024-05-10T04:07:30.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.440+0000 7f7580d4fb80 1 -- 172.21.15.150:0/1543302407 shutdown_connections 2024-05-10T04:07:30.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.440+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/1543302407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d8de0 0x5557604e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.440+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/1543302407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9b30 0x5557604d82c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.440+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/1543302407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9fe0 0x5557604d8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.440+0000 7f7580d4fb80 1 -- 172.21.15.150:0/1543302407 >> 172.21.15.150:0/1543302407 conn(0x55576043aa60 msgr2=0x5557604d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.440+0000 7f7580d4fb80 1 -- 172.21.15.150:0/1543302407 shutdown_connections 2024-05-10T04:07:30.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.440+0000 7f7580d4fb80 1 -- 172.21.15.150:0/1543302407 wait complete. 2024-05-10T04:07:30.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.440+0000 7f7580d4fb80 1 -- start start 2024-05-10T04:07:30.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.440+0000 7f7580d4fb80 1 Processor -- start 2024-05-10T04:07:30.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f7580d4fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d8de0 0x5557604e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f7580d4fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 0x5557604e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f7580d4fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9fe0 0x5557604edf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f7580d4fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555760394560 con 0x5557604d9fe0 2024-05-10T04:07:30.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f7580d4fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555760394200 con 0x5557604d8de0 2024-05-10T04:07:30.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f7580d4fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55576027b6a0 con 0x5557604d9b30 2024-05-10T04:07:30.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f7580d4fb80 10 client.0 _ll_register_callbacks cb 0x55576039db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:07:30.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f756ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9fe0 0x5557604edf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f756ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9fe0 0x5557604edf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48500/0 (socket says 172.21.15.150:48500) 2024-05-10T04:07:30.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f756ad9d700 1 -- 172.21.15.150:0/421697180 learned_addr learned my addr 172.21.15.150:0/421697180 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f756a59c700 1 --2- 172.21.15.150:0/421697180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d8de0 0x5557604e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f7569d9b700 1 --2- 172.21.15.150:0/421697180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 0x5557604e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f756ad9d700 1 -- 172.21.15.150:0/421697180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 msgr2=0x5557604e6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f756ad9d700 1 --2- 172.21.15.150:0/421697180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 0x5557604e6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f756ad9d700 1 -- 172.21.15.150:0/421697180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d8de0 msgr2=0x5557604e6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f756ad9d700 1 --2- 172.21.15.150:0/421697180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d8de0 0x5557604e6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f756ad9d700 1 -- 172.21.15.150:0/421697180 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c1098d0 con 0x5557604d9fe0 2024-05-10T04:07:30.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f756ad9d700 1 --2- 172.21.15.150:0/421697180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9fe0 0x5557604edf50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7564008e90 tx=0x7f75640114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09144133fd6dd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f7551ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:07:30.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f7551ffb700 1 -- 172.21.15.150:0/421697180 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756400f050 con 0x5557604d9fe0 2024-05-10T04:07:30.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f7551ffb700 1 -- 172.21.15.150:0/421697180 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7564011b80 con 0x5557604d9fe0 2024-05-10T04:07:30.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.441+0000 7f7580d4fb80 1 -- 172.21.15.150:0/421697180 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555760351270 con 0x5557604d9fe0 2024-05-10T04:07:30.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.442+0000 7f7551ffb700 1 -- 172.21.15.150:0/421697180 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7564011d40 con 0x5557604d9fe0 2024-05-10T04:07:30.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.442+0000 7f7580d4fb80 10 client.16437 fetch_fsmap learned FSMap version 25 2024-05-10T04:07:30.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.442+0000 7f7580d4fb80 1 -- 172.21.15.150:0/421697180 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555760225b60 con 0x5557604d9fe0 2024-05-10T04:07:30.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.442+0000 7f7551ffb700 1 -- 172.21.15.150:0/421697180 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555760351270 con 0x5557604d9fe0 2024-05-10T04:07:30.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.442+0000 7f7551ffb700 1 -- 172.21.15.150:0/421697180 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7564015470 con 0x5557604d9fe0 2024-05-10T04:07:30.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.442+0000 7f7580d4fb80 10 client.16437 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:07:30.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.442+0000 7f7580d4fb80 10 client.16437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T04:07:30.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.442+0000 7f7580d4fb80 20 client.16437 populate_metadata read hostname 'smithi150' 2024-05-10T04:07:30.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.442+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/421697180 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5557604f27b0 0x5557604f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.442+0000 7f7580d4fb80 4 client.16437 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:07:30.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.442+0000 7f7580d4fb80 1 -- 172.21.15.150:0/421697180 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557604a7780 con 0x5557604f27b0 2024-05-10T04:07:30.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.442+0000 7f756a59c700 1 --2- 172.21.15.150:0/421697180 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5557604f27b0 0x5557604f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.443+0000 7f7551ffb700 10 client.16437 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:07:30.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.443+0000 7f756a59c700 1 --2- 172.21.15.150:0/421697180 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5557604f27b0 0x5557604f4c00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52ea421b578c8a19 server_cookie=c0d2ceb99cbc0466 in_seq=0 out_seq=0 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.443+0000 7f7551ffb700 1 -- 172.21.15.150:0/421697180 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5557604a7780 con 0x5557604f27b0 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.443+0000 7f7551ffb700 10 client.16437 handle_command_reply: tid=0 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.444+0000 7f7580d4fb80 1 client.16437 shutdown 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.444+0000 7f7580d4fb80 1 -- 172.21.15.150:0/421697180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9fe0 msgr2=0x5557604edf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.444+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/421697180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9fe0 0x5557604edf50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7564008e90 tx=0x7f75640114a0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.444+0000 7f7580d4fb80 1 -- 172.21.15.150:0/421697180 shutdown_connections 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.444+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/421697180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d9fe0 0x5557604edf50 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.444+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/421697180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d8de0 0x5557604e6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.444+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/421697180 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5557604f27b0 0x5557604f4c00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.444+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/421697180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 0x5557604e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.444+0000 7f7551ffb700 0 client.16437 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.444+0000 7f7580d4fb80 1 -- 172.21.15.150:0/421697180 >> 172.21.15.150:0/421697180 conn(0x55576043aa60 msgr2=0x5557604d2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.444+0000 7f7580d4fb80 1 -- 172.21.15.150:0/421697180 shutdown_connections 2024-05-10T04:07:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.444+0000 7f7580d4fb80 1 -- 172.21.15.150:0/421697180 wait complete. 2024-05-10T04:07:30.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.444+0000 7f7580d4fb80 20 client.16437 trim_cache size 0 max 16384 2024-05-10T04:07:30.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f7580d4fb80 1 Processor -- start 2024-05-10T04:07:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f7580d4fb80 1 -- start start 2024-05-10T04:07:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f7580d4fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 0x5557604e6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f7580d4fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 0x5557604edb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f7580d4fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d9fe0 0x5557604e1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f7580d4fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557604e1000 con 0x5557604d8de0 2024-05-10T04:07:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f7580d4fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557604ee530 con 0x5557604d9fe0 2024-05-10T04:07:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f7580d4fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557604ee6f0 con 0x5557604d9b30 2024-05-10T04:07:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f756a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 0x5557604e6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f756a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 0x5557604e6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48506/0 (socket says 172.21.15.150:48506) 2024-05-10T04:07:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f756a59c700 1 -- 172.21.15.150:0/1636788463 learned_addr learned my addr 172.21.15.150:0/1636788463 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f756ad9d700 1 --2- 172.21.15.150:0/1636788463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d9fe0 0x5557604e1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f7569d9b700 1 --2- 172.21.15.150:0/1636788463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 0x5557604edb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f756a59c700 1 -- 172.21.15.150:0/1636788463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 msgr2=0x5557604edb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.457+0000 7f756a59c700 1 --2- 172.21.15.150:0/1636788463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 0x5557604edb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.458+0000 7f756a59c700 1 -- 172.21.15.150:0/1636788463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d9fe0 msgr2=0x5557604e1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.458+0000 7f756a59c700 1 --2- 172.21.15.150:0/1636788463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d9fe0 0x5557604e1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.458+0000 7f756a59c700 1 -- 172.21.15.150:0/1636788463 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557603be560 con 0x5557604d8de0 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.458+0000 7f756ad9d700 1 --2- 172.21.15.150:0/1636788463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d9fe0 0x5557604e1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.458+0000 7f756a59c700 1 --2- 172.21.15.150:0/1636788463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 0x5557604e6020 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7560003c10 tx=0x7f7560005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866de4c017d4b4ef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.458+0000 7f75597fa700 1 -- 172.21.15.150:0/1636788463 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756000fb20 con 0x5557604d8de0 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.458+0000 7f75597fa700 1 -- 172.21.15.150:0/1636788463 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f756000f700 con 0x5557604d8de0 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.458+0000 7f75597fa700 1 -- 172.21.15.150:0/1636788463 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756003a900 con 0x5557604d8de0 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 -- 172.21.15.150:0/1636788463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 msgr2=0x5557604e6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/1636788463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 0x5557604e6020 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7560003c10 tx=0x7f7560005040 comp rx=0 tx=0).stop 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 -- 172.21.15.150:0/1636788463 shutdown_connections 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/1636788463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d9fe0 0x5557604e1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/1636788463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 0x5557604e6020 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/1636788463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 0x5557604edb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 -- 172.21.15.150:0/1636788463 >> 172.21.15.150:0/1636788463 conn(0x55576043aa60 msgr2=0x5557604e08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 -- 172.21.15.150:0/1636788463 shutdown_connections 2024-05-10T04:07:30.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 -- 172.21.15.150:0/1636788463 wait complete. 2024-05-10T04:07:30.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 -- start start 2024-05-10T04:07:30.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 Processor -- start 2024-05-10T04:07:30.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 0x5557604ead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 0x5557604eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d9fe0 0x555760546c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557604ee210 con 0x5557604d8de0 2024-05-10T04:07:30.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557604ee380 con 0x5557604d9fe0 2024-05-10T04:07:30.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555760547190 con 0x5557604d9b30 2024-05-10T04:07:30.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7580d4fb80 10 client.0 _ll_register_callbacks cb 0x55576039db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:07:30.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f756a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 0x5557604ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f756a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 0x5557604ead90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48512/0 (socket says 172.21.15.150:48512) 2024-05-10T04:07:30.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f756a59c700 1 -- 172.21.15.150:0/942783083 learned_addr learned my addr 172.21.15.150:0/942783083 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.459+0000 7f7569d9b700 1 --2- 172.21.15.150:0/942783083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 0x5557604eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f756ad9d700 1 --2- 172.21.15.150:0/942783083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d9fe0 0x555760546c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f756a59c700 1 -- 172.21.15.150:0/942783083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 msgr2=0x5557604eb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f756a59c700 1 --2- 172.21.15.150:0/942783083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 0x5557604eb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f756a59c700 1 -- 172.21.15.150:0/942783083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d9fe0 msgr2=0x555760546c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f756a59c700 1 --2- 172.21.15.150:0/942783083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d9fe0 0x555760546c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f756a59c700 1 -- 172.21.15.150:0/942783083 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557602793b0 con 0x5557604d8de0 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f756a59c700 1 --2- 172.21.15.150:0/942783083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 0x5557604ead90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f756000b8c0 tx=0x7f756003a260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9feaae79528aa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f7551ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f7551ffb700 1 -- 172.21.15.150:0/942783083 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756003a840 con 0x5557604d8de0 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f7580d4fb80 1 -- 172.21.15.150:0/942783083 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f756001e040 con 0x5557604d8de0 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f7551ffb700 1 -- 172.21.15.150:0/942783083 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f75600050b0 con 0x5557604d8de0 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f7551ffb700 1 -- 172.21.15.150:0/942783083 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75600052b0 con 0x5557604d8de0 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f7580d4fb80 10 client.16443 fetch_fsmap learned FSMap version 25 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f7580d4fb80 1 -- 172.21.15.150:0/942783083 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55576027dcd0 con 0x5557604d8de0 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.460+0000 7f7551ffb700 1 -- 172.21.15.150:0/942783083 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f756001e040 con 0x5557604d8de0 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.461+0000 7f7551ffb700 1 -- 172.21.15.150:0/942783083 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f756004c070 con 0x5557604d8de0 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.461+0000 7f7580d4fb80 10 client.16443 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.461+0000 7f7580d4fb80 10 client.16443 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.461+0000 7f7580d4fb80 20 client.16443 populate_metadata read hostname 'smithi150' 2024-05-10T04:07:30.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.461+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/942783083 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5557605485a0 0x55576054a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.461+0000 7f7580d4fb80 4 client.16443 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.461+0000 7f7580d4fb80 1 -- 172.21.15.150:0/942783083 --> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5557603f0ec0 con 0x5557605485a0 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.461+0000 7f7569d9b700 1 --2- 172.21.15.150:0/942783083 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5557605485a0 0x55576054a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.461+0000 7f7551ffb700 10 client.16443 ms_handle_connect on v2:172.21.15.181:6836/1798187480 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.461+0000 7f7569d9b700 1 --2- 172.21.15.150:0/942783083 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5557605485a0 0x55576054a9c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6608caba7e49ba51 server_cookie=a1798a4b5ee90d80 in_seq=0 out_seq=0 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.461+0000 7f7551ffb700 1 -- 172.21.15.150:0/942783083 <== mds.2 v2:172.21.15.181:6836/1798187480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5557603f0ec0 con 0x5557605485a0 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.461+0000 7f7551ffb700 10 client.16443 handle_command_reply: tid=0 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.461+0000 7f7580d4fb80 1 client.16443 shutdown 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7580d4fb80 1 -- 172.21.15.150:0/942783083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 msgr2=0x5557604ead90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/942783083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 0x5557604ead90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f756000b8c0 tx=0x7f756003a260 comp rx=0 tx=0).stop 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7580d4fb80 1 -- 172.21.15.150:0/942783083 shutdown_connections 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/942783083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557604d9fe0 0x555760546c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/942783083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557604d8de0 0x5557604ead90 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/942783083 >> [v2:172.21.15.181:6836/1798187480,v1:172.21.15.181:6837/1798187480] conn(0x5557605485a0 0x55576054a9c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7580d4fb80 1 --2- 172.21.15.150:0/942783083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557604d9b30 0x5557604eb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7551ffb700 0 client.16443 ms_handle_reset on v2:172.21.15.181:6836/1798187480 2024-05-10T04:07:30.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7580d4fb80 1 -- 172.21.15.150:0/942783083 >> 172.21.15.150:0/942783083 conn(0x55576043aa60 msgr2=0x5557604e1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7580d4fb80 1 -- 172.21.15.150:0/942783083 shutdown_connections 2024-05-10T04:07:30.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7580d4fb80 1 -- 172.21.15.150:0/942783083 wait complete. 2024-05-10T04:07:30.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7580d4fb80 20 client.16443 trim_cache size 0 max 16384 2024-05-10T04:07:30.477 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:07:30.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7570a0b700 1 -- 172.21.15.150:0/2153381359 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f75540a11c0 msgr2=0x7f75540a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7570a0b700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f75540a11c0 0x7f75540a36d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7560006010 tx=0x7f756001a040 comp rx=0 tx=0).stop 2024-05-10T04:07:30.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7570a0b700 1 -- 172.21.15.150:0/2153381359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f756c101bf0 msgr2=0x7f756c197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7570a0b700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f756c101bf0 0x7f756c197430 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f755c00c920 tx=0x7f755c00cc30 comp rx=0 tx=0).stop 2024-05-10T04:07:30.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7570a0b700 1 -- 172.21.15.150:0/2153381359 shutdown_connections 2024-05-10T04:07:30.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7570a0b700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f75540a11c0 0x7f75540a36d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7570a0b700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f756c102590 0x7f756c19e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7570a0b700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c100ad0 0x7f756c196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7570a0b700 1 --2- 172.21.15.150:0/2153381359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f756c101bf0 0x7f756c197430 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7570a0b700 1 -- 172.21.15.150:0/2153381359 >> 172.21.15.150:0/2153381359 conn(0x7f756c0fc0e0 msgr2=0x7f756c104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7570a0b700 1 -- 172.21.15.150:0/2153381359 shutdown_connections 2024-05-10T04:07:30.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.462+0000 7f7570a0b700 1 -- 172.21.15.150:0/2153381359 wait complete. 2024-05-10T04:07:30.480 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T04:07:30.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.614+0000 7f5bf66b3700 1 -- 172.21.15.150:0/1495355430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf0101a70 msgr2=0x7f5bf0101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.614+0000 7f5bf66b3700 1 --2- 172.21.15.150:0/1495355430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf0101a70 0x7f5bf0101f00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009990 tx=0x7f5be0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.614+0000 7f5bf66b3700 1 -- 172.21.15.150:0/1495355430 shutdown_connections 2024-05-10T04:07:30.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.614+0000 7f5bf66b3700 1 --2- 172.21.15.150:0/1495355430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bf0102590 0x7f5bf0109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.614+0000 7f5bf66b3700 1 --2- 172.21.15.150:0/1495355430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf0101090 0x7f5bf01014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.614+0000 7f5bf66b3700 1 --2- 172.21.15.150:0/1495355430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf0101a70 0x7f5bf0101f00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.614+0000 7f5bf66b3700 1 -- 172.21.15.150:0/1495355430 >> 172.21.15.150:0/1495355430 conn(0x7f5bf00fb8b0 msgr2=0x7f5bf00fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.614+0000 7f5bf66b3700 1 -- 172.21.15.150:0/1495355430 shutdown_connections 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.614+0000 7f5bf66b3700 1 -- 172.21.15.150:0/1495355430 wait complete. 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.614+0000 7f5bf66b3700 1 Processor -- start 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bf66b3700 1 -- start start 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bf66b3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bf0101090 0x7f5bf0196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bf66b3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf0101a70 0x7f5bf0197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bf66b3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf0102590 0x7f5bf019e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bf66b3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf019e8d0 con 0x7f5bf0101a70 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bf66b3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf019ea40 con 0x7f5bf0101090 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bf66b3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf019ec00 con 0x7f5bf0102590 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bef7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf0101a70 0x7f5bf0197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bef7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf0101a70 0x7f5bf0197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48530/0 (socket says 172.21.15.150:48530) 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bef7fe700 1 -- 172.21.15.150:0/412328037 learned_addr learned my addr 172.21.15.150:0/412328037 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5beffff700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bf0101090 0x7f5bf0196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bf4c50700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf0102590 0x7f5bf019e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bef7fe700 1 -- 172.21.15.150:0/412328037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf0102590 msgr2=0x7f5bf019e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bef7fe700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf0102590 0x7f5bf019e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bef7fe700 1 -- 172.21.15.150:0/412328037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bf0101090 msgr2=0x7f5bf0196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bef7fe700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bf0101090 0x7f5bf0196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bef7fe700 1 -- 172.21.15.150:0/412328037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be00095f0 con 0x7f5bf0101a70 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.615+0000 7f5bf4c50700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf0102590 0x7f5bf019e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.616+0000 7f5bef7fe700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf0101a70 0x7f5bf0197330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5be00095c0 tx=0x7f5be000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.616+0000 7f5bed7fa700 1 -- 172.21.15.150:0/412328037 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be001d070 con 0x7f5bf0101a70 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.616+0000 7f5bf66b3700 1 -- 172.21.15.150:0/412328037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf019ee80 con 0x7f5bf0101a70 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.616+0000 7f5bed7fa700 1 -- 172.21.15.150:0/412328037 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5be0022470 con 0x7f5bf0101a70 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.616+0000 7f5bed7fa700 1 -- 172.21.15.150:0/412328037 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0017650 con 0x7f5bf0101a70 2024-05-10T04:07:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.616+0000 7f5bf66b3700 1 -- 172.21.15.150:0/412328037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf019f240 con 0x7f5bf0101a70 2024-05-10T04:07:30.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.618+0000 7f5bed7fa700 1 -- 172.21.15.150:0/412328037 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5be00225e0 con 0x7f5bf0101a70 2024-05-10T04:07:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.618+0000 7f5bed7fa700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5bdc0a11c0 0x7f5bdc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.619+0000 7f5beffff700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5bdc0a11c0 0x7f5bdc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.619+0000 7f5bed7fa700 1 -- 172.21.15.150:0/412328037 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5be00cde90 con 0x7f5bf0101a70 2024-05-10T04:07:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.619+0000 7f5beffff700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5bdc0a11c0 0x7f5bdc0a36d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0069860 tx=0x7f5bd8007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.619+0000 7f5c069f7b80 1 Processor -- start 2024-05-10T04:07:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.619+0000 7f5c069f7b80 1 -- start start 2024-05-10T04:07:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.619+0000 7f5c069f7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7878b0 0x561c7f787d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5c069f7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 0x561c7f788b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5c069f7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7890d0 0x561c7f78f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5c069f7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7f6ac150 con 0x561c7f7890d0 2024-05-10T04:07:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5c069f7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7f6a9b10 con 0x561c7f788780 2024-05-10T04:07:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5c069f7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7f6a5400 con 0x561c7f7878b0 2024-05-10T04:07:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5bf4c50700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7890d0 0x561c7f78f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5bf4c50700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7890d0 0x561c7f78f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48546/0 (socket says 172.21.15.150:48546) 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5bf4c50700 1 -- 172.21.15.150:0/640660073 learned_addr learned my addr 172.21.15.150:0/640660073 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5bef7fe700 1 --2- 172.21.15.150:0/640660073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 0x561c7f788b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5beffff700 1 --2- 172.21.15.150:0/640660073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7878b0 0x561c7f787d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5bf4c50700 1 -- 172.21.15.150:0/640660073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7878b0 msgr2=0x561c7f787d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5bf4c50700 1 --2- 172.21.15.150:0/640660073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7878b0 0x561c7f787d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5bf4c50700 1 -- 172.21.15.150:0/640660073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 msgr2=0x561c7f788b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5bf4c50700 1 --2- 172.21.15.150:0/640660073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 0x561c7f788b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5bf4c50700 1 -- 172.21.15.150:0/640660073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c7f5263b0 con 0x561c7f7890d0 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5bef7fe700 1 --2- 172.21.15.150:0/640660073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 0x561c7f788b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5bf4c50700 1 --2- 172.21.15.150:0/640660073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7890d0 0x561c7f78f9d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009bf0 tx=0x7f5be8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c0a4fbd41e64d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5be67fc700 1 -- 172.21.15.150:0/640660073 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8007e00 con 0x561c7f7890d0 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5be67fc700 1 -- 172.21.15.150:0/640660073 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5be80096a0 con 0x561c7f7890d0 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.620+0000 7f5be67fc700 1 -- 172.21.15.150:0/640660073 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8014400 con 0x561c7f7890d0 2024-05-10T04:07:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.621+0000 7f5c069f7b80 1 -- 172.21.15.150:0/640660073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7890d0 msgr2=0x561c7f78f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.621+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/640660073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7890d0 0x561c7f78f9d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009bf0 tx=0x7f5be8009f00 comp rx=0 tx=0).stop 2024-05-10T04:07:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.621+0000 7f5c069f7b80 1 -- 172.21.15.150:0/640660073 shutdown_connections 2024-05-10T04:07:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.621+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/640660073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7890d0 0x561c7f78f9d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.621+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/640660073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7878b0 0x561c7f787d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.621+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/640660073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 0x561c7f788b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.621+0000 7f5c069f7b80 1 -- 172.21.15.150:0/640660073 >> 172.21.15.150:0/640660073 conn(0x561c7f6e7a60 msgr2=0x561c7f77ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.621+0000 7f5c069f7b80 1 -- 172.21.15.150:0/640660073 shutdown_connections 2024-05-10T04:07:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.621+0000 7f5c069f7b80 1 -- 172.21.15.150:0/640660073 wait complete. 2024-05-10T04:07:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.622+0000 7f5c069f7b80 1 -- start start 2024-05-10T04:07:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.622+0000 7f5c069f7b80 1 Processor -- start 2024-05-10T04:07:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.622+0000 7f5c069f7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 0x561c7f7918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.622+0000 7f5c069f7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 0x561c7f791df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.622+0000 7f5c069f7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f798d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.622+0000 7f5c069f7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7f641560 con 0x561c7f7878b0 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.622+0000 7f5c069f7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7f641200 con 0x561c7f788780 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.622+0000 7f5c069f7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7f5286a0 con 0x561c7f7890d0 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.622+0000 7f5c069f7b80 10 client.0 _ll_register_callbacks cb 0x561c7f64ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.622+0000 7f5beffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 0x561c7f7918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.622+0000 7f5beffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 0x561c7f7918b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48550/0 (socket says 172.21.15.150:48550) 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.622+0000 7f5beffff700 1 -- 172.21.15.150:0/4047612168 learned_addr learned my addr 172.21.15.150:0/4047612168 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5bef7fe700 1 --2- 172.21.15.150:0/4047612168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 0x561c7f791df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5bf4c50700 1 --2- 172.21.15.150:0/4047612168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f798d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5beffff700 1 -- 172.21.15.150:0/4047612168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 msgr2=0x561c7f798d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5beffff700 1 --2- 172.21.15.150:0/4047612168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f798d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5beffff700 1 -- 172.21.15.150:0/4047612168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 msgr2=0x561c7f791df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5beffff700 1 --2- 172.21.15.150:0/4047612168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 0x561c7f791df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5beffff700 1 -- 172.21.15.150:0/4047612168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf019f240 con 0x561c7f7878b0 2024-05-10T04:07:30.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5beffff700 1 --2- 172.21.15.150:0/4047612168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 0x561c7f7918b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8011720 tx=0x7f5bd80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a115b0d8abec28 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5bcf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5bcf7fe700 1 -- 172.21.15.150:0/4047612168 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8004460 con 0x561c7f7878b0 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5bcf7fe700 1 -- 172.21.15.150:0/4047612168 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5bd8017070 con 0x561c7f7878b0 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5c069f7b80 1 -- 172.21.15.150:0/4047612168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c7f754780 con 0x561c7f7878b0 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5bcf7fe700 1 -- 172.21.15.150:0/4047612168 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bd8003680 con 0x561c7f7878b0 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5c069f7b80 10 client.16455 fetch_fsmap learned FSMap version 25 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5c069f7b80 1 -- 172.21.15.150:0/4047612168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f5be8009820 con 0x561c7f7878b0 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.623+0000 7f5bcf7fe700 1 -- 172.21.15.150:0/4047612168 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c7f754780 con 0x561c7f7878b0 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.624+0000 7f5bcf7fe700 1 -- 172.21.15.150:0/4047612168 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5bd801f800 con 0x561c7f7878b0 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.624+0000 7f5c069f7b80 10 client.16455 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.624+0000 7f5c069f7b80 10 client.16455 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.624+0000 7f5c069f7b80 20 client.16455 populate_metadata read hostname 'smithi150' 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.624+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/4047612168 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x561c7f79d600 0x561c7f79fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.624+0000 7f5c069f7b80 4 client.16455 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.624+0000 7f5c069f7b80 1 -- 172.21.15.150:0/4047612168 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c7f69dec0 con 0x561c7f79d600 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.624+0000 7f5bef7fe700 1 --2- 172.21.15.150:0/4047612168 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x561c7f79d600 0x561c7f79fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.624+0000 7f5bef7fe700 1 --2- 172.21.15.150:0/4047612168 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x561c7f79d600 0x561c7f79fa50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=641ba5fd15f02110 server_cookie=b1da4a569d268420 in_seq=0 out_seq=0 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.624+0000 7f5bcf7fe700 10 client.16455 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5bcf7fe700 1 -- 172.21.15.150:0/4047612168 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561c7f69dec0 con 0x561c7f79d600 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5bcf7fe700 10 client.16455 handle_command_reply: tid=0 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5c069f7b80 1 client.16455 shutdown 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5c069f7b80 1 -- 172.21.15.150:0/4047612168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 msgr2=0x561c7f7918b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/4047612168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 0x561c7f7918b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8011720 tx=0x7f5bd80047c0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5c069f7b80 1 -- 172.21.15.150:0/4047612168 shutdown_connections 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/4047612168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f798d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/4047612168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 0x561c7f7918b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/4047612168 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x561c7f79d600 0x561c7f79fa50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/4047612168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 0x561c7f791df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5bcf7fe700 0 client.16455 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:07:30.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5c069f7b80 1 -- 172.21.15.150:0/4047612168 >> 172.21.15.150:0/4047612168 conn(0x561c7f6e7a60 msgr2=0x561c7f77ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5c069f7b80 1 -- 172.21.15.150:0/4047612168 shutdown_connections 2024-05-10T04:07:30.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.625+0000 7f5c069f7b80 1 -- 172.21.15.150:0/4047612168 wait complete. 2024-05-10T04:07:30.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.626+0000 7f5c069f7b80 20 client.16455 trim_cache size 0 max 16384 2024-05-10T04:07:30.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.638+0000 7f5c069f7b80 1 Processor -- start 2024-05-10T04:07:30.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5c069f7b80 1 -- start start 2024-05-10T04:07:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5c069f7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 0x561c7f77c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5c069f7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 0x561c7f798ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5c069f7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f77ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5c069f7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7f780520 con 0x561c7f7878b0 2024-05-10T04:07:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5c069f7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7f780ad0 con 0x561c7f788780 2024-05-10T04:07:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5c069f7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7f78f820 con 0x561c7f7890d0 2024-05-10T04:07:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5beffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 0x561c7f77c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5beffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 0x561c7f77c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48552/0 (socket says 172.21.15.150:48552) 2024-05-10T04:07:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5beffff700 1 -- 172.21.15.150:0/2743054708 learned_addr learned my addr 172.21.15.150:0/2743054708 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5bef7fe700 1 --2- 172.21.15.150:0/2743054708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 0x561c7f798ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5bf4c50700 1 --2- 172.21.15.150:0/2743054708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f77ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5beffff700 1 -- 172.21.15.150:0/2743054708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 msgr2=0x561c7f77ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5beffff700 1 --2- 172.21.15.150:0/2743054708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f77ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5beffff700 1 -- 172.21.15.150:0/2743054708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 msgr2=0x561c7f798ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5beffff700 1 --2- 172.21.15.150:0/2743054708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 0x561c7f798ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5beffff700 1 -- 172.21.15.150:0/2743054708 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c7f4d2b60 con 0x561c7f7878b0 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5bf4c50700 1 --2- 172.21.15.150:0/2743054708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f77ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.639+0000 7f5beffff700 1 --2- 172.21.15.150:0/2743054708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 0x561c7f77c180 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8010a10 tx=0x7f5bd80273a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a06af0811805eb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.640+0000 7f5be6ffd700 1 -- 172.21.15.150:0/2743054708 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8004460 con 0x561c7f7878b0 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.640+0000 7f5be6ffd700 1 -- 172.21.15.150:0/2743054708 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bd8023070 con 0x561c7f7878b0 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.640+0000 7f5be6ffd700 1 -- 172.21.15.150:0/2743054708 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8003aa0 con 0x561c7f7878b0 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.640+0000 7f5c069f7b80 1 -- 172.21.15.150:0/2743054708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 msgr2=0x561c7f77c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.640+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/2743054708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 0x561c7f77c180 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8010a10 tx=0x7f5bd80273a0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.640+0000 7f5c069f7b80 1 -- 172.21.15.150:0/2743054708 shutdown_connections 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.640+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/2743054708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f77ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.640+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/2743054708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f7878b0 0x561c7f77c180 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.640+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/2743054708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f788780 0x561c7f798ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.640+0000 7f5c069f7b80 1 -- 172.21.15.150:0/2743054708 >> 172.21.15.150:0/2743054708 conn(0x561c7f6e7a60 msgr2=0x561c7f787f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.640+0000 7f5c069f7b80 1 -- 172.21.15.150:0/2743054708 shutdown_connections 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.640+0000 7f5c069f7b80 1 -- 172.21.15.150:0/2743054708 wait complete. 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5c069f7b80 1 -- start start 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5c069f7b80 1 Processor -- start 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5c069f7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f7878b0 0x561c7f7eda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5c069f7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f788780 0x561c7f7effd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5c069f7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f7f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5c069f7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7f7f2d40 con 0x561c7f788780 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5c069f7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7f7f2eb0 con 0x561c7f7878b0 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5c069f7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7f7f3070 con 0x561c7f7890d0 2024-05-10T04:07:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5c069f7b80 10 client.0 _ll_register_callbacks cb 0x561c7f64ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:07:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5bef7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f788780 0x561c7f7effd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5bef7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f788780 0x561c7f7effd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48556/0 (socket says 172.21.15.150:48556) 2024-05-10T04:07:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5bef7fe700 1 -- 172.21.15.150:0/2128510440 learned_addr learned my addr 172.21.15.150:0/2128510440 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5bf4c50700 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f7f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.641+0000 7f5beffff700 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f7878b0 0x561c7f7eda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5bef7fe700 1 -- 172.21.15.150:0/2128510440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 msgr2=0x561c7f7f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5bef7fe700 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f7f26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5bef7fe700 1 -- 172.21.15.150:0/2128510440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f7878b0 msgr2=0x561c7f7eda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5bef7fe700 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f7878b0 0x561c7f7eda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5bef7fe700 1 -- 172.21.15.150:0/2128510440 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be8009820 con 0x561c7f788780 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5bf4c50700 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f7f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5beffff700 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f7878b0 0x561c7f7eda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5bef7fe700 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f788780 0x561c7f7effd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009350 tx=0x7f5be00cb5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13d9f54c45b5091 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5bcf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5bcf7fe700 1 -- 172.21.15.150:0/2128510440 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be00cb820 con 0x561c7f788780 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5bcf7fe700 1 -- 172.21.15.150:0/2128510440 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5be009fe40 con 0x561c7f788780 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5c069f7b80 1 -- 172.21.15.150:0/2128510440 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c7f5fe270 con 0x561c7f788780 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5bcf7fe700 1 -- 172.21.15.150:0/2128510440 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5be0022b00 con 0x561c7f788780 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5c069f7b80 10 client.16461 fetch_fsmap learned FSMap version 25 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5c069f7b80 1 -- 172.21.15.150:0/2128510440 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561c7f66b560 con 0x561c7f788780 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5bcf7fe700 1 -- 172.21.15.150:0/2128510440 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c7f5fe270 con 0x561c7f788780 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5bcf7fe700 1 -- 172.21.15.150:0/2128510440 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5be009a670 con 0x561c7f788780 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5c069f7b80 10 client.16461 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5c069f7b80 10 client.16461 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:07:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.642+0000 7f5c069f7b80 20 client.16461 populate_metadata read hostname 'smithi150' 2024-05-10T04:07:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.643+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x561c7f7f4680 0x561c7f7f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.643+0000 7f5c069f7b80 4 client.16461 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:07:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.643+0000 7f5c069f7b80 1 -- 172.21.15.150:0/2128510440 --> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561c7f4f3cf0 con 0x561c7f7f4680 2024-05-10T04:07:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.643+0000 7f5beffff700 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x561c7f7f4680 0x561c7f7f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.643+0000 7f5bcf7fe700 10 client.16461 ms_handle_connect on v2:172.21.15.181:6838/1415332890 2024-05-10T04:07:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.643+0000 7f5beffff700 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x561c7f7f4680 0x561c7f7f6aa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59a1315b64db797c server_cookie=91242bbf76dc94d in_seq=0 out_seq=0 2024-05-10T04:07:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.643+0000 7f5bcf7fe700 1 -- 172.21.15.150:0/2128510440 <== mds.1 v2:172.21.15.181:6838/1415332890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561c7f4f3cf0 con 0x561c7f7f4680 2024-05-10T04:07:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.643+0000 7f5bcf7fe700 10 client.16461 handle_command_reply: tid=0 2024-05-10T04:07:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.643+0000 7f5c069f7b80 1 client.16461 shutdown 2024-05-10T04:07:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5c069f7b80 1 -- 172.21.15.150:0/2128510440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f788780 msgr2=0x561c7f7effd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f788780 0x561c7f7effd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009350 tx=0x7f5be00cb5e0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5c069f7b80 1 -- 172.21.15.150:0/2128510440 shutdown_connections 2024-05-10T04:07:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561c7f7890d0 0x561c7f7f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.181:6838/1415332890,v1:172.21.15.181:6839/1415332890] conn(0x561c7f7f4680 0x561c7f7f6aa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c7f7878b0 0x561c7f7eda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5c069f7b80 1 --2- 172.21.15.150:0/2128510440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c7f788780 0x561c7f7effd0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5bcf7fe700 0 client.16461 ms_handle_reset on v2:172.21.15.181:6838/1415332890 2024-05-10T04:07:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5c069f7b80 1 -- 172.21.15.150:0/2128510440 >> 172.21.15.150:0/2128510440 conn(0x561c7f6e7a60 msgr2=0x561c7f787f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5c069f7b80 1 -- 172.21.15.150:0/2128510440 shutdown_connections 2024-05-10T04:07:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5c069f7b80 1 -- 172.21.15.150:0/2128510440 wait complete. 2024-05-10T04:07:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5c069f7b80 20 client.16461 trim_cache size 0 max 16384 2024-05-10T04:07:30.653 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:07:30.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5bf66b3700 1 -- 172.21.15.150:0/412328037 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5bdc0a11c0 msgr2=0x7f5bdc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5bf66b3700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5bdc0a11c0 0x7f5bdc0a36d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0069860 tx=0x7f5bd8007400 comp rx=0 tx=0).stop 2024-05-10T04:07:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5bf66b3700 1 -- 172.21.15.150:0/412328037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf0101a70 msgr2=0x7f5bf0197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5bf66b3700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf0101a70 0x7f5bf0197330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5be00095c0 tx=0x7f5be000f690 comp rx=0 tx=0).stop 2024-05-10T04:07:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5bf66b3700 1 -- 172.21.15.150:0/412328037 shutdown_connections 2024-05-10T04:07:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5bf66b3700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f5bdc0a11c0 0x7f5bdc0a36d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5bf66b3700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf0102590 0x7f5bf019e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5bf66b3700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bf0101090 0x7f5bf0196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5bf66b3700 1 --2- 172.21.15.150:0/412328037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bf0101a70 0x7f5bf0197330 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5bf66b3700 1 -- 172.21.15.150:0/412328037 >> 172.21.15.150:0/412328037 conn(0x7f5bf00fb8b0 msgr2=0x7f5bf0105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5bf66b3700 1 -- 172.21.15.150:0/412328037 shutdown_connections 2024-05-10T04:07:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.644+0000 7f5bf66b3700 1 -- 172.21.15.150:0/412328037 wait complete. 2024-05-10T04:07:30.658 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:30 smithi150 ceph-mon[42477]: pgmap v377: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:07:30.658 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:30 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2388881109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:07:30.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T04:07:30.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:30 smithi181 ceph-mon[46602]: pgmap v377: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:07:30.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:30 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2388881109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:07:30.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:30 smithi161 ceph-mon[46918]: pgmap v377: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:07:30.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:30 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2388881109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:07:30.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.792+0000 7ffbebbcf700 1 -- 172.21.15.150:0/2683287582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe4100ae0 msgr2=0x7ffbe4100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.792+0000 7ffbebbcf700 1 --2- 172.21.15.150:0/2683287582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe4100ae0 0x7ffbe4100ef0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0009990 tx=0x7ffbe0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.792+0000 7ffbebbcf700 1 -- 172.21.15.150:0/2683287582 shutdown_connections 2024-05-10T04:07:30.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.792+0000 7ffbebbcf700 1 --2- 172.21.15.150:0/2683287582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe41025a0 0x7ffbe4108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.792+0000 7ffbebbcf700 1 --2- 172.21.15.150:0/2683287582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe4100ae0 0x7ffbe4100ef0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.792+0000 7ffbebbcf700 1 --2- 172.21.15.150:0/2683287582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbe4101c00 0x7ffbe4102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.792+0000 7ffbebbcf700 1 -- 172.21.15.150:0/2683287582 >> 172.21.15.150:0/2683287582 conn(0x7ffbe40fc0f0 msgr2=0x7ffbe40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.792+0000 7ffbebbcf700 1 -- 172.21.15.150:0/2683287582 shutdown_connections 2024-05-10T04:07:30.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.792+0000 7ffbebbcf700 1 -- 172.21.15.150:0/2683287582 wait complete. 2024-05-10T04:07:30.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.793+0000 7ffbebbcf700 1 Processor -- start 2024-05-10T04:07:30.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.793+0000 7ffbebbcf700 1 -- start start 2024-05-10T04:07:30.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbebbcf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbe4100ae0 0x7ffbe4196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbebbcf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe4101c00 0x7ffbe4197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbebbcf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe41025a0 0x7ffbe419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbebbcf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe419e910 con 0x7ffbe41025a0 2024-05-10T04:07:30.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbebbcf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe419ea80 con 0x7ffbe4100ae0 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbebbcf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe419ec40 con 0x7ffbe4101c00 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbea16c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe41025a0 0x7ffbe419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbea16c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe41025a0 0x7ffbe419e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48576/0 (socket says 172.21.15.150:48576) 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbea16c700 1 -- 172.21.15.150:0/822461372 learned_addr learned my addr 172.21.15.150:0/822461372 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbe916a700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe4101c00 0x7ffbe4197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbe996b700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbe4100ae0 0x7ffbe4196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbea16c700 1 -- 172.21.15.150:0/822461372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe4101c00 msgr2=0x7ffbe4197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbea16c700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe4101c00 0x7ffbe4197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbea16c700 1 -- 172.21.15.150:0/822461372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbe4100ae0 msgr2=0x7ffbe4196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbea16c700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbe4100ae0 0x7ffbe4196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbea16c700 1 -- 172.21.15.150:0/822461372 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe00095f0 con 0x7ffbe41025a0 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbe996b700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbe4100ae0 0x7ffbe4196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbea16c700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe41025a0 0x7ffbe419e300 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc007ee0 tx=0x7ffbdc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbd6ffd700 1 -- 172.21.15.150:0/822461372 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc00ddd0 con 0x7ffbe41025a0 2024-05-10T04:07:30.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbd6ffd700 1 -- 172.21.15.150:0/822461372 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbdc00f040 con 0x7ffbe41025a0 2024-05-10T04:07:30.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbebbcf700 1 -- 172.21.15.150:0/822461372 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe419ef20 con 0x7ffbe41025a0 2024-05-10T04:07:30.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbd6ffd700 1 -- 172.21.15.150:0/822461372 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc0048e0 con 0x7ffbe41025a0 2024-05-10T04:07:30.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.794+0000 7ffbebbcf700 1 -- 172.21.15.150:0/822461372 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe419f2f0 con 0x7ffbe41025a0 2024-05-10T04:07:30.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.797+0000 7ffbd6ffd700 1 -- 172.21.15.150:0/822461372 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ffbdc003a00 con 0x7ffbe41025a0 2024-05-10T04:07:30.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.797+0000 7ffbd6ffd700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffbd00a11c0 0x7ffbd00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.797+0000 7ffbd6ffd700 1 -- 172.21.15.150:0/822461372 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffbdc0cc470 con 0x7ffbe41025a0 2024-05-10T04:07:30.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.797+0000 7ffbe996b700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffbd00a11c0 0x7ffbd00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.798+0000 7ffbe996b700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffbd00a11c0 0x7ffbd00a36d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000b7b0 tx=0x7ffbe001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.798+0000 7ffbfbf13b80 1 Processor -- start 2024-05-10T04:07:30.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.798+0000 7ffbfbf13b80 1 -- start start 2024-05-10T04:07:30.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.798+0000 7ffbfbf13b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7d6a0 0x560beec7daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.798+0000 7ffbfbf13b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 0x560beec7e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.798+0000 7ffbfbf13b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7eec0 0x560beec857c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.798+0000 7ffbfbf13b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beeba4150 con 0x560beec7e570 2024-05-10T04:07:30.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbfbf13b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beeba1b10 con 0x560beec7d6a0 2024-05-10T04:07:30.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbfbf13b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beeb9d400 con 0x560beec7eec0 2024-05-10T04:07:30.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbe916a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 0x560beec7e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbe916a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 0x560beec7e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48584/0 (socket says 172.21.15.150:48584) 2024-05-10T04:07:30.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbe916a700 1 -- 172.21.15.150:0/3816233184 learned_addr learned my addr 172.21.15.150:0/3816233184 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbe996b700 1 --2- 172.21.15.150:0/3816233184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7d6a0 0x560beec7daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbea16c700 1 --2- 172.21.15.150:0/3816233184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7eec0 0x560beec857c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbe916a700 1 -- 172.21.15.150:0/3816233184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7eec0 msgr2=0x560beec857c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbe916a700 1 --2- 172.21.15.150:0/3816233184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7eec0 0x560beec857c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbe916a700 1 -- 172.21.15.150:0/3816233184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7d6a0 msgr2=0x560beec7daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbe916a700 1 --2- 172.21.15.150:0/3816233184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7d6a0 0x560beec7daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbe916a700 1 -- 172.21.15.150:0/3816233184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560beea1e3b0 con 0x560beec7e570 2024-05-10T04:07:30.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbea16c700 1 --2- 172.21.15.150:0/3816233184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7eec0 0x560beec857c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:07:30.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbe916a700 1 --2- 172.21.15.150:0/3816233184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 0x560beec7e980 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009ca0 tx=0x7ffbd80115d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2d72bb1cda80d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbcffff700 1 -- 172.21.15.150:0/3816233184 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8016070 con 0x560beec7e570 2024-05-10T04:07:30.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbcffff700 1 -- 172.21.15.150:0/3816233184 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbd8004cf0 con 0x560beec7e570 2024-05-10T04:07:30.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.799+0000 7ffbcffff700 1 -- 172.21.15.150:0/3816233184 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd800e6d0 con 0x560beec7e570 2024-05-10T04:07:30.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.800+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/3816233184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 msgr2=0x560beec7e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.800+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/3816233184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 0x560beec7e980 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009ca0 tx=0x7ffbd80115d0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.800+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/3816233184 shutdown_connections 2024-05-10T04:07:30.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.800+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/3816233184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7eec0 0x560beec857c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.800+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/3816233184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7d6a0 0x560beec7daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.800+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/3816233184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 0x560beec7e980 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.800+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/3816233184 >> 172.21.15.150:0/3816233184 conn(0x560beebdfa60 msgr2=0x560beec77fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.800+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/3816233184 shutdown_connections 2024-05-10T04:07:30.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.800+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/3816233184 wait complete. 2024-05-10T04:07:30.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbfbf13b80 1 -- start start 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbfbf13b80 1 Processor -- start 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbfbf13b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 0x560beec876a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbfbf13b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7e570 0x560beec87be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbfbf13b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 0x560beec8eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbfbf13b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beeb39560 con 0x560beec7eec0 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbfbf13b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beeb39200 con 0x560beec7e570 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbfbf13b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beea206a0 con 0x560beec7d6a0 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbfbf13b80 10 client.0 _ll_register_callbacks cb 0x560beeb42b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbea16c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 0x560beec8eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbea16c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 0x560beec8eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48588/0 (socket says 172.21.15.150:48588) 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbea16c700 1 -- 172.21.15.150:0/646138224 learned_addr learned my addr 172.21.15.150:0/646138224 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbe916a700 1 --2- 172.21.15.150:0/646138224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7e570 0x560beec87be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.801+0000 7ffbe996b700 1 --2- 172.21.15.150:0/646138224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 0x560beec876a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbea16c700 1 -- 172.21.15.150:0/646138224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 msgr2=0x560beec876a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbea16c700 1 --2- 172.21.15.150:0/646138224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 0x560beec876a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbea16c700 1 -- 172.21.15.150:0/646138224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7e570 msgr2=0x560beec87be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbea16c700 1 --2- 172.21.15.150:0/646138224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7e570 0x560beec87be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbea16c700 1 -- 172.21.15.150:0/646138224 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd8009980 con 0x560beec7eec0 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbea16c700 1 --2- 172.21.15.150:0/646138224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 0x560beec8eb40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc01b000 tx=0x7ffbdc099220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26995fffa476293f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbcd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbcd7fa700 1 -- 172.21.15.150:0/646138224 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc0995b0 con 0x560beec7eec0 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbcd7fa700 1 -- 172.21.15.150:0/646138224 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffbdc09d3e0 con 0x560beec7eec0 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/646138224 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560beeaf6270 con 0x560beec7eec0 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbcd7fa700 1 -- 172.21.15.150:0/646138224 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbdc0a1a30 con 0x560beec7eec0 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbfbf13b80 10 client.16473 fetch_fsmap learned FSMap version 25 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/646138224 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560bee9cab60 con 0x560beec7eec0 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbcd7fa700 1 -- 172.21.15.150:0/646138224 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560beeaf6270 con 0x560beec7eec0 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbcd7fa700 1 -- 172.21.15.150:0/646138224 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffbdc0840b0 con 0x560beec7eec0 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbfbf13b80 10 client.16473 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:07:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbfbf13b80 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbfbf13b80 20 client.16473 populate_metadata read hostname 'smithi150' 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/646138224 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560beec932d0 0x560beec95720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbfbf13b80 4 client.16473 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.802+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/646138224 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560beec4c780 con 0x560beec932d0 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.803+0000 7ffbe996b700 1 --2- 172.21.15.150:0/646138224 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560beec932d0 0x560beec95720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.803+0000 7ffbcd7fa700 10 client.16473 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.803+0000 7ffbe996b700 1 --2- 172.21.15.150:0/646138224 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560beec932d0 0x560beec95720 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=223661a137ae41ee server_cookie=44b5da5a6b4e7917 in_seq=0 out_seq=0 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbcd7fa700 1 -- 172.21.15.150:0/646138224 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560beec4c780 con 0x560beec932d0 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbcd7fa700 10 client.16473 handle_command_reply: tid=0 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbfbf13b80 1 client.16473 shutdown 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/646138224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 msgr2=0x560beec8eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/646138224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 0x560beec8eb40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc01b000 tx=0x7ffbdc099220 comp rx=0 tx=0).stop 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/646138224 shutdown_connections 2024-05-10T04:07:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/646138224 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560beec932d0 0x560beec95720 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/646138224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 0x560beec8eb40 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/646138224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 0x560beec876a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbcd7fa700 0 client.16473 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:07:30.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/646138224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7e570 0x560beec87be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/646138224 >> 172.21.15.150:0/646138224 conn(0x560beebdfa60 msgr2=0x560beec77fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/646138224 shutdown_connections 2024-05-10T04:07:30.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/646138224 wait complete. 2024-05-10T04:07:30.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.804+0000 7ffbfbf13b80 20 client.16473 trim_cache size 0 max 16384 2024-05-10T04:07:30.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.817+0000 7ffbfbf13b80 1 Processor -- start 2024-05-10T04:07:30.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.817+0000 7ffbfbf13b80 1 -- start start 2024-05-10T04:07:30.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.817+0000 7ffbfbf13b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 0x560beec9a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.817+0000 7ffbfbf13b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7e570 0x560beec8eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.817+0000 7ffbfbf13b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 0x560beec82350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.817+0000 7ffbfbf13b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beec887b0 con 0x560beec7eec0 2024-05-10T04:07:30.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.817+0000 7ffbfbf13b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beec797a0 con 0x560beec7e570 2024-05-10T04:07:30.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.817+0000 7ffbfbf13b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beec727f0 con 0x560beec7d6a0 2024-05-10T04:07:30.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbea16c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 0x560beec82350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbea16c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 0x560beec82350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48598/0 (socket says 172.21.15.150:48598) 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbea16c700 1 -- 172.21.15.150:0/1982233045 learned_addr learned my addr 172.21.15.150:0/1982233045 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbe996b700 1 --2- 172.21.15.150:0/1982233045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 0x560beec9a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbe916a700 1 --2- 172.21.15.150:0/1982233045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7e570 0x560beec8eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbea16c700 1 -- 172.21.15.150:0/1982233045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 msgr2=0x560beec9a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbea16c700 1 --2- 172.21.15.150:0/1982233045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 0x560beec9a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbea16c700 1 -- 172.21.15.150:0/1982233045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7e570 msgr2=0x560beec8eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbea16c700 1 --2- 172.21.15.150:0/1982233045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7e570 0x560beec8eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbea16c700 1 -- 172.21.15.150:0/1982233045 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560beeb63560 con 0x560beec7eec0 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbea16c700 1 --2- 172.21.15.150:0/1982233045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 0x560beec82350 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0992b0 tx=0x7ffbdc0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0a0b8b2ad11bc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbd4d7f700 1 -- 172.21.15.150:0/1982233045 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc097950 con 0x560beec7eec0 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbd4d7f700 1 -- 172.21.15.150:0/1982233045 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbdc09d3e0 con 0x560beec7eec0 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.818+0000 7ffbd4d7f700 1 -- 172.21.15.150:0/1982233045 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc0a1ac0 con 0x560beec7eec0 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.819+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/1982233045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 msgr2=0x560beec82350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.819+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/1982233045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 0x560beec82350 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0992b0 tx=0x7ffbdc0092e0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.819+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/1982233045 shutdown_connections 2024-05-10T04:07:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.819+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/1982233045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7eec0 0x560beec82350 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.819+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/1982233045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 0x560beec9a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.819+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/1982233045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7e570 0x560beec8eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.819+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/1982233045 >> 172.21.15.150:0/1982233045 conn(0x560beebdfa60 msgr2=0x560beec78ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.819+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/1982233045 shutdown_connections 2024-05-10T04:07:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.819+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/1982233045 wait complete. 2024-05-10T04:07:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.819+0000 7ffbfbf13b80 1 -- start start 2024-05-10T04:07:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.819+0000 7ffbfbf13b80 1 Processor -- start 2024-05-10T04:07:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbfbf13b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 0x560beec8c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbfbf13b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 0x560beec8c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbfbf13b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7eec0 0x560beec8d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbfbf13b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beec8dc50 con 0x560beec7e570 2024-05-10T04:07:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbfbf13b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beec8ddc0 con 0x560beec7eec0 2024-05-10T04:07:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbfbf13b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beec8df80 con 0x560beec7d6a0 2024-05-10T04:07:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbfbf13b80 10 client.0 _ll_register_callbacks cb 0x560beeb42b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:07:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbe916a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 0x560beec8c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbe916a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 0x560beec8c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48604/0 (socket says 172.21.15.150:48604) 2024-05-10T04:07:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbe916a700 1 -- 172.21.15.150:0/213105251 learned_addr learned my addr 172.21.15.150:0/213105251 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbe996b700 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 0x560beec8c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbea16c700 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7eec0 0x560beec8d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbe916a700 1 -- 172.21.15.150:0/213105251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 msgr2=0x560beec8c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbe916a700 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 0x560beec8c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbe916a700 1 -- 172.21.15.150:0/213105251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7eec0 msgr2=0x560beec8d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:07:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbe916a700 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7eec0 0x560beec8d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbe916a700 1 -- 172.21.15.150:0/213105251 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bee9cab60 con 0x560beec7e570 2024-05-10T04:07:30.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.820+0000 7ffbe996b700 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 0x560beec8c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:07:30.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbe916a700 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 0x560beec8c7e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009ca0 tx=0x7ffbd8005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c6151e0812c96f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:30.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbcd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:07:30.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbcd7fa700 1 -- 172.21.15.150:0/213105251 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8016040 con 0x560beec7e570 2024-05-10T04:07:30.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbcd7fa700 1 -- 172.21.15.150:0/213105251 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffbd8005540 con 0x560beec7e570 2024-05-10T04:07:30.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/213105251 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffbd8004cf0 con 0x560beec7e570 2024-05-10T04:07:30.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbcd7fa700 1 -- 172.21.15.150:0/213105251 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbd800e470 con 0x560beec7e570 2024-05-10T04:07:30.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbfbf13b80 10 client.16479 fetch_fsmap learned FSMap version 25 2024-05-10T04:07:30.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/213105251 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560beea22cd0 con 0x560beec7e570 2024-05-10T04:07:30.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbcd7fa700 1 -- 172.21.15.150:0/213105251 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbd8004cf0 con 0x560beec7e570 2024-05-10T04:07:30.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbcd7fa700 1 -- 172.21.15.150:0/213105251 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffbd802c020 con 0x560beec7e570 2024-05-10T04:07:30.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbfbf13b80 10 client.16479 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T04:07:30.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbfbf13b80 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:07:30.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbfbf13b80 20 client.16479 populate_metadata read hostname 'smithi150' 2024-05-10T04:07:30.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560beece9f90 0x560beecec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:30.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbfbf13b80 4 client.16479 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:07:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.821+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/213105251 --> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560beeb95ec0 con 0x560beece9f90 2024-05-10T04:07:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbe996b700 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560beece9f90 0x560beecec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbcd7fa700 10 client.16479 ms_handle_connect on v2:172.21.15.161:6838/267491715 2024-05-10T04:07:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbe996b700 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560beece9f90 0x560beecec3b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d4d03adc1273522 server_cookie=c3a7f2d1ffd0ffe3 in_seq=0 out_seq=0 2024-05-10T04:07:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbcd7fa700 1 -- 172.21.15.150:0/213105251 <== mds.0 v2:172.21.15.161:6838/267491715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560beeb95ec0 con 0x560beece9f90 2024-05-10T04:07:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbcd7fa700 10 client.16479 handle_command_reply: tid=0 2024-05-10T04:07:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbfbf13b80 1 client.16479 shutdown 2024-05-10T04:07:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/213105251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 msgr2=0x560beec8c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 0x560beec8c7e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009ca0 tx=0x7ffbd8005090 comp rx=0 tx=0).stop 2024-05-10T04:07:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/213105251 shutdown_connections 2024-05-10T04:07:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.161:6838/267491715,v1:172.21.15.161:6839/267491715] conn(0x560beece9f90 0x560beecec3b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560beec7eec0 0x560beec8d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560beec7d6a0 0x560beec8c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbcd7fa700 0 client.16479 ms_handle_reset on v2:172.21.15.161:6838/267491715 2024-05-10T04:07:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbfbf13b80 1 --2- 172.21.15.150:0/213105251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560beec7e570 0x560beec8c7e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/213105251 >> 172.21.15.150:0/213105251 conn(0x560beebdfa60 msgr2=0x560beec78ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/213105251 shutdown_connections 2024-05-10T04:07:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.822+0000 7ffbfbf13b80 1 -- 172.21.15.150:0/213105251 wait complete. 2024-05-10T04:07:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.823+0000 7ffbfbf13b80 20 client.16479 trim_cache size 0 max 16384 2024-05-10T04:07:30.835 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:07:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.823+0000 7ffbebbcf700 1 -- 172.21.15.150:0/822461372 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffbd00a11c0 msgr2=0x7ffbd00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.823+0000 7ffbebbcf700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffbd00a11c0 0x7ffbd00a36d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000b7b0 tx=0x7ffbe001a040 comp rx=0 tx=0).stop 2024-05-10T04:07:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.823+0000 7ffbebbcf700 1 -- 172.21.15.150:0/822461372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe41025a0 msgr2=0x7ffbe419e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.823+0000 7ffbebbcf700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe41025a0 0x7ffbe419e300 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc007ee0 tx=0x7ffbdc00d3b0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.823+0000 7ffbebbcf700 1 -- 172.21.15.150:0/822461372 shutdown_connections 2024-05-10T04:07:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.823+0000 7ffbebbcf700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ffbd00a11c0 0x7ffbd00a36d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.823+0000 7ffbebbcf700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe41025a0 0x7ffbe419e300 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.823+0000 7ffbebbcf700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbe4100ae0 0x7ffbe4196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.823+0000 7ffbebbcf700 1 --2- 172.21.15.150:0/822461372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe4101c00 0x7ffbe4197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.823+0000 7ffbebbcf700 1 -- 172.21.15.150:0/822461372 >> 172.21.15.150:0/822461372 conn(0x7ffbe40fc0f0 msgr2=0x7ffbe4104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.823+0000 7ffbebbcf700 1 -- 172.21.15.150:0/822461372 shutdown_connections 2024-05-10T04:07:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:30.823+0000 7ffbebbcf700 1 -- 172.21.15.150:0/822461372 wait complete. 2024-05-10T04:07:31.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:31 smithi150 ceph-mon[42477]: pgmap v378: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:31.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:31 smithi181 ceph-mon[46602]: pgmap v378: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:31.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:31 smithi161 ceph-mon[46918]: pgmap v378: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:31.840 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-05-10T04:07:31.840 INFO:tasks.fwd_scrub:done joining 2024-05-10T04:07:31.840 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-10T04:07:31.852 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:07:32.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.998+0000 7f0bf031a700 1 -- 172.21.15.150:0/2843878191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be8100530 msgr2=0x7f0be810a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.998+0000 7f0bf031a700 1 --2- 172.21.15.150:0/2843878191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be8100530 0x7f0be810a150 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0be000b0a0 tx=0x7f0be000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.998+0000 7f0bf031a700 1 -- 172.21.15.150:0/2843878191 shutdown_connections 2024-05-10T04:07:32.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.998+0000 7f0bf031a700 1 --2- 172.21.15.150:0/2843878191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be8100530 0x7f0be810a150 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.998+0000 7f0bf031a700 1 --2- 172.21.15.150:0/2843878191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0be81031a0 0x7f0be81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.998+0000 7f0bf031a700 1 --2- 172.21.15.150:0/2843878191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0be80ffbe0 0x7f0be80ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.998+0000 7f0bf031a700 1 -- 172.21.15.150:0/2843878191 >> 172.21.15.150:0/2843878191 conn(0x7f0be80fb8b0 msgr2=0x7f0be80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:32.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.998+0000 7f0bf031a700 1 -- 172.21.15.150:0/2843878191 shutdown_connections 2024-05-10T04:07:32.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.998+0000 7f0bf031a700 1 -- 172.21.15.150:0/2843878191 wait complete. 2024-05-10T04:07:32.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.999+0000 7f0bf031a700 1 Processor -- start 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.999+0000 7f0bf031a700 1 -- start start 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.999+0000 7f0bf031a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0be80ffbe0 0x7f0be8102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.999+0000 7f0bf031a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0be8100530 0x7f0be8100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.999+0000 7f0bf031a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be81031a0 0x7f0be8101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.999+0000 7f0bf031a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be81019b0 con 0x7f0be81031a0 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.999+0000 7f0bf031a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8101b20 con 0x7f0be8100530 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.999+0000 7f0bf031a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8101ce0 con 0x7f0be80ffbe0 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.999+0000 7f0bee8b7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be81031a0 0x7f0be8101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.999+0000 7f0bee8b7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be81031a0 0x7f0be8101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48614/0 (socket says 172.21.15.150:48614) 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.999+0000 7f0bee8b7700 1 -- 172.21.15.150:0/3423046818 learned_addr learned my addr 172.21.15.150:0/3423046818 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.999+0000 7f0bed8b5700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0be8100530 0x7f0be8100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:31.999+0000 7f0bee0b6700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0be80ffbe0 0x7f0be8102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.000+0000 7f0bee8b7700 1 -- 172.21.15.150:0/3423046818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0be80ffbe0 msgr2=0x7f0be8102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.000+0000 7f0bee8b7700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0be80ffbe0 0x7f0be8102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.000+0000 7f0bee8b7700 1 -- 172.21.15.150:0/3423046818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0be8100530 msgr2=0x7f0be8100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.000+0000 7f0bee8b7700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0be8100530 0x7f0be8100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.000+0000 7f0bee8b7700 1 -- 172.21.15.150:0/3423046818 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be0009d00 con 0x7f0be81031a0 2024-05-10T04:07:32.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.000+0000 7f0bee8b7700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be81031a0 0x7f0be8101350 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0be000b070 tx=0x7f0be000bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:32.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.000+0000 7f0bdf7fe700 1 -- 172.21.15.150:0/3423046818 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be000e040 con 0x7f0be81031a0 2024-05-10T04:07:32.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.000+0000 7f0bdf7fe700 1 -- 172.21.15.150:0/3423046818 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0be000be30 con 0x7f0be81031a0 2024-05-10T04:07:32.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.000+0000 7f0bf031a700 1 -- 172.21.15.150:0/3423046818 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be81a2270 con 0x7f0be81031a0 2024-05-10T04:07:32.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.000+0000 7f0bdf7fe700 1 -- 172.21.15.150:0/3423046818 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be001b680 con 0x7f0be81031a0 2024-05-10T04:07:32.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.000+0000 7f0bf031a700 1 -- 172.21.15.150:0/3423046818 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be81a2690 con 0x7f0be81031a0 2024-05-10T04:07:32.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.001+0000 7f0bf031a700 1 -- 172.21.15.150:0/3423046818 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be8108ff0 con 0x7f0be81031a0 2024-05-10T04:07:32.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.002+0000 7f0bdf7fe700 1 -- 172.21.15.150:0/3423046818 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0be001d470 con 0x7f0be81031a0 2024-05-10T04:07:32.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.003+0000 7f0bdf7fe700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0bd40a1120 0x7f0bd40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:32.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.003+0000 7f0bdf7fe700 1 -- 172.21.15.150:0/3423046818 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0be00ce340 con 0x7f0be81031a0 2024-05-10T04:07:32.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.003+0000 7f0bee0b6700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0bd40a1120 0x7f0bd40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:32.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.003+0000 7f0bee0b6700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0bd40a1120 0x7f0bd40a3630 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0be80698c0 tx=0x7f0bd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:32.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.006+0000 7f0bdf7fe700 1 -- 172.21.15.150:0/3423046818 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0be0097850 con 0x7f0be81031a0 2024-05-10T04:07:32.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.207+0000 7f0bf031a700 1 -- 172.21.15.150:0/3423046818 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0be810d510 con 0x7f0be81031a0 2024-05-10T04:07:32.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.208+0000 7f0bdf7fe700 1 -- 172.21.15.150:0/3423046818 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24554 (secure 0 0 0) 0x7f0be0064000 con 0x7f0be81031a0 2024-05-10T04:07:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:07:32.211 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":87,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:03:30.185138+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:07:32.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.211+0000 7f0bf031a700 1 -- 172.21.15.150:0/3423046818 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0bd40a1120 msgr2=0x7f0bd40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.211+0000 7f0bf031a700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0bd40a1120 0x7f0bd40a3630 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0be80698c0 tx=0x7f0bd8009260 comp rx=0 tx=0).stop 2024-05-10T04:07:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.211+0000 7f0bf031a700 1 -- 172.21.15.150:0/3423046818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be81031a0 msgr2=0x7f0be8101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.211+0000 7f0bf031a700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be81031a0 0x7f0be8101350 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0be000b070 tx=0x7f0be000bb60 comp rx=0 tx=0).stop 2024-05-10T04:07:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.211+0000 7f0bf031a700 1 -- 172.21.15.150:0/3423046818 shutdown_connections 2024-05-10T04:07:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.211+0000 7f0bf031a700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f0bd40a1120 0x7f0bd40a3630 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.211+0000 7f0bf031a700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be81031a0 0x7f0be8101350 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.211+0000 7f0bf031a700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0be80ffbe0 0x7f0be8102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.211+0000 7f0bf031a700 1 --2- 172.21.15.150:0/3423046818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0be8100530 0x7f0be8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.211+0000 7f0bf031a700 1 -- 172.21.15.150:0/3423046818 >> 172.21.15.150:0/3423046818 conn(0x7f0be80fb8b0 msgr2=0x7f0be8106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.211+0000 7f0bf031a700 1 -- 172.21.15.150:0/3423046818 shutdown_connections 2024-05-10T04:07:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.211+0000 7f0bf031a700 1 -- 172.21.15.150:0/3423046818 wait complete. 2024-05-10T04:07:32.226 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:02:30.076694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:02:30.760710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:02:33.850714+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:07:32.227 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:07:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.361+0000 7fc08edcf700 1 -- 172.21.15.150:0/133898219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0881036c0 msgr2=0x7fc088109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.361+0000 7fc08edcf700 1 --2- 172.21.15.150:0/133898219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0881036c0 0x7fc088109f70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc08000b0a0 tx=0x7fc08000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.361+0000 7fc08edcf700 1 -- 172.21.15.150:0/133898219 shutdown_connections 2024-05-10T04:07:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.361+0000 7fc08edcf700 1 --2- 172.21.15.150:0/133898219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0881036c0 0x7fc088109f70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.361+0000 7fc08edcf700 1 --2- 172.21.15.150:0/133898219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc088101c00 0x7fc088102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.361+0000 7fc08edcf700 1 --2- 172.21.15.150:0/133898219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc088102d20 0x7fc088103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.361+0000 7fc08edcf700 1 -- 172.21.15.150:0/133898219 >> 172.21.15.150:0/133898219 conn(0x7fc0880fd210 msgr2=0x7fc0880ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.361+0000 7fc08edcf700 1 -- 172.21.15.150:0/133898219 shutdown_connections 2024-05-10T04:07:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.361+0000 7fc08edcf700 1 -- 172.21.15.150:0/133898219 wait complete. 2024-05-10T04:07:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.361+0000 7fc08edcf700 1 Processor -- start 2024-05-10T04:07:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.361+0000 7fc08edcf700 1 -- start start 2024-05-10T04:07:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08edcf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc088101c00 0x7fc088197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08edcf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc088102d20 0x7fc088198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08edcf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0881036c0 0x7fc08819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08edcf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08819fad0 con 0x7fc088101c00 2024-05-10T04:07:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08edcf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08819fc40 con 0x7fc088102d20 2024-05-10T04:07:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08edcf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08819fe00 con 0x7fc0881036c0 2024-05-10T04:07:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08cb6b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc088101c00 0x7fc088197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08cb6b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc088101c00 0x7fc088197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48634/0 (socket says 172.21.15.150:48634) 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08cb6b700 1 -- 172.21.15.150:0/232761935 learned_addr learned my addr 172.21.15.150:0/232761935 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc087fff700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc088102d20 0x7fc088198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08d36c700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0881036c0 0x7fc08819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08cb6b700 1 -- 172.21.15.150:0/232761935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0881036c0 msgr2=0x7fc08819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08cb6b700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0881036c0 0x7fc08819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08cb6b700 1 -- 172.21.15.150:0/232761935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc088102d20 msgr2=0x7fc088198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08cb6b700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc088102d20 0x7fc088198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08cb6b700 1 -- 172.21.15.150:0/232761935 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080009d00 con 0x7fc088101c00 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08cb6b700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc088101c00 0x7fc088197ff0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00d950 tx=0x7fc07c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc085ffb700 1 -- 172.21.15.150:0/232761935 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c004a30 con 0x7fc088101c00 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc08edcf700 1 -- 172.21.15.150:0/232761935 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0881a00e0 con 0x7fc088101c00 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc085ffb700 1 -- 172.21.15.150:0/232761935 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c007620 con 0x7fc088101c00 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.362+0000 7fc085ffb700 1 -- 172.21.15.150:0/232761935 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00fa90 con 0x7fc088101c00 2024-05-10T04:07:32.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.363+0000 7fc08edcf700 1 -- 172.21.15.150:0/232761935 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0881a05f0 con 0x7fc088101c00 2024-05-10T04:07:32.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.364+0000 7fc08edcf700 1 -- 172.21.15.150:0/232761935 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc088108e10 con 0x7fc088101c00 2024-05-10T04:07:32.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.365+0000 7fc085ffb700 1 -- 172.21.15.150:0/232761935 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc07c00fc10 con 0x7fc088101c00 2024-05-10T04:07:32.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.365+0000 7fc085ffb700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc0780a11c0 0x7fc0780a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:32.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.366+0000 7fc087fff700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc0780a11c0 0x7fc0780a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:32.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.366+0000 7fc085ffb700 1 -- 172.21.15.150:0/232761935 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc07c0cc680 con 0x7fc088101c00 2024-05-10T04:07:32.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.366+0000 7fc087fff700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc0780a11c0 0x7fc0780a36d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc088199090 tx=0x7fc07400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.370+0000 7fc085ffb700 1 -- 172.21.15.150:0/232761935 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc07c095aa0 con 0x7fc088101c00 2024-05-10T04:07:32.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.569+0000 7fc08edcf700 1 -- 172.21.15.150:0/232761935 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0881087f0 con 0x7fc088101c00 2024-05-10T04:07:32.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.570+0000 7fc085ffb700 1 -- 172.21.15.150:0/232761935 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc07c0990c0 con 0x7fc088101c00 2024-05-10T04:07:32.572 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:07:32.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.572+0000 7fc08edcf700 1 -- 172.21.15.150:0/232761935 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc0780a11c0 msgr2=0x7fc0780a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.572+0000 7fc08edcf700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc0780a11c0 0x7fc0780a36d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc088199090 tx=0x7fc07400b410 comp rx=0 tx=0).stop 2024-05-10T04:07:32.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.572+0000 7fc08edcf700 1 -- 172.21.15.150:0/232761935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc088101c00 msgr2=0x7fc088197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.572+0000 7fc08edcf700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc088101c00 0x7fc088197ff0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00d950 tx=0x7fc07c00dc60 comp rx=0 tx=0).stop 2024-05-10T04:07:32.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.572+0000 7fc08edcf700 1 -- 172.21.15.150:0/232761935 shutdown_connections 2024-05-10T04:07:32.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.572+0000 7fc08edcf700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fc0780a11c0 0x7fc0780a36d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.572+0000 7fc08edcf700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0881036c0 0x7fc08819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.572+0000 7fc08edcf700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc088101c00 0x7fc088197ff0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.572+0000 7fc08edcf700 1 --2- 172.21.15.150:0/232761935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc088102d20 0x7fc088198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.572+0000 7fc08edcf700 1 -- 172.21.15.150:0/232761935 >> 172.21.15.150:0/232761935 conn(0x7fc0880fd210 msgr2=0x7fc088105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:32.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.572+0000 7fc08edcf700 1 -- 172.21.15.150:0/232761935 shutdown_connections 2024-05-10T04:07:32.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.572+0000 7fc08edcf700 1 -- 172.21.15.150:0/232761935 wait complete. 2024-05-10T04:07:32.586 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:32 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3423046818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:07:32.587 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:07:32.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.725+0000 7fdcc61a9700 1 -- 172.21.15.150:0/3471067750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc0107880 msgr2=0x7fdcc0109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.725+0000 7fdcc61a9700 1 --2- 172.21.15.150:0/3471067750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc0107880 0x7fdcc0109d90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009f90 tx=0x7fdcb800b330 comp rx=0 tx=0).stop 2024-05-10T04:07:32.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.725+0000 7fdcc61a9700 1 -- 172.21.15.150:0/3471067750 shutdown_connections 2024-05-10T04:07:32.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.725+0000 7fdcc61a9700 1 --2- 172.21.15.150:0/3471067750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc0107880 0x7fdcc0109d90 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.725+0000 7fdcc61a9700 1 --2- 172.21.15.150:0/3471067750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcc0069090 0x7fdcc0104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.725+0000 7fdcc61a9700 1 --2- 172.21.15.150:0/3471067750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcc0104d10 0x7fdcc01071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.725+0000 7fdcc61a9700 1 -- 172.21.15.150:0/3471067750 >> 172.21.15.150:0/3471067750 conn(0x7fdcc00fc0d0 msgr2=0x7fdcc00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:32.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.725+0000 7fdcc61a9700 1 -- 172.21.15.150:0/3471067750 shutdown_connections 2024-05-10T04:07:32.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.725+0000 7fdcc61a9700 1 -- 172.21.15.150:0/3471067750 wait complete. 2024-05-10T04:07:32.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcc61a9700 1 Processor -- start 2024-05-10T04:07:32.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcc61a9700 1 -- start start 2024-05-10T04:07:32.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcc61a9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcc0069090 0x7fdcc0195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:32.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcc61a9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc0104d10 0x7fdcc0196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:32.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcc61a9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcc0107880 0x7fdcc019d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:32.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcc61a9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc019d8d0 con 0x7fdcc0104d10 2024-05-10T04:07:32.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcc61a9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc019da40 con 0x7fdcc0107880 2024-05-10T04:07:32.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcc61a9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc019dc00 con 0x7fdcc0069090 2024-05-10T04:07:32.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcbeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc0104d10 0x7fdcc0196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:32.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcbeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc0104d10 0x7fdcc0196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48656/0 (socket says 172.21.15.150:48656) 2024-05-10T04:07:32.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcbeffd700 1 -- 172.21.15.150:0/4095047508 learned_addr learned my addr 172.21.15.150:0/4095047508 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcbf7fe700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcc0069090 0x7fdcc0195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcbffff700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcc0107880 0x7fdcc019d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcbeffd700 1 -- 172.21.15.150:0/4095047508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcc0069090 msgr2=0x7fdcc0195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcbeffd700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcc0069090 0x7fdcc0195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcbeffd700 1 -- 172.21.15.150:0/4095047508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcc0107880 msgr2=0x7fdcc019d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.726+0000 7fdcbeffd700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcc0107880 0x7fdcc019d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.727+0000 7fdcbeffd700 1 -- 172.21.15.150:0/4095047508 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb8009c40 con 0x7fdcc0104d10 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.727+0000 7fdcbeffd700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc0104d10 0x7fdcc0196340 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400e9a0 tx=0x7fdcb400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.727+0000 7fdcbcff9700 1 -- 172.21.15.150:0/4095047508 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4009790 con 0x7fdcc0104d10 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.727+0000 7fdcbcff9700 1 -- 172.21.15.150:0/4095047508 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcb400cbd0 con 0x7fdcc0104d10 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.727+0000 7fdcc61a9700 1 -- 172.21.15.150:0/4095047508 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc019dee0 con 0x7fdcc0104d10 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.727+0000 7fdcbcff9700 1 -- 172.21.15.150:0/4095047508 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4018680 con 0x7fdcc0104d10 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.727+0000 7fdcc61a9700 1 -- 172.21.15.150:0/4095047508 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc019e310 con 0x7fdcc0104d10 2024-05-10T04:07:32.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.728+0000 7fdcc61a9700 1 -- 172.21.15.150:0/4095047508 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc00fdc80 con 0x7fdcc0104d10 2024-05-10T04:07:32.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.729+0000 7fdcbcff9700 1 -- 172.21.15.150:0/4095047508 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fdcb400cd40 con 0x7fdcc0104d10 2024-05-10T04:07:32.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.730+0000 7fdcbcff9700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdca80a11c0 0x7fdca80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:32.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.730+0000 7fdcbcff9700 1 -- 172.21.15.150:0/4095047508 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdcb4014070 con 0x7fdcc0104d10 2024-05-10T04:07:32.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.730+0000 7fdcbf7fe700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdca80a11c0 0x7fdca80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:32.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.730+0000 7fdcbf7fe700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdca80a11c0 0x7fdca80a36d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdcc01971f0 tx=0x7fdcb0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:32.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.733+0000 7fdcbcff9700 1 -- 172.21.15.150:0/4095047508 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdcb4096860 con 0x7fdcc0104d10 2024-05-10T04:07:32.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:32 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3423046818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:07:32.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:32 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3423046818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:07:32.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.931+0000 7fdcc61a9700 1 -- 172.21.15.150:0/4095047508 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdcc010b220 con 0x7fdcc0104d10 2024-05-10T04:07:32.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.931+0000 7fdcbcff9700 1 -- 172.21.15.150:0/4095047508 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdcb4099e80 con 0x7fdcc0104d10 2024-05-10T04:07:32.934 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:07:32.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.933+0000 7fdcc61a9700 1 -- 172.21.15.150:0/4095047508 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdca80a11c0 msgr2=0x7fdca80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.933+0000 7fdcc61a9700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdca80a11c0 0x7fdca80a36d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdcc01971f0 tx=0x7fdcb0009310 comp rx=0 tx=0).stop 2024-05-10T04:07:32.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.933+0000 7fdcc61a9700 1 -- 172.21.15.150:0/4095047508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc0104d10 msgr2=0x7fdcc0196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:32.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.933+0000 7fdcc61a9700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc0104d10 0x7fdcc0196340 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400e9a0 tx=0x7fdcb400ed60 comp rx=0 tx=0).stop 2024-05-10T04:07:32.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.934+0000 7fdcc61a9700 1 -- 172.21.15.150:0/4095047508 shutdown_connections 2024-05-10T04:07:32.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.934+0000 7fdcc61a9700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fdca80a11c0 0x7fdca80a36d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.934+0000 7fdcc61a9700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcc0107880 0x7fdcc019d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.934+0000 7fdcc61a9700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcc0069090 0x7fdcc0195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.934+0000 7fdcc61a9700 1 --2- 172.21.15.150:0/4095047508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc0104d10 0x7fdcc0196340 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:32.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.934+0000 7fdcc61a9700 1 -- 172.21.15.150:0/4095047508 >> 172.21.15.150:0/4095047508 conn(0x7fdcc00fc0d0 msgr2=0x7fdcc0103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:32.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.934+0000 7fdcc61a9700 1 -- 172.21.15.150:0/4095047508 shutdown_connections 2024-05-10T04:07:32.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:32.934+0000 7fdcc61a9700 1 -- 172.21.15.150:0/4095047508 wait complete. 2024-05-10T04:07:32.949 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:07:33.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.082+0000 7fcbb655a700 1 -- 172.21.15.150:0/1717631132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb01031a0 msgr2=0x7fcbb01035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:33.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.082+0000 7fcbb655a700 1 --2- 172.21.15.150:0/1717631132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb01031a0 0x7fcbb01035b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009990 tx=0x7fcba0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.082+0000 7fcbb655a700 1 -- 172.21.15.150:0/1717631132 shutdown_connections 2024-05-10T04:07:33.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.082+0000 7fcbb655a700 1 --2- 172.21.15.150:0/1717631132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb0100530 0x7fcbb010a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.082+0000 7fcbb655a700 1 --2- 172.21.15.150:0/1717631132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb01031a0 0x7fcbb01035b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.082+0000 7fcbb655a700 1 --2- 172.21.15.150:0/1717631132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbb00ffbe0 0x7fcbb00ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.082+0000 7fcbb655a700 1 -- 172.21.15.150:0/1717631132 >> 172.21.15.150:0/1717631132 conn(0x7fcbb00fb8b0 msgr2=0x7fcbb00fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:33.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.082+0000 7fcbb655a700 1 -- 172.21.15.150:0/1717631132 shutdown_connections 2024-05-10T04:07:33.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.082+0000 7fcbb655a700 1 -- 172.21.15.150:0/1717631132 wait complete. 2024-05-10T04:07:33.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.083+0000 7fcbb655a700 1 Processor -- start 2024-05-10T04:07:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.083+0000 7fcbb655a700 1 -- start start 2024-05-10T04:07:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.083+0000 7fcbb655a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb00ffbe0 0x7fcbb0197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.083+0000 7fcbb655a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbb0100530 0x7fcbb01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.083+0000 7fcbb655a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb01031a0 0x7fcbb019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.083+0000 7fcbb655a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb019f990 con 0x7fcbb00ffbe0 2024-05-10T04:07:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.083+0000 7fcbb655a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb019fb00 con 0x7fcbb0100530 2024-05-10T04:07:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.083+0000 7fcbb655a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb019fcc0 con 0x7fcbb01031a0 2024-05-10T04:07:33.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.083+0000 7fcbaffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb00ffbe0 0x7fcbb0197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:33.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.083+0000 7fcbaffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb00ffbe0 0x7fcbb0197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48666/0 (socket says 172.21.15.150:48666) 2024-05-10T04:07:33.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.083+0000 7fcbaffff700 1 -- 172.21.15.150:0/705674158 learned_addr learned my addr 172.21.15.150:0/705674158 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:33.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.083+0000 7fcbb4af7700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb01031a0 0x7fcbb019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:33.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbaf7fe700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbb0100530 0x7fcbb01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:33.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbaffff700 1 -- 172.21.15.150:0/705674158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb01031a0 msgr2=0x7fcbb019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:33.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbaffff700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb01031a0 0x7fcbb019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbaffff700 1 -- 172.21.15.150:0/705674158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbb0100530 msgr2=0x7fcbb01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:33.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbaffff700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbb0100530 0x7fcbb01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbaffff700 1 -- 172.21.15.150:0/705674158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00095f0 con 0x7fcbb00ffbe0 2024-05-10T04:07:33.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbb4af7700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb01031a0 0x7fcbb019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:07:33.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbaf7fe700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbb0100530 0x7fcbb01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:07:33.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbaffff700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb00ffbe0 0x7fcbb0197eb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcba0000c00 tx=0x7fcba0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbad7fa700 1 -- 172.21.15.150:0/705674158 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba001d070 con 0x7fcbb00ffbe0 2024-05-10T04:07:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbad7fa700 1 -- 172.21.15.150:0/705674158 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba000fc00 con 0x7fcbb00ffbe0 2024-05-10T04:07:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbb655a700 1 -- 172.21.15.150:0/705674158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb019ff40 con 0x7fcbb00ffbe0 2024-05-10T04:07:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbad7fa700 1 -- 172.21.15.150:0/705674158 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0017650 con 0x7fcbb00ffbe0 2024-05-10T04:07:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.084+0000 7fcbb655a700 1 -- 172.21.15.150:0/705674158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb01a02b0 con 0x7fcbb00ffbe0 2024-05-10T04:07:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.085+0000 7fcbb655a700 1 -- 172.21.15.150:0/705674158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb0108ff0 con 0x7fcbb00ffbe0 2024-05-10T04:07:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.086+0000 7fcbad7fa700 1 -- 172.21.15.150:0/705674158 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fcba0021410 con 0x7fcbb00ffbe0 2024-05-10T04:07:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.087+0000 7fcbad7fa700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fcb980a11c0 0x7fcb980a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.087+0000 7fcbaf7fe700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fcb980a11c0 0x7fcb980a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.087+0000 7fcbad7fa700 1 -- 172.21.15.150:0/705674158 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcba00ce780 con 0x7fcbb00ffbe0 2024-05-10T04:07:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.088+0000 7fcbaf7fe700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fcb980a11c0 0x7fcb980a36d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0198f50 tx=0x7fcba4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:33.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.091+0000 7fcbad7fa700 1 -- 172.21.15.150:0/705674158 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcba0097c90 con 0x7fcbb00ffbe0 2024-05-10T04:07:33.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.290+0000 7fcbb655a700 1 -- 172.21.15.150:0/705674158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcbb010d3d0 con 0x7fcbb00ffbe0 2024-05-10T04:07:33.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.291+0000 7fcbad7fa700 1 -- 172.21.15.150:0/705674158 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcba009b2b0 con 0x7fcbb00ffbe0 2024-05-10T04:07:33.293 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:07:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.293+0000 7fcbb655a700 1 -- 172.21.15.150:0/705674158 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fcb980a11c0 msgr2=0x7fcb980a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.293+0000 7fcbb655a700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fcb980a11c0 0x7fcb980a36d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0198f50 tx=0x7fcba4009260 comp rx=0 tx=0).stop 2024-05-10T04:07:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.293+0000 7fcbb655a700 1 -- 172.21.15.150:0/705674158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb00ffbe0 msgr2=0x7fcbb0197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.293+0000 7fcbb655a700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb00ffbe0 0x7fcbb0197eb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcba0000c00 tx=0x7fcba0009580 comp rx=0 tx=0).stop 2024-05-10T04:07:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.293+0000 7fcbb655a700 1 -- 172.21.15.150:0/705674158 shutdown_connections 2024-05-10T04:07:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.293+0000 7fcbb655a700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fcb980a11c0 0x7fcb980a36d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.293+0000 7fcbb655a700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb01031a0 0x7fcbb019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.293+0000 7fcbb655a700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb00ffbe0 0x7fcbb0197eb0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.293+0000 7fcbb655a700 1 --2- 172.21.15.150:0/705674158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbb0100530 0x7fcbb01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.293+0000 7fcbb655a700 1 -- 172.21.15.150:0/705674158 >> 172.21.15.150:0/705674158 conn(0x7fcbb00fb8b0 msgr2=0x7fcbb0106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.293+0000 7fcbb655a700 1 -- 172.21.15.150:0/705674158 shutdown_connections 2024-05-10T04:07:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.293+0000 7fcbb655a700 1 -- 172.21.15.150:0/705674158 wait complete. 2024-05-10T04:07:33.308 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:07:33.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.445+0000 7fb786463700 1 -- 172.21.15.150:0/2449788070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb780102900 msgr2=0x7fb780109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:33.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.445+0000 7fb786463700 1 --2- 172.21.15.150:0/2449788070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb780102900 0x7fb780109200 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb77800b0a0 tx=0x7fb77800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.446+0000 7fb786463700 1 -- 172.21.15.150:0/2449788070 shutdown_connections 2024-05-10T04:07:33.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.446+0000 7fb786463700 1 --2- 172.21.15.150:0/2449788070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb780102900 0x7fb780109200 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.446+0000 7fb786463700 1 --2- 172.21.15.150:0/2449788070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7801010c0 0x7fb7801014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.446+0000 7fb786463700 1 --2- 172.21.15.150:0/2449788070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb780101fb0 0x7fb7801023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.446+0000 7fb786463700 1 -- 172.21.15.150:0/2449788070 >> 172.21.15.150:0/2449788070 conn(0x7fb7800fb880 msgr2=0x7fb7800fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.446+0000 7fb786463700 1 -- 172.21.15.150:0/2449788070 shutdown_connections 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.446+0000 7fb786463700 1 -- 172.21.15.150:0/2449788070 wait complete. 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.446+0000 7fb786463700 1 Processor -- start 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.446+0000 7fb786463700 1 -- start start 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb786463700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7801010c0 0x7fb7801007a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb786463700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb780101fb0 0x7fb78019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb786463700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb780102900 0x7fb78019f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb786463700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78019fc30 con 0x7fb780102900 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb786463700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78019fda0 con 0x7fb7801010c0 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb786463700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78019ff60 con 0x7fb780101fb0 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb784a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb780102900 0x7fb78019f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb784a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb780102900 0x7fb78019f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48692/0 (socket says 172.21.15.150:48692) 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb784a00700 1 -- 172.21.15.150:0/2976080329 learned_addr learned my addr 172.21.15.150:0/2976080329 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:33.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb77f7fe700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb780101fb0 0x7fb78019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:33.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb77ffff700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7801010c0 0x7fb7801007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:33.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb784a00700 1 -- 172.21.15.150:0/2976080329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb780101fb0 msgr2=0x7fb78019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:33.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb784a00700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb780101fb0 0x7fb78019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb784a00700 1 -- 172.21.15.150:0/2976080329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7801010c0 msgr2=0x7fb7801007a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:33.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb784a00700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7801010c0 0x7fb7801007a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb784a00700 1 -- 172.21.15.150:0/2976080329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb778009d00 con 0x7fb780102900 2024-05-10T04:07:33.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.447+0000 7fb784a00700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb780102900 0x7fb78019f530 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb778006070 tx=0x7fb778004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:33.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.448+0000 7fb77d7fa700 1 -- 172.21.15.150:0/2976080329 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77800e040 con 0x7fb780102900 2024-05-10T04:07:33.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.448+0000 7fb77d7fa700 1 -- 172.21.15.150:0/2976080329 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb77801db50 con 0x7fb780102900 2024-05-10T04:07:33.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.448+0000 7fb786463700 1 -- 172.21.15.150:0/2976080329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7801a01e0 con 0x7fb780102900 2024-05-10T04:07:33.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.448+0000 7fb77d7fa700 1 -- 172.21.15.150:0/2976080329 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77801bb50 con 0x7fb780102900 2024-05-10T04:07:33.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.448+0000 7fb786463700 1 -- 172.21.15.150:0/2976080329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7801a0660 con 0x7fb780102900 2024-05-10T04:07:33.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.449+0000 7fb786463700 1 -- 172.21.15.150:0/2976080329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7801080a0 con 0x7fb780102900 2024-05-10T04:07:33.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.450+0000 7fb77d7fa700 1 -- 172.21.15.150:0/2976080329 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb77801bcb0 con 0x7fb780102900 2024-05-10T04:07:33.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.450+0000 7fb77d7fa700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb7680a11c0 0x7fb7680a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:33.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.451+0000 7fb77ffff700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb7680a11c0 0x7fb7680a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:33.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.451+0000 7fb77d7fa700 1 -- 172.21.15.150:0/2976080329 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb7780ce7f0 con 0x7fb780102900 2024-05-10T04:07:33.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.451+0000 7fb77ffff700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb7680a11c0 0x7fb7680a36d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb770005950 tx=0x7fb77000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:33.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.455+0000 7fb77d7fa700 1 -- 172.21.15.150:0/2976080329 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb778097d00 con 0x7fb780102900 2024-05-10T04:07:33.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.653+0000 7fb786463700 1 -- 172.21.15.150:0/2976080329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb780068730 con 0x7fb780102900 2024-05-10T04:07:33.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.653+0000 7fb77d7fa700 1 -- 172.21.15.150:0/2976080329 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb780068730 con 0x7fb780102900 2024-05-10T04:07:33.656 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:07:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.656+0000 7fb786463700 1 -- 172.21.15.150:0/2976080329 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb7680a11c0 msgr2=0x7fb7680a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.656+0000 7fb786463700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb7680a11c0 0x7fb7680a36d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb770005950 tx=0x7fb77000a300 comp rx=0 tx=0).stop 2024-05-10T04:07:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.656+0000 7fb786463700 1 -- 172.21.15.150:0/2976080329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb780102900 msgr2=0x7fb78019f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.656+0000 7fb786463700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb780102900 0x7fb78019f530 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb778006070 tx=0x7fb778004c80 comp rx=0 tx=0).stop 2024-05-10T04:07:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.656+0000 7fb786463700 1 -- 172.21.15.150:0/2976080329 shutdown_connections 2024-05-10T04:07:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.656+0000 7fb786463700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fb7680a11c0 0x7fb7680a36d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.656+0000 7fb786463700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb780102900 0x7fb78019f530 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.656+0000 7fb786463700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7801010c0 0x7fb7801007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.656+0000 7fb786463700 1 --2- 172.21.15.150:0/2976080329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb780101fb0 0x7fb78019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.656+0000 7fb786463700 1 -- 172.21.15.150:0/2976080329 >> 172.21.15.150:0/2976080329 conn(0x7fb7800fb880 msgr2=0x7fb7801051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.656+0000 7fb786463700 1 -- 172.21.15.150:0/2976080329 shutdown_connections 2024-05-10T04:07:33.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.656+0000 7fb786463700 1 -- 172.21.15.150:0/2976080329 wait complete. 2024-05-10T04:07:33.670 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:33 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/232761935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:07:33.670 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:33 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4095047508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:07:33.670 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:33 smithi150 ceph-mon[42477]: pgmap v379: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:33.773 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:33 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/705674158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:07:33.774 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:33 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/232761935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:07:33.774 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:33 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4095047508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:07:33.774 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:33 smithi161 ceph-mon[46918]: pgmap v379: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:33.774 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:33 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/705674158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:07:33.774 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:33 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/232761935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:07:33.774 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:33 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4095047508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:07:33.774 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:33 smithi181 ceph-mon[46602]: pgmap v379: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:33.774 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:33 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/705674158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:07:33.775 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-10T04:07:33.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.907+0000 7fea11417700 1 -- 172.21.15.150:0/3914781951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0c107a10 msgr2=0x7fea0c109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:33.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.907+0000 7fea11417700 1 --2- 172.21.15.150:0/3914781951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0c107a10 0x7fea0c109e30 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fea0400b0a0 tx=0x7fea0400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.907+0000 7fea11417700 1 -- 172.21.15.150:0/3914781951 shutdown_connections 2024-05-10T04:07:33.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.907+0000 7fea11417700 1 --2- 172.21.15.150:0/3914781951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0c107a10 0x7fea0c109e30 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.907+0000 7fea11417700 1 --2- 172.21.15.150:0/3914781951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea0c0fff40 0x7fea0c100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.907+0000 7fea11417700 1 --2- 172.21.15.150:0/3914781951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea0c1050b0 0x7fea0c1074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.907+0000 7fea11417700 1 -- 172.21.15.150:0/3914781951 >> 172.21.15.150:0/3914781951 conn(0x7fea0c0fb8b0 msgr2=0x7fea0c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:33.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.907+0000 7fea11417700 1 -- 172.21.15.150:0/3914781951 shutdown_connections 2024-05-10T04:07:33.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.907+0000 7fea11417700 1 -- 172.21.15.150:0/3914781951 wait complete. 2024-05-10T04:07:33.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.907+0000 7fea11417700 1 Processor -- start 2024-05-10T04:07:33.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea11417700 1 -- start start 2024-05-10T04:07:33.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea11417700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0c0fff40 0x7fea0c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:33.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea11417700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea0c1050b0 0x7fea0c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:33.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea11417700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea0c107a10 0x7fea0c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:33.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea11417700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c19fb60 con 0x7fea0c0fff40 2024-05-10T04:07:33.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea11417700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c19fcd0 con 0x7fea0c1050b0 2024-05-10T04:07:33.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea11417700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c19fe90 con 0x7fea0c107a10 2024-05-10T04:07:33.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea0affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0c0fff40 0x7fea0c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:33.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea0affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0c0fff40 0x7fea0c198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48712/0 (socket says 172.21.15.150:48712) 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea0affd700 1 -- 172.21.15.150:0/2307634280 learned_addr learned my addr 172.21.15.150:0/2307634280 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea0b7fe700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea0c107a10 0x7fea0c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea0a7fc700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea0c1050b0 0x7fea0c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea0affd700 1 -- 172.21.15.150:0/2307634280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea0c107a10 msgr2=0x7fea0c19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea0affd700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea0c107a10 0x7fea0c19f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea0affd700 1 -- 172.21.15.150:0/2307634280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea0c1050b0 msgr2=0x7fea0c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea0affd700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea0c1050b0 0x7fea0c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea0affd700 1 -- 172.21.15.150:0/2307634280 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04009d00 con 0x7fea0c0fff40 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea0a7fc700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea0c1050b0 0x7fea0c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea0b7fe700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea0c107a10 0x7fea0c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.908+0000 7fea0affd700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0c0fff40 0x7fea0c198080 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fea0000d980 tx=0x7fea0000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.909+0000 7fe9f3fff700 1 -- 172.21.15.150:0/2307634280 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00009b40 con 0x7fea0c0fff40 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.909+0000 7fe9f3fff700 1 -- 172.21.15.150:0/2307634280 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea0000de60 con 0x7fea0c0fff40 2024-05-10T04:07:33.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.909+0000 7fe9f3fff700 1 -- 172.21.15.150:0/2307634280 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0000f620 con 0x7fea0c0fff40 2024-05-10T04:07:33.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.909+0000 7fea11417700 1 -- 172.21.15.150:0/2307634280 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0c1a0170 con 0x7fea0c0fff40 2024-05-10T04:07:33.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.909+0000 7fea11417700 1 -- 172.21.15.150:0/2307634280 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0c1a0630 con 0x7fea0c0fff40 2024-05-10T04:07:33.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.910+0000 7fea11417700 1 -- 172.21.15.150:0/2307634280 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0c0fd460 con 0x7fea0c0fff40 2024-05-10T04:07:33.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.911+0000 7fe9f3fff700 1 -- 172.21.15.150:0/2307634280 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fea00010460 con 0x7fea0c0fff40 2024-05-10T04:07:33.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.912+0000 7fe9f3fff700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9f40a1120 0x7fe9f40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:33.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.912+0000 7fea0a7fc700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9f40a1120 0x7fe9f40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:33.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.912+0000 7fe9f3fff700 1 -- 172.21.15.150:0/2307634280 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fea000cc7a0 con 0x7fea0c0fff40 2024-05-10T04:07:33.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.912+0000 7fea0a7fc700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9f40a1120 0x7fe9f40a3630 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fea0c199120 tx=0x7fe9fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:33.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:33.916+0000 7fe9f3fff700 1 -- 172.21.15.150:0/2307634280 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fea00095c20 con 0x7fea0c0fff40 2024-05-10T04:07:34.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.173+0000 7fea11417700 1 -- 172.21.15.150:0/2307634280 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fea0c068730 con 0x7fea0c0fff40 2024-05-10T04:07:34.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.181+0000 7fe9f3fff700 1 -- 172.21.15.150:0/2307634280 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733415 (secure 0 0 0) 0x7fea00020090 con 0x7fea0c0fff40 2024-05-10T04:07:34.194 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:07:34.194 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:07:34.194 INFO:teuthology.orchestra.run.smithi150.stdout: "epoch": 18, 2024-05-10T04:07:34.194 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.194 INFO:teuthology.orchestra.run.smithi150.stdout: "active_gid": 14152, 2024-05-10T04:07:34.194 INFO:teuthology.orchestra.run.smithi150.stdout: "active_name": "x", 2024-05-10T04:07:34.194 INFO:teuthology.orchestra.run.smithi150.stdout: "active_addrs": { 2024-05-10T04:07:34.194 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6800", 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1954703650 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6801", 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1954703650 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: "active_addr": "172.21.15.150:6801/1954703650", 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: "active_change": "2024-05-10T03:57:29.013577+0000", 2024-05-10T04:07:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: "active_mgr_features": 4540138322906710015, 2024-05-10T04:07:34.196 INFO:teuthology.orchestra.run.smithi150.stdout: "available": true, 2024-05-10T04:07:34.196 INFO:teuthology.orchestra.run.smithi150.stdout: "standbys": [ 2024-05-10T04:07:34.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.196 INFO:teuthology.orchestra.run.smithi150.stdout: "gid": 24112, 2024-05-10T04:07:34.196 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "z", 2024-05-10T04:07:34.196 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_features": 4540138322906710015, 2024-05-10T04:07:34.196 INFO:teuthology.orchestra.run.smithi150.stdout: "available_modules": [ 2024-05-10T04:07:34.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.196 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "alerts", 2024-05-10T04:07:34.196 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.196 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.196 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.196 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:34.197 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:34.197 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.197 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.197 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.197 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:34.197 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.197 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.197 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.197 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T04:07:34.197 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.197 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.197 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.198 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.198 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.198 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.198 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.198 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.198 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.198 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.198 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.198 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.198 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.198 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.198 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.198 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.199 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.199 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.199 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.199 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.199 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.199 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.199 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.199 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.199 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.199 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.199 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.199 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.200 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.200 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.200 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.200 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.200 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.200 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.200 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.200 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.200 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.200 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.200 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.200 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.201 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.201 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.201 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.201 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.201 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.201 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.201 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.201 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.201 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.201 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.201 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.201 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.201 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.202 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.202 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.202 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.202 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.202 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.202 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.202 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.202 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.202 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.202 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.202 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.202 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.203 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.203 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.203 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.203 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.203 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_destination": { 2024-05-10T04:07:34.203 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_destination", 2024-05-10T04:07:34.203 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.203 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.203 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Email address to send alerts to", 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_from_name": { 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_from_name", 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.204 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.205 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "Ceph", 2024-05-10T04:07:34.205 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.205 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.205 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.205 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Email From: name", 2024-05-10T04:07:34.205 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_host": { 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_host", 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP server", 2024-05-10T04:07:34.206 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_password": { 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_password", 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Password to authenticate with", 2024-05-10T04:07:34.207 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.208 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.208 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.208 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_port": { 2024-05-10T04:07:34.208 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_port", 2024-05-10T04:07:34.208 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.208 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.208 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.208 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "465", 2024-05-10T04:07:34.208 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.208 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.208 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.208 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP port", 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_sender": { 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_sender", 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP envelope sender", 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.209 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.210 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.210 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_ssl": { 2024-05-10T04:07:34.210 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_ssl", 2024-05-10T04:07:34.210 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.210 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.210 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.210 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.210 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.210 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.210 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_user": { 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_user", 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.211 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "User to authenticate as", 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "balancer", 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: "active": { 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "active", 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.212 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "begin_time": { 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "begin_time", 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0000", 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: "begin_weekday": { 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "begin_weekday", 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0", 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "6", 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_max_iterations": { 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_max_iterations", 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.215 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "25", 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "250", 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_metrics": { 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_metrics", 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.216 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_step": { 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_step", 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.217 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0.5", 2024-05-10T04:07:34.218 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0.001", 2024-05-10T04:07:34.218 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "0.999", 2024-05-10T04:07:34.218 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.218 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "aggressiveness of optimization", 2024-05-10T04:07:34.218 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T04:07:34.218 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.218 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.218 INFO:teuthology.orchestra.run.smithi150.stdout: "end_time": { 2024-05-10T04:07:34.218 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "end_time", 2024-05-10T04:07:34.218 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.218 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.218 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "2359", 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: "end_weekday": { 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "end_weekday", 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.219 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0", 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "6", 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.220 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.221 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.222 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.222 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.222 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.222 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.222 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.222 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.222 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.222 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.222 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.222 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.222 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.223 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.224 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: "min_score": { 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "min_score", 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.225 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": { 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mode", 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.226 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "upmap", 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: "crush-compat", 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: "none", 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap" 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Balancer mode", 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.227 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_ids": { 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pool_ids", 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:34.228 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap_max_deviation": { 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "upmap_max_deviation", 2024-05-10T04:07:34.229 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap_max_optimizations": { 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "upmap_max_optimizations", 2024-05-10T04:07:34.230 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.231 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cephadm", 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_down_multiplier": { 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_down_multiplier", 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3.0", 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T04:07:34.232 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_refresh_rate": { 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_refresh_rate", 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "20", 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T04:07:34.233 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_starting_port": { 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_starting_port", 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "4721", 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.234 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_ptrace": { 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_ptrace", 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T04:07:34.235 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "autotune_interval": { 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "autotune_interval", 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.236 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: "autotune_memory_target_ratio": { 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0.7", 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.238 INFO:teuthology.orchestra.run.smithi150.stdout: "cgroups_split": { 2024-05-10T04:07:34.238 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cgroups_split", 2024-05-10T04:07:34.238 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.238 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.238 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.238 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.238 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.238 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.238 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.238 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T04:07:34.238 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.238 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.238 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "config_checks_enabled": { 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_checks_enabled", 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "config_dashboard": { 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_dashboard", 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_alertmanager": { 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_alertmanager", 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_base": { 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_base", 2024-05-10T04:07:34.241 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Container image name, without the tag", 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_elasticsearch": { 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_elasticsearch", 2024-05-10T04:07:34.242 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "elasticsearch container image", 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_grafana": { 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_grafana", 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.243 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_haproxy": { 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_haproxy", 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.244 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "HAproxy container image", 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_agent": { 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_agent", 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.245 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.246 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T04:07:34.246 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.246 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.246 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.246 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger agent container image", 2024-05-10T04:07:34.246 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.246 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.246 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.246 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_collector": { 2024-05-10T04:07:34.246 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_collector", 2024-05-10T04:07:34.246 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.246 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger collector container image", 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_query": { 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_query", 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.247 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger query container image", 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_keepalived": { 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_keepalived", 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.248 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Keepalived container image", 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_loki": { 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_loki", 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.249 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Loki container image", 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_node_exporter": { 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_node_exporter", 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.250 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_nvmeof": { 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_nvmeof", 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.251 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Nvme-of container image", 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_prometheus": { 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_prometheus", 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.252 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_promtail": { 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_promtail", 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T04:07:34.253 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Promtail container image", 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_snmp_gateway": { 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_snmp_gateway", 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T04:07:34.254 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SNMP Gateway container image", 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "container_init": { 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_init", 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.255 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "daemon_cache_timeout": { 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "daemon_cache_timeout", 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.256 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "default_cephadm_command_timeout": { 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "900", 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T04:07:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.258 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.258 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.258 INFO:teuthology.orchestra.run.smithi150.stdout: "default_registry": { 2024-05-10T04:07:34.258 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "default_registry", 2024-05-10T04:07:34.258 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.258 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.258 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.258 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io", 2024-05-10T04:07:34.258 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.258 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.258 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.258 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: "device_cache_timeout": { 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_cache_timeout", 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1800", 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.259 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache device inventory", 2024-05-10T04:07:34.260 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.260 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.260 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.260 INFO:teuthology.orchestra.run.smithi150.stdout: "device_enhanced_scan": { 2024-05-10T04:07:34.260 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_enhanced_scan", 2024-05-10T04:07:34.260 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.260 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.260 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.260 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.260 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.260 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: "facts_cache_timeout": { 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "facts_cache_timeout", 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.261 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache host facts data", 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: "host_check_interval": { 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "host_check_interval", 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.262 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to perform a host check", 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: "hw_monitoring": { 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "hw_monitoring", 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.263 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: "inventory_list_all": { 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "inventory_list_all", 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.264 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: "log_refresh_metadata": { 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_refresh_metadata", 2024-05-10T04:07:34.266 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T04:07:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.268 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.269 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.270 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.270 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.270 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.270 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.270 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.270 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.270 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.270 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.270 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.270 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.270 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.271 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "*", 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.272 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "max_count_per_host": { 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_count_per_host", 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "max number of daemons per service per host", 2024-05-10T04:07:34.273 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: "max_osd_draining_count": { 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_osd_draining_count", 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T04:07:34.274 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: "migration_current": { 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "migration_current", 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "internal - do not modify", 2024-05-10T04:07:34.275 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": { 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mode", 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "root", 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm-package", 2024-05-10T04:07:34.276 INFO:teuthology.orchestra.run.smithi150.stdout: "root" 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: "oob_default_addr": { 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "oob_default_addr", 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "169.254.1.1", 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.277 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.278 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.278 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T04:07:34.278 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.278 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.278 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.278 INFO:teuthology.orchestra.run.smithi150.stdout: "prometheus_alerts_path": { 2024-05-10T04:07:34.278 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prometheus_alerts_path", 2024-05-10T04:07:34.278 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.278 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.278 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.278 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T04:07:34.278 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_insecure": { 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_insecure", 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.280 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.280 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.280 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T04:07:34.280 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.280 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.280 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.280 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_password": { 2024-05-10T04:07:34.280 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_password", 2024-05-10T04:07:34.280 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.280 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.280 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.280 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_url": { 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_url", 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.281 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_username": { 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_username", 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.282 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.283 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.283 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.283 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T04:07:34.283 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.283 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.283 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.283 INFO:teuthology.orchestra.run.smithi150.stdout: "secure_monitoring_stack": { 2024-05-10T04:07:34.283 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "secure_monitoring_stack", 2024-05-10T04:07:34.283 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.283 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.283 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "service_discovery_port": { 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "service_discovery_port", 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.284 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8765", 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "cephadm service discovery port", 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: "ssh_config_file": { 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssh_config_file", 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.285 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T04:07:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: "use_agent": { 2024-05-10T04:07:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "use_agent", 2024-05-10T04:07:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T04:07:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: "use_repo_digest": { 2024-05-10T04:07:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "use_repo_digest", 2024-05-10T04:07:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_failed_host_check": { 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_failed_host_check", 2024-05-10T04:07:34.288 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_stray_daemons": { 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_stray_daemons", 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.289 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_stray_hosts": { 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_stray_hosts", 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.290 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crash", 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.291 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.292 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.293 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.294 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "retain_interval": { 2024-05-10T04:07:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "retain_interval", 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "31536000", 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_recent_interval": { 2024-05-10T04:07:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_recent_interval", 2024-05-10T04:07:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1209600", 2024-05-10T04:07:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T04:07:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "dashboard", 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "AUDIT_API_ENABLED": { 2024-05-10T04:07:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T04:07:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T04:07:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T04:07:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T04:07:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T04:07:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T04:07:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T04:07:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T04:07:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T04:07:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T04:07:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T04:07:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_URL": { 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_URL", 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T04:07:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T04:07:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T04:07:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T04:07:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T04:07:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8", 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T04:07:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T04:07:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "45", 2024-05-10T04:07:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T04:07:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:07:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "cross_origin_url": { 2024-05-10T04:07:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cross_origin_url", 2024-05-10T04:07:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "crt_file": { 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crt_file", 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "debug": { 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "debug", 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable/disable debug options", 2024-05-10T04:07:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "jwt_token_ttl": { 2024-05-10T04:07:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "jwt_token_ttl", 2024-05-10T04:07:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "28800", 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "key_file": { 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "key_file", 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "motd": { 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "motd", 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "The message of the day", 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "redirect_resolve_ip_addr": { 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:07:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:07:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "::", 2024-05-10T04:07:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8080", 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl": { 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl", 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl_server_port": { 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl_server_port", 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8443", 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_behaviour": { 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_behaviour", 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "redirect", 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "redirect" 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_error_status_code": { 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_error_status_code", 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "500", 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "400", 2024-05-10T04:07:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "599", 2024-05-10T04:07:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "url_prefix": { 2024-05-10T04:07:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "url_prefix", 2024-05-10T04:07:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "devicehealth", 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "enable_monitoring": { 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enable_monitoring", 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "monitor device health metrics", 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "mark_out_threshold": { 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mark_out_threshold", 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "2419200", 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": { 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pool_name", 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "device_health_metrics", 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "retention_period": { 2024-05-10T04:07:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "retention_period", 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15552000", 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long to retain device health metrics", 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "scrape_frequency": { 2024-05-10T04:07:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "scrape_frequency", 2024-05-10T04:07:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "86400", 2024-05-10T04:07:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T04:07:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "self_heal": { 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "self_heal", 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_threshold": { 2024-05-10T04:07:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_threshold", 2024-05-10T04:07:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "7257600", 2024-05-10T04:07:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T04:07:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "diskprediction_local", 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "predict_interval": { 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "predict_interval", 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "86400", 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "predictor_model": { 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "predictor_model", 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "prophetstor", 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "influx", 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": false, 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "influxdb python module not found", 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "batch_size": { 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "batch_size", 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5000", 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "database": { 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "database", 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "ceph", 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "hostname": { 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "hostname", 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB server hostname", 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "30", 2024-05-10T04:07:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "5", 2024-05-10T04:07:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T04:07:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "password": { 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "password", 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "password of InfluxDB server user", 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "port": { 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "port", 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8086", 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB server port", 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl": { 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl", 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "false", 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "threads": { 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "threads", 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:07:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "32", 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "username": { 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "username", 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "username of InfluxDB server user", 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "verify_ssl": { 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "verify_ssl", 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "true", 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "insights", 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "iostat", 2024-05-10T04:07:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "k8sevents", 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_event_retention_days": { 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ceph_event_retention_days", 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "7", 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "config_check_secs": { 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_check_secs", 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "10", 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "localpool", 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "failure_domain": { 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "failure_domain", 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "host", 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "failure domain for any created local pool", 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": { 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "min_size", 2024-05-10T04:07:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default min_size for any created local pool", 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "num_rep": { 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "num_rep", 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3", 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default replica count for any created local pool", 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": { 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pg_num", 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "128", 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "prefix": { 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prefix", 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "name prefix for any created local pool", 2024-05-10T04:07:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "subtree": { 2024-05-10T04:07:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "subtree", 2024-05-10T04:07:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "rack", 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mds_autoscaler", 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mirroring", 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "nfs", 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "orchestrator", 2024-05-10T04:07:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "fail_fs": { 2024-05-10T04:07:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "fail_fs", 2024-05-10T04:07:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator": { 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "orchestrator", 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm", 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "rook", 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "test_orchestrator" 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Orchestrator backend", 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "osd_perf_query", 2024-05-10T04:07:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "osd_support", 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pg_autoscaler", 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.469 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.470 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.471 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:34.471 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.471 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.471 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.471 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.471 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.471 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.471 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.471 INFO:teuthology.orchestra.run.smithi150.stdout: "threshold": { 2024-05-10T04:07:34.471 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "threshold", 2024-05-10T04:07:34.471 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.471 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3.0", 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1.0", 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "scaling threshold", 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "progress", 2024-05-10T04:07:34.472 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_pg_recovery_event": { 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_pg_recovery_event", 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "enabled": { 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enabled", 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.474 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.475 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.476 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.477 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.478 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: "max_completed_events": { 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_completed_events", 2024-05-10T04:07:34.479 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "50", 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "number of past completed events to remember", 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.480 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.481 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.481 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:07:34.481 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.481 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.481 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.481 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long the module is going to sleep", 2024-05-10T04:07:34.481 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.481 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.481 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.481 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.481 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.482 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prometheus", 2024-05-10T04:07:34.482 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.482 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.482 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.482 INFO:teuthology.orchestra.run.smithi150.stdout: "cache": { 2024-05-10T04:07:34.482 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cache", 2024-05-10T04:07:34.482 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.482 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.482 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.482 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.482 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.482 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.482 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.483 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.483 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.483 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.483 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.483 INFO:teuthology.orchestra.run.smithi150.stdout: "exclude_perf_counters": { 2024-05-10T04:07:34.483 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "exclude_perf_counters", 2024-05-10T04:07:34.483 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.483 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.483 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.483 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.483 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.483 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.484 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.484 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T04:07:34.484 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T04:07:34.484 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.484 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.484 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.484 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.484 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.484 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.484 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.485 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.486 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.486 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.486 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.486 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.486 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.486 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.486 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.486 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.486 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.486 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.486 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.486 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.487 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.487 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.487 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.487 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.487 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.487 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.487 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.487 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.487 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.487 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.487 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.487 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.488 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.488 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.488 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.488 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.488 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.488 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.488 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.488 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.488 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.488 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.488 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.488 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_stats_pools": { 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_stats_pools", 2024-05-10T04:07:34.489 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T04:07:34.490 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "300", 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: "scrape_interval": { 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "scrape_interval", 2024-05-10T04:07:34.491 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15.0", 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:07:34.492 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.493 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.493 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.493 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "::", 2024-05-10T04:07:34.493 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.493 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.493 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.493 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T04:07:34.493 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.493 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.493 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.493 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:07:34.493 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "9283", 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: "stale_cache_strategy": { 2024-05-10T04:07:34.494 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "stale_cache_strategy", 2024-05-10T04:07:34.495 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.495 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.495 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.495 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "log", 2024-05-10T04:07:34.495 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.495 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.495 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.495 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.495 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.495 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.495 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.495 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_behaviour": { 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_behaviour", 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "default", 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "default", 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "error" 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.496 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.497 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_error_status_code": { 2024-05-10T04:07:34.497 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_error_status_code", 2024-05-10T04:07:34.497 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.497 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.497 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.497 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "500", 2024-05-10T04:07:34.497 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "400", 2024-05-10T04:07:34.497 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "599", 2024-05-10T04:07:34.497 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.497 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.497 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.497 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_support", 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.499 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.501 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.502 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: "max_concurrent_snap_create": { 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_concurrent_snap_create", 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.503 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "mirror_snapshot_schedule": { 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.504 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "trash_purge_schedule": { 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "trash_purge_schedule", 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "restful", 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: "enable_auth": { 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enable_auth", 2024-05-10T04:07:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "key_file": { 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "key_file", 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.507 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.508 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.509 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.511 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.511 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.511 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.511 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.511 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.511 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.511 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.511 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.511 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.511 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.512 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.513 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.514 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rgw", 2024-05-10T04:07:34.515 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.516 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.520 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rook", 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.521 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.522 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.523 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.524 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.526 INFO:teuthology.orchestra.run.smithi150.stdout: "storage_class": { 2024-05-10T04:07:34.526 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "storage_class", 2024-05-10T04:07:34.526 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.526 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.526 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.526 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "local", 2024-05-10T04:07:34.526 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.526 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.526 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.526 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T04:07:34.526 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.526 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "selftest", 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.527 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.528 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.530 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "roption1": { 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "roption1", 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.532 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.533 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.533 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.533 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.533 INFO:teuthology.orchestra.run.smithi150.stdout: "roption2": { 2024-05-10T04:07:34.533 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "roption2", 2024-05-10T04:07:34.533 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.533 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.533 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.533 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "xyz", 2024-05-10T04:07:34.533 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.533 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.533 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.534 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.534 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.534 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.534 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption1": { 2024-05-10T04:07:34.534 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption1", 2024-05-10T04:07:34.534 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.534 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.534 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.534 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.534 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.534 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption2": { 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption2", 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.535 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption3": { 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption3", 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.536 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption4": { 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption4", 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.537 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.538 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.538 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.538 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption5": { 2024-05-10T04:07:34.538 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption5", 2024-05-10T04:07:34.538 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.538 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.538 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.538 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.538 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.538 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption6": { 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption6", 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.539 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption7": { 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption7", 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "42", 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "testkey": { 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testkey", 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.541 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: "testlkey": { 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testlkey", 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.542 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "testnewline": { 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testnewline", 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.543 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snap_schedule", 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_m_granularity": { 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_m_granularity", 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.544 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: "dump_on_update": { 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "dump_on_update", 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.545 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "dump database to debug log on update", 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.546 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.547 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.549 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.550 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.551 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.551 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.551 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.551 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "stats", 2024-05-10T04:07:34.551 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.551 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.551 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.551 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.551 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.551 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.551 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.552 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.553 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.554 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.555 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "status", 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.556 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.558 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.559 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.560 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.561 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "telegraf", 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "address": { 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "address", 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.562 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15", 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.563 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.564 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.565 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.565 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.565 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.565 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.565 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.565 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.565 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.565 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.565 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.565 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.565 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.565 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.566 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.568 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.568 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.568 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.568 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.568 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.568 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.568 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.568 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.568 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.568 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.568 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.568 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.568 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "telemetry", 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_basic": { 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_basic", 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.569 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_crash": { 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_crash", 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.570 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_device": { 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_device", 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.571 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_ident": { 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_ident", 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.572 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_perf": { 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_perf", 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.573 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: "contact": { 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "contact", 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.574 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: "description": { 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "description", 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.575 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "device_url": { 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_url", 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "enabled": { 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enabled", 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.577 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.578 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.578 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.578 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.578 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:34.578 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:34.578 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.578 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.578 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.578 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "24", 2024-05-10T04:07:34.578 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "8", 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "last_opt_revision": { 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "last_opt_revision", 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1", 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.579 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: "leaderboard": { 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "leaderboard", 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.580 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "leaderboard_description": { 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "leaderboard_description", 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.581 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.583 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.584 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.585 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.586 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: "organization": { 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "organization", 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.587 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "proxy": { 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "proxy", 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.588 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "url": { 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "url", 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "test_orchestrator", 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.590 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.591 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.592 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.592 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.592 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.592 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.592 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.592 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.592 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.592 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.592 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.592 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.592 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.592 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.593 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.594 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "volumes", 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.595 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.596 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.597 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.598 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.599 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "max_concurrent_clones": { 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_concurrent_clones", 2024-05-10T04:07:34.600 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "4", 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "snapshot_clone_delay": { 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snapshot_clone_delay", 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.601 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "snapshot_clone_no_wait": { 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.602 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix", 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.603 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "discovery_interval": { 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "discovery_interval", 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "100", 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.605 INFO:teuthology.orchestra.run.smithi150.stdout: "identifier": { 2024-05-10T04:07:34.605 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "identifier", 2024-05-10T04:07:34.605 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.605 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.605 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.605 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.605 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.605 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.605 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.605 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.605 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.605 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.606 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.608 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.609 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.610 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.610 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.610 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.610 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.610 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.610 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.610 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.610 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.610 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.610 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.610 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.610 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.610 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.611 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.611 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.611 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.611 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.611 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.611 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.611 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.611 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.611 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.611 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.611 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.611 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_host": { 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_host", 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.612 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.613 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.613 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.613 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.613 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_port": { 2024-05-10T04:07:34.613 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_port", 2024-05-10T04:07:34.613 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.613 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.613 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.613 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10051", 2024-05-10T04:07:34.613 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.613 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.613 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_sender": { 2024-05-10T04:07:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_sender", 2024-05-10T04:07:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T04:07:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: "gid": 24116, 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "y", 2024-05-10T04:07:34.615 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_features": 4540138322906710015, 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "available_modules": [ 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "alerts", 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.616 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.617 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.618 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.619 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.619 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.619 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.619 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.619 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.619 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.619 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.619 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.619 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.619 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.619 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.619 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.620 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.621 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_destination": { 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_destination", 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.622 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Email address to send alerts to", 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_from_name": { 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_from_name", 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "Ceph", 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Email From: name", 2024-05-10T04:07:34.623 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_host": { 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_host", 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP server", 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.624 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_password": { 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_password", 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Password to authenticate with", 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_port": { 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_port", 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "465", 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP port", 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_sender": { 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_sender", 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP envelope sender", 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_ssl": { 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_ssl", 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_user": { 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_user", 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "User to authenticate as", 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.630 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "balancer", 2024-05-10T04:07:34.630 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.630 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.630 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.630 INFO:teuthology.orchestra.run.smithi150.stdout: "active": { 2024-05-10T04:07:34.630 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "active", 2024-05-10T04:07:34.630 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.630 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.630 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.630 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.630 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "begin_time": { 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "begin_time", 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0000", 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.631 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: "begin_weekday": { 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "begin_weekday", 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0", 2024-05-10T04:07:34.632 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "6", 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_max_iterations": { 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_max_iterations", 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "25", 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "250", 2024-05-10T04:07:34.633 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.634 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T04:07:34.634 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.634 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.634 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.634 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_metrics": { 2024-05-10T04:07:34.634 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_metrics", 2024-05-10T04:07:34.634 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.634 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.634 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.634 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T04:07:34.634 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.634 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_step": { 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_step", 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0.5", 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0.001", 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "0.999", 2024-05-10T04:07:34.635 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "aggressiveness of optimization", 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: "end_time": { 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "end_time", 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "2359", 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.636 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "end_weekday": { 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "end_weekday", 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0", 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "6", 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.637 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.638 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.639 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.640 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.641 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.642 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.643 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.643 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.643 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.643 INFO:teuthology.orchestra.run.smithi150.stdout: "min_score": { 2024-05-10T04:07:34.643 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "min_score", 2024-05-10T04:07:34.643 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.643 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.643 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.643 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:34.643 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.644 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.644 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.644 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T04:07:34.644 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.644 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.644 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.646 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": { 2024-05-10T04:07:34.646 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mode", 2024-05-10T04:07:34.646 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.646 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.646 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.646 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "upmap", 2024-05-10T04:07:34.646 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.646 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.646 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.646 INFO:teuthology.orchestra.run.smithi150.stdout: "crush-compat", 2024-05-10T04:07:34.646 INFO:teuthology.orchestra.run.smithi150.stdout: "none", 2024-05-10T04:07:34.646 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap" 2024-05-10T04:07:34.646 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.647 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Balancer mode", 2024-05-10T04:07:34.647 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.647 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.647 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.647 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_ids": { 2024-05-10T04:07:34.647 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pool_ids", 2024-05-10T04:07:34.647 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.647 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.647 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.647 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.647 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.647 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.648 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap_max_deviation": { 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "upmap_max_deviation", 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:07:34.649 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap_max_optimizations": { 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "upmap_max_optimizations", 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.650 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.651 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.651 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T04:07:34.651 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.651 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.651 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.651 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cephadm", 2024-05-10T04:07:34.651 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.651 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.651 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_down_multiplier": { 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_down_multiplier", 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3.0", 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_refresh_rate": { 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_refresh_rate", 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "20", 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_starting_port": { 2024-05-10T04:07:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_starting_port", 2024-05-10T04:07:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "4721", 2024-05-10T04:07:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T04:07:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.655 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.655 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_ptrace": { 2024-05-10T04:07:34.655 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_ptrace", 2024-05-10T04:07:34.655 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.655 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.655 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.655 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.655 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.655 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.655 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.655 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T04:07:34.655 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T04:07:34.656 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.656 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.656 INFO:teuthology.orchestra.run.smithi150.stdout: "autotune_interval": { 2024-05-10T04:07:34.656 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "autotune_interval", 2024-05-10T04:07:34.656 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.656 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.656 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.656 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:34.656 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.656 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.656 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.656 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: "autotune_memory_target_ratio": { 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0.7", 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.657 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T04:07:34.658 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.658 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.658 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.658 INFO:teuthology.orchestra.run.smithi150.stdout: "cgroups_split": { 2024-05-10T04:07:34.658 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cgroups_split", 2024-05-10T04:07:34.658 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.658 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.658 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.658 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.658 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.658 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.658 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: "config_checks_enabled": { 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_checks_enabled", 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T04:07:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: "config_dashboard": { 2024-05-10T04:07:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_dashboard", 2024-05-10T04:07:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.660 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_alertmanager": { 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_alertmanager", 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.661 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_base": { 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_base", 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.662 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Container image name, without the tag", 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_elasticsearch": { 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_elasticsearch", 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.663 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "elasticsearch container image", 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_grafana": { 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_grafana", 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.664 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_haproxy": { 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_haproxy", 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "HAproxy container image", 2024-05-10T04:07:34.665 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_agent": { 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_agent", 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger agent container image", 2024-05-10T04:07:34.666 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_collector": { 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_collector", 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger collector container image", 2024-05-10T04:07:34.667 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_query": { 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_query", 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger query container image", 2024-05-10T04:07:34.668 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.669 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.669 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.669 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_keepalived": { 2024-05-10T04:07:34.669 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_keepalived", 2024-05-10T04:07:34.669 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.669 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.669 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.669 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T04:07:34.669 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.669 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.669 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.669 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Keepalived container image", 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_loki": { 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_loki", 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.670 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Loki container image", 2024-05-10T04:07:34.671 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.671 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.671 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.671 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_node_exporter": { 2024-05-10T04:07:34.671 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_node_exporter", 2024-05-10T04:07:34.671 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.671 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.671 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.671 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T04:07:34.671 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.671 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.671 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.672 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:07:34.672 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.672 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.672 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.672 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_nvmeof": { 2024-05-10T04:07:34.672 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_nvmeof", 2024-05-10T04:07:34.672 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.672 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.672 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.672 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-10T04:07:34.672 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.672 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Nvme-of container image", 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_prometheus": { 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_prometheus", 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.673 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:07:34.674 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.674 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.674 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.674 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_promtail": { 2024-05-10T04:07:34.674 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_promtail", 2024-05-10T04:07:34.674 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.674 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.674 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.674 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T04:07:34.674 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.674 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.674 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Promtail container image", 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_snmp_gateway": { 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_snmp_gateway", 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.675 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SNMP Gateway container image", 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: "container_init": { 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_init", 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.676 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: "daemon_cache_timeout": { 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "daemon_cache_timeout", 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.677 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: "default_cephadm_command_timeout": { 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "900", 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.678 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.679 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T04:07:34.679 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.679 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.679 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.679 INFO:teuthology.orchestra.run.smithi150.stdout: "default_registry": { 2024-05-10T04:07:34.679 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "default_registry", 2024-05-10T04:07:34.679 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.679 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.679 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.679 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io", 2024-05-10T04:07:34.679 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "device_cache_timeout": { 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_cache_timeout", 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1800", 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.680 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.681 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.681 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache device inventory", 2024-05-10T04:07:34.681 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.681 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.681 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.681 INFO:teuthology.orchestra.run.smithi150.stdout: "device_enhanced_scan": { 2024-05-10T04:07:34.681 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_enhanced_scan", 2024-05-10T04:07:34.681 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.681 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.681 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.681 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.681 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.682 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.682 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.682 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T04:07:34.682 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.682 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.682 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.682 INFO:teuthology.orchestra.run.smithi150.stdout: "facts_cache_timeout": { 2024-05-10T04:07:34.682 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "facts_cache_timeout", 2024-05-10T04:07:34.682 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.682 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.682 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache host facts data", 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: "host_check_interval": { 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "host_check_interval", 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.683 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.684 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:34.684 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.684 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.684 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.684 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to perform a host check", 2024-05-10T04:07:34.684 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.684 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.684 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.684 INFO:teuthology.orchestra.run.smithi150.stdout: "hw_monitoring": { 2024-05-10T04:07:34.684 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "hw_monitoring", 2024-05-10T04:07:34.684 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.684 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T04:07:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: "inventory_list_all": { 2024-05-10T04:07:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "inventory_list_all", 2024-05-10T04:07:34.685 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.686 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.686 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.686 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.686 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.686 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.686 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.686 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T04:07:34.686 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.686 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.686 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.686 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.687 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.687 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.687 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.687 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.687 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.687 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.687 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.687 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.687 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.687 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.687 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.687 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.687 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.688 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.688 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.688 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.688 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.688 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.688 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.688 INFO:teuthology.orchestra.run.smithi150.stdout: "log_refresh_metadata": { 2024-05-10T04:07:34.688 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_refresh_metadata", 2024-05-10T04:07:34.688 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.688 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.688 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.688 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.689 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.690 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.690 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.690 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.690 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T04:07:34.690 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.690 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.690 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.690 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.690 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.690 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.690 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.690 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.691 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "*", 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.694 INFO:teuthology.orchestra.run.smithi150.stdout: "max_count_per_host": { 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_count_per_host", 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "max number of daemons per service per host", 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "max_osd_draining_count": { 2024-05-10T04:07:34.695 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_osd_draining_count", 2024-05-10T04:07:34.696 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.696 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.696 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.696 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.696 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.696 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.696 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.696 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T04:07:34.696 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.696 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.696 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.696 INFO:teuthology.orchestra.run.smithi150.stdout: "migration_current": { 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "migration_current", 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "internal - do not modify", 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.697 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": { 2024-05-10T04:07:34.698 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mode", 2024-05-10T04:07:34.698 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.698 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.698 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.698 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "root", 2024-05-10T04:07:34.698 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.698 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.698 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.698 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm-package", 2024-05-10T04:07:34.698 INFO:teuthology.orchestra.run.smithi150.stdout: "root" 2024-05-10T04:07:34.698 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.698 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "oob_default_addr": { 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "oob_default_addr", 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "169.254.1.1", 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T04:07:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "prometheus_alerts_path": { 2024-05-10T04:07:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prometheus_alerts_path", 2024-05-10T04:07:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T04:07:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T04:07:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_insecure": { 2024-05-10T04:07:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_insecure", 2024-05-10T04:07:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T04:07:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_password": { 2024-05-10T04:07:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_password", 2024-05-10T04:07:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_url": { 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_url", 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_username": { 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_username", 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "secure_monitoring_stack": { 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "secure_monitoring_stack", 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "service_discovery_port": { 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "service_discovery_port", 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8765", 2024-05-10T04:07:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "cephadm service discovery port", 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "ssh_config_file": { 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssh_config_file", 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "use_agent": { 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "use_agent", 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "use_repo_digest": { 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "use_repo_digest", 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_failed_host_check": { 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_failed_host_check", 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_stray_daemons": { 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_stray_daemons", 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_stray_hosts": { 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_stray_hosts", 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crash", 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "retain_interval": { 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "retain_interval", 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "31536000", 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_recent_interval": { 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_recent_interval", 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1209600", 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "dashboard", 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T04:07:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T04:07:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T04:07:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T04:07:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T04:07:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T04:07:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "AUDIT_API_ENABLED": { 2024-05-10T04:07:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T04:07:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T04:07:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T04:07:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:07:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_URL": { 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_URL", 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T04:07:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T04:07:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:07:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T04:07:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T04:07:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T04:07:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T04:07:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T04:07:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T04:07:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T04:07:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T04:07:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T04:07:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T04:07:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T04:07:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T04:07:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T04:07:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T04:07:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T04:07:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T04:07:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T04:07:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T04:07:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8", 2024-05-10T04:07:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T04:07:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T04:07:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "45", 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T04:07:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T04:07:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "cross_origin_url": { 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cross_origin_url", 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "crt_file": { 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crt_file", 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "debug": { 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "debug", 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable/disable debug options", 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "jwt_token_ttl": { 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "jwt_token_ttl", 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "28800", 2024-05-10T04:07:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "key_file": { 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "key_file", 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.776 INFO:teuthology.orchestra.run.smithi150.stdout: "motd": { 2024-05-10T04:07:34.777 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "motd", 2024-05-10T04:07:34.777 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.777 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.777 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.777 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.777 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.777 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.777 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.777 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "The message of the day", 2024-05-10T04:07:34.777 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.777 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.777 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.778 INFO:teuthology.orchestra.run.smithi150.stdout: "redirect_resolve_ip_addr": { 2024-05-10T04:07:34.778 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T04:07:34.778 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.778 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.778 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.778 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.778 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.778 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.778 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.778 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.778 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.778 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "::", 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.779 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8080", 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.781 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl": { 2024-05-10T04:07:34.781 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl", 2024-05-10T04:07:34.781 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.781 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.781 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.781 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.781 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.781 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.781 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.781 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.781 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.781 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl_server_port": { 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl_server_port", 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8443", 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.782 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_behaviour": { 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_behaviour", 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "redirect", 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: "redirect" 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.783 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.784 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.784 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.784 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.784 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_error_status_code": { 2024-05-10T04:07:34.784 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_error_status_code", 2024-05-10T04:07:34.784 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.784 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.784 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.784 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "500", 2024-05-10T04:07:34.784 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "400", 2024-05-10T04:07:34.784 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "599", 2024-05-10T04:07:34.784 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "url_prefix": { 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "url_prefix", 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.785 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.786 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.786 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.786 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.786 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "devicehealth", 2024-05-10T04:07:34.786 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.786 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.786 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.786 INFO:teuthology.orchestra.run.smithi150.stdout: "enable_monitoring": { 2024-05-10T04:07:34.786 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enable_monitoring", 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "monitor device health metrics", 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.787 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "mark_out_threshold": { 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mark_out_threshold", 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "2419200", 2024-05-10T04:07:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T04:07:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": { 2024-05-10T04:07:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pool_name", 2024-05-10T04:07:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "device_health_metrics", 2024-05-10T04:07:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T04:07:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "retention_period": { 2024-05-10T04:07:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "retention_period", 2024-05-10T04:07:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15552000", 2024-05-10T04:07:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long to retain device health metrics", 2024-05-10T04:07:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "scrape_frequency": { 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "scrape_frequency", 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "86400", 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "self_heal": { 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "self_heal", 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_threshold": { 2024-05-10T04:07:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_threshold", 2024-05-10T04:07:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "7257600", 2024-05-10T04:07:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T04:07:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "diskprediction_local", 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "predict_interval": { 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "predict_interval", 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "86400", 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "predictor_model": { 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "predictor_model", 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "prophetstor", 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "influx", 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": false, 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "influxdb python module not found", 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "batch_size": { 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "batch_size", 2024-05-10T04:07:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5000", 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "database": { 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "database", 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "ceph", 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "hostname": { 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "hostname", 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB server hostname", 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "30", 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "5", 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "password": { 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "password", 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "password of InfluxDB server user", 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "port": { 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "port", 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8086", 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB server port", 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl": { 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl", 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "false", 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "threads": { 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "threads", 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "32", 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "username": { 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "username", 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "username of InfluxDB server user", 2024-05-10T04:07:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "verify_ssl": { 2024-05-10T04:07:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "verify_ssl", 2024-05-10T04:07:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "true", 2024-05-10T04:07:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T04:07:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "insights", 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "iostat", 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "k8sevents", 2024-05-10T04:07:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_event_retention_days": { 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ceph_event_retention_days", 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "7", 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "config_check_secs": { 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_check_secs", 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "10", 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "localpool", 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "failure_domain": { 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "failure_domain", 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "host", 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "failure domain for any created local pool", 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": { 2024-05-10T04:07:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "min_size", 2024-05-10T04:07:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default min_size for any created local pool", 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "num_rep": { 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "num_rep", 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3", 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default replica count for any created local pool", 2024-05-10T04:07:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": { 2024-05-10T04:07:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pg_num", 2024-05-10T04:07:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "128", 2024-05-10T04:07:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "prefix": { 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prefix", 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "name prefix for any created local pool", 2024-05-10T04:07:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "subtree": { 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "subtree", 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "rack", 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T04:07:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mds_autoscaler", 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mirroring", 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "nfs", 2024-05-10T04:07:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "orchestrator", 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "fail_fs": { 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "fail_fs", 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator": { 2024-05-10T04:07:34.875 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "orchestrator", 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm", 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: "rook", 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: "test_orchestrator" 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Orchestrator backend", 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.876 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "osd_perf_query", 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.877 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.878 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.879 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.880 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.881 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.882 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.882 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.882 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "osd_support", 2024-05-10T04:07:34.882 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.882 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.882 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.882 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.882 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.882 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.882 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.882 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.884 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.884 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.884 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.884 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.884 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.884 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.884 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.884 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.884 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.884 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.884 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.884 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.884 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.885 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.886 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pg_autoscaler", 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.887 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.888 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.888 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.888 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.888 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.888 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.888 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.888 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.888 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.888 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.888 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.888 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.888 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.888 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.889 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.890 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.891 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:34.892 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: "threshold": { 2024-05-10T04:07:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "threshold", 2024-05-10T04:07:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3.0", 2024-05-10T04:07:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1.0", 2024-05-10T04:07:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "scaling threshold", 2024-05-10T04:07:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T04:07:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "progress", 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_pg_recovery_event": { 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_pg_recovery_event", 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.895 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "enabled": { 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enabled", 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.896 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.897 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.898 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.898 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.898 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.898 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.898 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.898 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.898 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.898 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.898 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.898 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.898 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.898 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.899 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.899 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.899 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.899 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.899 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.899 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.899 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.899 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.899 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.899 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.899 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.899 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.900 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.901 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.902 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.902 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.902 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.902 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.902 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.902 INFO:teuthology.orchestra.run.smithi150.stdout: "max_completed_events": { 2024-05-10T04:07:34.902 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_completed_events", 2024-05-10T04:07:34.902 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.902 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.902 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.902 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "50", 2024-05-10T04:07:34.902 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "number of past completed events to remember", 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:07:34.903 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long the module is going to sleep", 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prometheus", 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.904 INFO:teuthology.orchestra.run.smithi150.stdout: "cache": { 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cache", 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "exclude_perf_counters": { 2024-05-10T04:07:34.905 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "exclude_perf_counters", 2024-05-10T04:07:34.906 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.906 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.906 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.906 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.906 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.906 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.906 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.906 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T04:07:34.906 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T04:07:34.906 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.906 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.906 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.907 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.908 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.909 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.910 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.910 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.910 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.910 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.910 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.910 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.910 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.910 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.910 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.910 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.910 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.910 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.911 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.911 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.911 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.911 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.911 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.911 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.911 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.911 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.911 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.911 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.911 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.911 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_stats_pools": { 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_stats_pools", 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "300", 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.913 INFO:teuthology.orchestra.run.smithi150.stdout: "scrape_interval": { 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "scrape_interval", 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15.0", 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.914 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:07:34.915 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:07:34.915 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.915 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.915 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.915 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "::", 2024-05-10T04:07:34.915 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.915 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.915 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.915 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T04:07:34.915 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.915 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.915 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "9283", 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.916 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: "stale_cache_strategy": { 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "stale_cache_strategy", 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "log", 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.917 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.918 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_behaviour": { 2024-05-10T04:07:34.918 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_behaviour", 2024-05-10T04:07:34.918 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.918 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.918 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.918 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "default", 2024-05-10T04:07:34.918 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.918 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.918 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.918 INFO:teuthology.orchestra.run.smithi150.stdout: "default", 2024-05-10T04:07:34.918 INFO:teuthology.orchestra.run.smithi150.stdout: "error" 2024-05-10T04:07:34.918 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_error_status_code": { 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_error_status_code", 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "500", 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "400", 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "599", 2024-05-10T04:07:34.919 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_support", 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.920 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.921 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.922 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.923 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: "max_concurrent_snap_create": { 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_concurrent_snap_create", 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.925 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: "mirror_snapshot_schedule": { 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.926 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: "trash_purge_schedule": { 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "trash_purge_schedule", 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.927 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "restful", 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "enable_auth": { 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enable_auth", 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "key_file": { 2024-05-10T04:07:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "key_file", 2024-05-10T04:07:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rgw", 2024-05-10T04:07:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rook", 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "storage_class": { 2024-05-10T04:07:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "storage_class", 2024-05-10T04:07:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "local", 2024-05-10T04:07:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T04:07:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "selftest", 2024-05-10T04:07:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "roption1": { 2024-05-10T04:07:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "roption1", 2024-05-10T04:07:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "roption2": { 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "roption2", 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "xyz", 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption1": { 2024-05-10T04:07:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption1", 2024-05-10T04:07:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption2": { 2024-05-10T04:07:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption2", 2024-05-10T04:07:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption3": { 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption3", 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption4": { 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption4", 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption5": { 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption5", 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption6": { 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption6", 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption7": { 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption7", 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "42", 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "testkey": { 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testkey", 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "testlkey": { 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testlkey", 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "testnewline": { 2024-05-10T04:07:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testnewline", 2024-05-10T04:07:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snap_schedule", 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_m_granularity": { 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_m_granularity", 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "dump_on_update": { 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "dump_on_update", 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "dump database to debug log on update", 2024-05-10T04:07:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "stats", 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "status", 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "telegraf", 2024-05-10T04:07:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "address": { 2024-05-10T04:07:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "address", 2024-05-10T04:07:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15", 2024-05-10T04:07:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "telemetry", 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_basic": { 2024-05-10T04:07:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_basic", 2024-05-10T04:07:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T04:07:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_crash": { 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_crash", 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_device": { 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_device", 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_ident": { 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_ident", 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_perf": { 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_perf", 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "contact": { 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "contact", 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "description": { 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "description", 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "device_url": { 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_url", 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "enabled": { 2024-05-10T04:07:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enabled", 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "24", 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "8", 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "last_opt_revision": { 2024-05-10T04:07:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "last_opt_revision", 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1", 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "leaderboard": { 2024-05-10T04:07:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "leaderboard", 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "leaderboard_description": { 2024-05-10T04:07:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "leaderboard_description", 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "organization": { 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "organization", 2024-05-10T04:07:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "proxy": { 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "proxy", 2024-05-10T04:07:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "url": { 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "url", 2024-05-10T04:07:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.014 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.014 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.014 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T04:07:35.014 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.014 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.014 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.014 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.014 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.014 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.014 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "test_orchestrator", 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.015 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.016 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.017 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.018 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.019 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "volumes", 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.020 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.021 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.022 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.022 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.022 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.022 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.022 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.022 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.022 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.022 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.022 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.022 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.022 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.022 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.023 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.024 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: "max_concurrent_clones": { 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_concurrent_clones", 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.025 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "4", 2024-05-10T04:07:35.026 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.026 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.026 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.026 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T04:07:35.026 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.026 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.026 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.026 INFO:teuthology.orchestra.run.smithi150.stdout: "snapshot_clone_delay": { 2024-05-10T04:07:35.026 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snapshot_clone_delay", 2024-05-10T04:07:35.026 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.026 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.026 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: "snapshot_clone_no_wait": { 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.027 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.028 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.028 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.028 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.028 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.028 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T04:07:35.028 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.028 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.028 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.028 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix", 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "discovery_interval": { 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "discovery_interval", 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "100", 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.029 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: "identifier": { 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "identifier", 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.030 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.031 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.032 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.033 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.034 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.034 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.034 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.034 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.034 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.034 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.034 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.034 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.034 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.034 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.034 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.034 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.035 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.036 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_host": { 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_host", 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.037 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_port": { 2024-05-10T04:07:35.038 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_port", 2024-05-10T04:07:35.038 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.038 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.038 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.038 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10051", 2024-05-10T04:07:35.038 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.038 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.038 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.038 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.038 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.038 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.038 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_sender": { 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_sender", 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: "modules": [ 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm", 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: "dashboard", 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: "iostat", 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: "nfs", 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: "restful" 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: "available_modules": [ 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.040 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "alerts", 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.041 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.042 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.043 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.044 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.044 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.044 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.044 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.044 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.044 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.044 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.044 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.044 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.044 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.044 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.044 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.045 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.046 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_destination": { 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_destination", 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Email address to send alerts to", 2024-05-10T04:07:35.047 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_from_name": { 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_from_name", 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "Ceph", 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Email From: name", 2024-05-10T04:07:35.048 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_host": { 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_host", 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP server", 2024-05-10T04:07:35.049 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_password": { 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_password", 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Password to authenticate with", 2024-05-10T04:07:35.050 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_port": { 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_port", 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "465", 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP port", 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.051 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_sender": { 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_sender", 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP envelope sender", 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_ssl": { 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_ssl", 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_user": { 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_user", 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "User to authenticate as", 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.054 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "balancer", 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: "active": { 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "active", 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.055 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: "begin_time": { 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "begin_time", 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0000", 2024-05-10T04:07:35.056 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: "begin_weekday": { 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "begin_weekday", 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:35.057 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0", 2024-05-10T04:07:35.058 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "6", 2024-05-10T04:07:35.058 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.058 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T04:07:35.058 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T04:07:35.058 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.058 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.058 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_max_iterations": { 2024-05-10T04:07:35.058 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_max_iterations", 2024-05-10T04:07:35.058 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:35.058 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.058 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.058 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "25", 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "250", 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_metrics": { 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_metrics", 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.059 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_step": { 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_step", 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0.5", 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0.001", 2024-05-10T04:07:35.060 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "0.999", 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "aggressiveness of optimization", 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: "end_time": { 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "end_time", 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "2359", 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.061 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.062 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.062 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T04:07:35.062 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T04:07:35.062 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.062 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.062 INFO:teuthology.orchestra.run.smithi150.stdout: "end_weekday": { 2024-05-10T04:07:35.062 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "end_weekday", 2024-05-10T04:07:35.062 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:35.062 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.062 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.062 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:35.062 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0", 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "6", 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.063 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.064 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.066 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.066 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.066 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.066 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.066 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.066 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.066 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.066 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.066 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.066 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.066 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.066 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.066 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.067 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: "min_score": { 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "min_score", 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.068 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": { 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mode", 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "upmap", 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.069 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.070 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.070 INFO:teuthology.orchestra.run.smithi150.stdout: "crush-compat", 2024-05-10T04:07:35.070 INFO:teuthology.orchestra.run.smithi150.stdout: "none", 2024-05-10T04:07:35.070 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap" 2024-05-10T04:07:35.070 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.070 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Balancer mode", 2024-05-10T04:07:35.070 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.070 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.070 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.070 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_ids": { 2024-05-10T04:07:35.070 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pool_ids", 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:35.071 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap_max_deviation": { 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "upmap_max_deviation", 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.072 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap_max_optimizations": { 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "upmap_max_optimizations", 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.073 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cephadm", 2024-05-10T04:07:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_down_multiplier": { 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_down_multiplier", 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3.0", 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.075 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_refresh_rate": { 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_refresh_rate", 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "20", 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.076 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.077 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.077 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_starting_port": { 2024-05-10T04:07:35.077 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_starting_port", 2024-05-10T04:07:35.077 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.077 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.077 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.077 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "4721", 2024-05-10T04:07:35.077 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.077 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.077 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.077 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T04:07:35.077 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.078 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.078 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.078 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_ptrace": { 2024-05-10T04:07:35.078 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_ptrace", 2024-05-10T04:07:35.078 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.078 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.078 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.078 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.078 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.078 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.078 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: "autotune_interval": { 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "autotune_interval", 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.079 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.080 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T04:07:35.080 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.080 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.080 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.080 INFO:teuthology.orchestra.run.smithi150.stdout: "autotune_memory_target_ratio": { 2024-05-10T04:07:35.080 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T04:07:35.080 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:35.080 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.080 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.080 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0.7", 2024-05-10T04:07:35.080 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.080 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.081 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.081 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T04:07:35.081 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.081 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.081 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.081 INFO:teuthology.orchestra.run.smithi150.stdout: "cgroups_split": { 2024-05-10T04:07:35.081 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cgroups_split", 2024-05-10T04:07:35.081 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.081 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.081 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.081 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.082 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.082 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.082 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.082 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T04:07:35.082 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.082 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.082 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.082 INFO:teuthology.orchestra.run.smithi150.stdout: "config_checks_enabled": { 2024-05-10T04:07:35.082 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_checks_enabled", 2024-05-10T04:07:35.082 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.082 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "config_dashboard": { 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_dashboard", 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.083 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_alertmanager": { 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_alertmanager", 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.084 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_base": { 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_base", 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.085 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Container image name, without the tag", 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_elasticsearch": { 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_elasticsearch", 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.086 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "elasticsearch container image", 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_grafana": { 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_grafana", 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-10T04:07:35.087 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_haproxy": { 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_haproxy", 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.088 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "HAproxy container image", 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_agent": { 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_agent", 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.089 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger agent container image", 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_collector": { 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_collector", 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.090 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.091 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger collector container image", 2024-05-10T04:07:35.091 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.091 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.091 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.091 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_query": { 2024-05-10T04:07:35.091 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_query", 2024-05-10T04:07:35.091 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.091 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.091 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.091 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T04:07:35.091 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger query container image", 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_keepalived": { 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_keepalived", 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T04:07:35.092 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.093 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.093 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.093 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Keepalived container image", 2024-05-10T04:07:35.093 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.093 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.093 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.093 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_loki": { 2024-05-10T04:07:35.094 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_loki", 2024-05-10T04:07:35.094 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.094 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.094 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.094 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T04:07:35.094 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.094 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.094 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.094 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Loki container image", 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_node_exporter": { 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_node_exporter", 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.095 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_nvmeof": { 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_nvmeof", 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.096 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Nvme-of container image", 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_prometheus": { 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_prometheus", 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:07:35.097 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.098 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.098 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.098 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_promtail": { 2024-05-10T04:07:35.098 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_promtail", 2024-05-10T04:07:35.098 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.098 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.098 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.098 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T04:07:35.098 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.098 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.098 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Promtail container image", 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_snmp_gateway": { 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_snmp_gateway", 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SNMP Gateway container image", 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: "container_init": { 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_init", 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.100 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "daemon_cache_timeout": { 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "daemon_cache_timeout", 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T04:07:35.101 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.102 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.102 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.102 INFO:teuthology.orchestra.run.smithi150.stdout: "default_cephadm_command_timeout": { 2024-05-10T04:07:35.102 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T04:07:35.102 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.102 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.102 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.102 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "900", 2024-05-10T04:07:35.102 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.102 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.102 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.102 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: "default_registry": { 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "default_registry", 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io", 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.103 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "device_cache_timeout": { 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_cache_timeout", 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1800", 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache device inventory", 2024-05-10T04:07:35.104 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: "device_enhanced_scan": { 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_enhanced_scan", 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T04:07:35.105 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: "facts_cache_timeout": { 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "facts_cache_timeout", 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache host facts data", 2024-05-10T04:07:35.106 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: "host_check_interval": { 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "host_check_interval", 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to perform a host check", 2024-05-10T04:07:35.107 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.108 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.108 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.108 INFO:teuthology.orchestra.run.smithi150.stdout: "hw_monitoring": { 2024-05-10T04:07:35.108 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "hw_monitoring", 2024-05-10T04:07:35.108 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.108 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.108 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.108 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.108 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.108 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.108 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.108 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: "inventory_list_all": { 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "inventory_list_all", 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.109 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.110 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.111 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.111 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.111 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.111 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.111 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.111 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.111 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.111 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.111 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.111 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.112 INFO:teuthology.orchestra.run.smithi150.stdout: "log_refresh_metadata": { 2024-05-10T04:07:35.112 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_refresh_metadata", 2024-05-10T04:07:35.112 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.112 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.112 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.112 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.112 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.112 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.112 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.112 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T04:07:35.112 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.112 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.112 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.114 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.114 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.114 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.114 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.114 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.114 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.114 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.114 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.114 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.114 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.114 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.114 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.114 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.115 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.115 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.115 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.115 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.115 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.115 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.115 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.115 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.115 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.115 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.115 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.115 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.116 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.117 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "*", 2024-05-10T04:07:35.118 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.118 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.118 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.118 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T04:07:35.118 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.118 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.118 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.118 INFO:teuthology.orchestra.run.smithi150.stdout: "max_count_per_host": { 2024-05-10T04:07:35.118 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_count_per_host", 2024-05-10T04:07:35.118 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.118 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.118 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "max number of daemons per service per host", 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "max_osd_draining_count": { 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_osd_draining_count", 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.119 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: "migration_current": { 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "migration_current", 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.120 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "internal - do not modify", 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": { 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mode", 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.121 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "root", 2024-05-10T04:07:35.122 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.122 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.122 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.122 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm-package", 2024-05-10T04:07:35.122 INFO:teuthology.orchestra.run.smithi150.stdout: "root" 2024-05-10T04:07:35.122 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.122 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T04:07:35.122 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.122 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.122 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.122 INFO:teuthology.orchestra.run.smithi150.stdout: "oob_default_addr": { 2024-05-10T04:07:35.122 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "oob_default_addr", 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "169.254.1.1", 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "prometheus_alerts_path": { 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prometheus_alerts_path", 2024-05-10T04:07:35.123 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_insecure": { 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_insecure", 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.124 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_password": { 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_password", 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.125 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T04:07:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_url": { 2024-05-10T04:07:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_url", 2024-05-10T04:07:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_username": { 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_username", 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "secure_monitoring_stack": { 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "secure_monitoring_stack", 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.128 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: "service_discovery_port": { 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "service_discovery_port", 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8765", 2024-05-10T04:07:35.129 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.131 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.131 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.131 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "cephadm service discovery port", 2024-05-10T04:07:35.131 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.131 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.131 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.131 INFO:teuthology.orchestra.run.smithi150.stdout: "ssh_config_file": { 2024-05-10T04:07:35.131 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssh_config_file", 2024-05-10T04:07:35.131 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "use_agent": { 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "use_agent", 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.132 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: "use_repo_digest": { 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "use_repo_digest", 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.133 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_failed_host_check": { 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_failed_host_check", 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.134 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_stray_daemons": { 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_stray_daemons", 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.135 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_stray_hosts": { 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_stray_hosts", 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.136 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crash", 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.137 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.138 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.139 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.140 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.140 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.140 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.140 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.140 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.140 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.140 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.140 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.140 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.140 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.140 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.140 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.141 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.142 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.142 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.142 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.142 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.142 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.142 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.142 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.142 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.142 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.142 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.142 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.142 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "retain_interval": { 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "retain_interval", 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "31536000", 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.143 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_recent_interval": { 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_recent_interval", 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1209600", 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "dashboard", 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.145 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.146 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.147 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.148 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.148 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.148 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.148 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.148 INFO:teuthology.orchestra.run.smithi150.stdout: "AUDIT_API_ENABLED": { 2024-05-10T04:07:35.148 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T04:07:35.148 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.148 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.148 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.148 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.148 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.148 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.149 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T04:07:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T04:07:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.150 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.151 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.151 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.151 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.151 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.151 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.151 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T04:07:35.151 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T04:07:35.151 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.151 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.151 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.151 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.151 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.152 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.153 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.154 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.154 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.154 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.154 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.154 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.154 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T04:07:35.154 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T04:07:35.154 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.154 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.154 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.154 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.154 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.155 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.156 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.157 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.158 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.159 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.160 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_URL": { 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_URL", 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.161 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.162 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.163 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.164 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.165 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.166 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.166 INFO:teuthology.orchestra.run.smithi150.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T04:07:35.166 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T04:07:35.166 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.166 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.166 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.166 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.166 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.166 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.166 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.166 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.166 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.167 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.168 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.169 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.170 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.171 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.172 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.173 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.174 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T04:07:35.174 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T04:07:35.174 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.174 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.174 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.174 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.174 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.174 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.174 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.174 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.174 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.174 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.175 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.177 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T04:07:35.178 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T04:07:35.178 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.178 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.178 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.178 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:35.178 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.178 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.178 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.178 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.178 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.178 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.178 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8", 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.179 INFO:teuthology.orchestra.run.smithi150.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "45", 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T04:07:35.180 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T04:07:35.181 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T04:07:35.182 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.183 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.183 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.183 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.183 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.183 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.183 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.183 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.183 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.183 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.183 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.183 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T04:07:35.183 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T04:07:35.184 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T04:07:35.185 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.185 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.185 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.185 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.185 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.185 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.185 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.185 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.185 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.185 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.185 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.185 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T04:07:35.186 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T04:07:35.186 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.186 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.186 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.186 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:35.186 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.186 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.186 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.186 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.186 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.186 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.186 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.187 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "cross_origin_url": { 2024-05-10T04:07:35.188 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cross_origin_url", 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "crt_file": { 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crt_file", 2024-05-10T04:07:35.189 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.190 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.190 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.190 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.190 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.190 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.190 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.190 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.190 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.190 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.190 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.190 INFO:teuthology.orchestra.run.smithi150.stdout: "debug": { 2024-05-10T04:07:35.190 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "debug", 2024-05-10T04:07:35.191 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.191 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.191 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.191 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.191 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.191 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.191 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.191 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable/disable debug options", 2024-05-10T04:07:35.191 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.191 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.191 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.191 INFO:teuthology.orchestra.run.smithi150.stdout: "jwt_token_ttl": { 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "jwt_token_ttl", 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "28800", 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.192 INFO:teuthology.orchestra.run.smithi150.stdout: "key_file": { 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "key_file", 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.193 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.195 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.196 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.197 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: "motd": { 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "motd", 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.198 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "The message of the day", 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: "redirect_resolve_ip_addr": { 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.199 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.200 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "::", 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.201 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8080", 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl": { 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl", 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.202 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.203 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.203 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.203 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.203 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.203 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.203 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.203 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.203 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl_server_port": { 2024-05-10T04:07:35.203 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl_server_port", 2024-05-10T04:07:35.203 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.203 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8443", 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_behaviour": { 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_behaviour", 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.204 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "redirect", 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "redirect" 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_error_status_code": { 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_error_status_code", 2024-05-10T04:07:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.206 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.206 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.206 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "500", 2024-05-10T04:07:35.206 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "400", 2024-05-10T04:07:35.206 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "599", 2024-05-10T04:07:35.206 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.206 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.206 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.206 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.206 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.206 INFO:teuthology.orchestra.run.smithi150.stdout: "url_prefix": { 2024-05-10T04:07:35.206 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "url_prefix", 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "devicehealth", 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "enable_monitoring": { 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enable_monitoring", 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.208 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "monitor device health metrics", 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.209 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.210 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.211 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.211 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.211 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.211 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.211 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.211 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.211 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.211 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.211 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.211 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.211 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.211 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.212 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.213 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: "mark_out_threshold": { 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mark_out_threshold", 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "2419200", 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T04:07:35.214 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": { 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pool_name", 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "device_health_metrics", 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.215 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: "retention_period": { 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "retention_period", 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15552000", 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long to retain device health metrics", 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.216 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: "scrape_frequency": { 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "scrape_frequency", 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "86400", 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.217 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: "self_heal": { 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "self_heal", 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.218 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.219 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_threshold": { 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_threshold", 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "7257600", 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "diskprediction_local", 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.221 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.222 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.223 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.224 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: "predict_interval": { 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "predict_interval", 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "86400", 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.226 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: "predictor_model": { 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "predictor_model", 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "prophetstor", 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.227 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.228 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "influx", 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": false, 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "influxdb python module not found", 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: "batch_size": { 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "batch_size", 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5000", 2024-05-10T04:07:35.229 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.230 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.230 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.230 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T04:07:35.230 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.230 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.230 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.230 INFO:teuthology.orchestra.run.smithi150.stdout: "database": { 2024-05-10T04:07:35.230 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "database", 2024-05-10T04:07:35.230 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.230 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.230 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.230 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "ceph", 2024-05-10T04:07:35.231 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.231 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.231 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.231 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T04:07:35.231 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.231 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.231 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.231 INFO:teuthology.orchestra.run.smithi150.stdout: "hostname": { 2024-05-10T04:07:35.231 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "hostname", 2024-05-10T04:07:35.231 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.231 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.231 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.232 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.232 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.232 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.232 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.232 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB server hostname", 2024-05-10T04:07:35.232 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.232 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.232 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.232 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:35.232 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:35.232 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.232 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "30", 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "5", 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.233 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.234 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.235 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.236 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.237 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.238 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.238 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.238 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.238 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.238 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.238 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.238 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.238 INFO:teuthology.orchestra.run.smithi150.stdout: "password": { 2024-05-10T04:07:35.238 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "password", 2024-05-10T04:07:35.239 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.239 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.239 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.239 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.239 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.239 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.239 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.239 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "password of InfluxDB server user", 2024-05-10T04:07:35.239 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.239 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.239 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.240 INFO:teuthology.orchestra.run.smithi150.stdout: "port": { 2024-05-10T04:07:35.240 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "port", 2024-05-10T04:07:35.240 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.240 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.240 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.240 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8086", 2024-05-10T04:07:35.240 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.240 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.240 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.240 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB server port", 2024-05-10T04:07:35.240 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.240 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.240 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl": { 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl", 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "false", 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.242 INFO:teuthology.orchestra.run.smithi150.stdout: "threads": { 2024-05-10T04:07:35.242 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "threads", 2024-05-10T04:07:35.242 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.242 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.242 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.242 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:07:35.242 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:07:35.242 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "32", 2024-05-10T04:07:35.242 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.242 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T04:07:35.242 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.242 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.242 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "username": { 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "username", 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "username of InfluxDB server user", 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "verify_ssl": { 2024-05-10T04:07:35.243 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "verify_ssl", 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "true", 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "insights", 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.245 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.246 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.247 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.247 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.247 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.247 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.247 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.247 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.247 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.247 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.247 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.247 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.247 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.248 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.249 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.250 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.250 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.250 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.250 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.250 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.250 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.250 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "iostat", 2024-05-10T04:07:35.250 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.250 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.250 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.250 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.251 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.252 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.253 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.254 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.254 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.254 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.254 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.254 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.254 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.254 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.254 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.254 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.254 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.254 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.254 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.257 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.257 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.257 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.257 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "k8sevents", 2024-05-10T04:07:35.258 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.259 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.259 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.259 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_event_retention_days": { 2024-05-10T04:07:35.259 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ceph_event_retention_days", 2024-05-10T04:07:35.259 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.259 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.259 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.259 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "7", 2024-05-10T04:07:35.259 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.259 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.259 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.259 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: "config_check_secs": { 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_check_secs", 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "10", 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.260 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.261 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.262 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.263 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.264 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.265 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "localpool", 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: "failure_domain": { 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "failure_domain", 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.266 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "host", 2024-05-10T04:07:35.267 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.267 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.267 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.267 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "failure domain for any created local pool", 2024-05-10T04:07:35.267 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T04:07:35.267 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.267 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.267 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.267 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.267 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.267 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.267 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.268 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.269 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.270 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.271 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": { 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "min_size", 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.272 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.273 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default min_size for any created local pool", 2024-05-10T04:07:35.273 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T04:07:35.273 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.273 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.273 INFO:teuthology.orchestra.run.smithi150.stdout: "num_rep": { 2024-05-10T04:07:35.273 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "num_rep", 2024-05-10T04:07:35.273 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.273 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.273 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.273 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3", 2024-05-10T04:07:35.273 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.273 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default replica count for any created local pool", 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": { 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pg_num", 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "128", 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.274 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "prefix": { 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prefix", 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.275 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "name prefix for any created local pool", 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: "subtree": { 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "subtree", 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "rack", 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.276 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mds_autoscaler", 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.277 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.278 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.279 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.279 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.279 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.279 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.279 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.279 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.279 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.279 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.279 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.279 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.279 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.279 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.280 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.281 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.282 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mirroring", 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.283 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.284 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.285 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.286 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.286 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.286 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.286 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.286 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.286 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.286 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.286 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.286 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.286 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.286 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.286 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.286 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.287 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "nfs", 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.288 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.289 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.290 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.290 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.290 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.290 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.290 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.290 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.290 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.290 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.290 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.290 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.290 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.290 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.291 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.291 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.291 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.291 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.291 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.291 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.291 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.291 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.291 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.291 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.291 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.292 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.292 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.292 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.292 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.292 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.292 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.292 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.292 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.293 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.293 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.293 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.293 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.293 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.293 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.293 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.294 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.294 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.294 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.294 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.294 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.294 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.294 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.294 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.294 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.294 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.294 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "orchestrator", 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: "fail_fs": { 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "fail_fs", 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.295 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.296 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.297 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.298 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.299 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.300 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator": { 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "orchestrator", 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm", 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "rook", 2024-05-10T04:07:35.301 INFO:teuthology.orchestra.run.smithi150.stdout: "test_orchestrator" 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Orchestrator backend", 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "osd_perf_query", 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.302 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.303 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.304 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.305 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.306 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.307 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "osd_support", 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.308 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.309 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.310 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.311 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.311 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.311 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.311 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.311 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.311 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.311 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.311 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.311 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.311 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.311 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.311 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.311 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.312 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pg_autoscaler", 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.313 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.314 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.316 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.317 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.318 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.318 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.318 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.318 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.318 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.318 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:35.318 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:35.318 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.318 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.318 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.318 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:35.318 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "threshold": { 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "threshold", 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3.0", 2024-05-10T04:07:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1.0", 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "scaling threshold", 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "progress", 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.320 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_pg_recovery_event": { 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_pg_recovery_event", 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.321 INFO:teuthology.orchestra.run.smithi150.stdout: "enabled": { 2024-05-10T04:07:35.322 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enabled", 2024-05-10T04:07:35.322 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.322 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.322 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.322 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.322 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.322 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.322 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.322 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.322 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.322 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.322 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.323 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.325 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.326 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.326 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.326 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.326 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.326 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.326 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.326 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.326 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.326 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.326 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.326 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.326 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "max_completed_events": { 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_completed_events", 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.327 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "50", 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "number of past completed events to remember", 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.328 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long the module is going to sleep", 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prometheus", 2024-05-10T04:07:35.329 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "cache": { 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cache", 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.330 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "exclude_perf_counters": { 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "exclude_perf_counters", 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.331 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.332 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.333 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.334 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.335 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_stats_pools": { 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_stats_pools", 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.336 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.337 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "300", 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: "scrape_interval": { 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "scrape_interval", 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.338 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15.0", 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.339 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "::", 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "9283", 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: "stale_cache_strategy": { 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "stale_cache_strategy", 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.341 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "log", 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_behaviour": { 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_behaviour", 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.342 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "default", 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: "default", 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: "error" 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_error_status_code": { 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_error_status_code", 2024-05-10T04:07:35.343 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "500", 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "400", 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "599", 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.344 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_support", 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.345 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.346 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.347 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.348 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.349 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: "max_concurrent_snap_create": { 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_concurrent_snap_create", 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.350 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: "mirror_snapshot_schedule": { 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: "trash_purge_schedule": { 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "trash_purge_schedule", 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.352 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.353 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.353 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "restful", 2024-05-10T04:07:35.353 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.353 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.353 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.353 INFO:teuthology.orchestra.run.smithi150.stdout: "enable_auth": { 2024-05-10T04:07:35.353 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enable_auth", 2024-05-10T04:07:35.353 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.353 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.353 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.353 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.354 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.354 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.354 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.354 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.354 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.354 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.354 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.354 INFO:teuthology.orchestra.run.smithi150.stdout: "key_file": { 2024-05-10T04:07:35.354 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "key_file", 2024-05-10T04:07:35.354 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.354 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.354 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.355 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.356 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.357 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.358 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.359 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.360 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.361 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rgw", 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.362 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.363 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.364 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.365 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.366 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.367 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rook", 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.368 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.369 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.370 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "storage_class": { 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "storage_class", 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.372 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "local", 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "selftest", 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.374 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.375 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.376 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.377 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.378 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.378 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.378 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.378 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.378 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.378 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.378 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.378 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.378 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.378 INFO:teuthology.orchestra.run.smithi150.stdout: "roption1": { 2024-05-10T04:07:35.378 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "roption1", 2024-05-10T04:07:35.378 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: "roption2": { 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "roption2", 2024-05-10T04:07:35.379 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "xyz", 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption1": { 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption1", 2024-05-10T04:07:35.380 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption2": { 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption2", 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.381 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption3": { 2024-05-10T04:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption3", 2024-05-10T04:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption4": { 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption4", 2024-05-10T04:07:35.383 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption5": { 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption5", 2024-05-10T04:07:35.384 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption6": { 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption6", 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.385 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption7": { 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption7", 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.386 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "42", 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "testkey": { 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testkey", 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: "testlkey": { 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testlkey", 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.388 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "testnewline": { 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testnewline", 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.389 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snap_schedule", 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.390 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_m_granularity": { 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_m_granularity", 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: "dump_on_update": { 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "dump_on_update", 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "dump database to debug log on update", 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.393 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.394 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.394 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.394 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.394 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.394 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.394 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.394 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.394 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.394 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.394 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.394 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.395 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.396 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.397 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.398 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "stats", 2024-05-10T04:07:35.398 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.398 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.398 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.398 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.398 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.398 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.398 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.398 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.399 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.400 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.401 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.402 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "status", 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.403 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.404 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.405 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.406 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.407 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "telegraf", 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.408 INFO:teuthology.orchestra.run.smithi150.stdout: "address": { 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "address", 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.409 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15", 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.410 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.411 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.413 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.414 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.414 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.414 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.414 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.414 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.414 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.414 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.414 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.414 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.414 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.414 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.416 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "telemetry", 2024-05-10T04:07:35.416 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.416 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.416 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.416 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_basic": { 2024-05-10T04:07:35.416 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_basic", 2024-05-10T04:07:35.416 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.416 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.416 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.416 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.417 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.417 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.417 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.417 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T04:07:35.417 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.417 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.417 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.417 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_crash": { 2024-05-10T04:07:35.417 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_crash", 2024-05-10T04:07:35.417 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.417 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.417 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_device": { 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_device", 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.419 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.419 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.419 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.419 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.419 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T04:07:35.419 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.419 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.419 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.419 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_ident": { 2024-05-10T04:07:35.419 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_ident", 2024-05-10T04:07:35.419 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_perf": { 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_perf", 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.420 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: "contact": { 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "contact", 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.421 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: "description": { 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "description", 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.422 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: "device_url": { 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_url", 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.423 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: "enabled": { 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enabled", 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.424 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.425 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.425 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.425 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.425 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.425 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.425 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.425 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.425 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.425 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.425 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:35.425 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:35.425 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "24", 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "8", 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: "last_opt_revision": { 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "last_opt_revision", 2024-05-10T04:07:35.426 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.427 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.427 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.427 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1", 2024-05-10T04:07:35.427 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.427 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.427 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.427 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.427 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.427 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.427 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.427 INFO:teuthology.orchestra.run.smithi150.stdout: "leaderboard": { 2024-05-10T04:07:35.427 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "leaderboard", 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: "leaderboard_description": { 2024-05-10T04:07:35.428 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "leaderboard_description", 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.429 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.430 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.431 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.432 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.433 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "organization": { 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "organization", 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.434 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: "proxy": { 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "proxy", 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "url": { 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "url", 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.436 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "test_orchestrator", 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.437 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.438 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.439 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.440 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.440 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.440 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.440 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.440 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.440 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.440 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.440 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.440 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.440 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.440 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.440 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.441 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.442 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.442 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.442 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.442 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.442 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.442 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.442 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.442 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.442 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.442 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.442 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.442 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.442 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "volumes", 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.443 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.444 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.444 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.444 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.444 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.444 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.444 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.444 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.444 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.444 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.444 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.444 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.444 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.446 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.447 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "max_concurrent_clones": { 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_concurrent_clones", 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "4", 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.448 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: "snapshot_clone_delay": { 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snapshot_clone_delay", 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.449 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: "snapshot_clone_no_wait": { 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.450 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix", 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: "discovery_interval": { 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "discovery_interval", 2024-05-10T04:07:35.451 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "100", 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "identifier": { 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "identifier", 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.452 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:07:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.454 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:07:35.455 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:07:35.456 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.457 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.458 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_host": { 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_host", 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.459 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_port": { 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_port", 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10051", 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.460 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_sender": { 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_sender", 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:07:35.461 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: "services": { 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: "dashboard": "https://172.21.15.150:8443/" 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: "always_on_modules": { 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: "octopus": [ 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: "balancer", 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: "crash", 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: "devicehealth", 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator", 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscaler", 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: "progress", 2024-05-10T04:07:35.462 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_support", 2024-05-10T04:07:35.465 INFO:teuthology.orchestra.run.smithi150.stdout: "status", 2024-05-10T04:07:35.465 INFO:teuthology.orchestra.run.smithi150.stdout: "telemetry", 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: "volumes" 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: "pacific": [ 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: "balancer", 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: "crash", 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: "devicehealth", 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator", 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscaler", 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: "progress", 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_support", 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: "status", 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: "telemetry", 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: "volumes" 2024-05-10T04:07:35.466 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: "quincy": [ 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: "balancer", 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: "crash", 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: "devicehealth", 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator", 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscaler", 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: "progress", 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_support", 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: "status", 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: "telemetry", 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: "volumes" 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: "reef": [ 2024-05-10T04:07:35.467 INFO:teuthology.orchestra.run.smithi150.stdout: "balancer", 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: "crash", 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: "devicehealth", 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator", 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscaler", 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: "progress", 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_support", 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: "status", 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: "telemetry", 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: "volumes" 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: "last_failure_osd_epoch": 3, 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: "active_clients": [ 2024-05-10T04:07:35.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "devicehealth", 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:0", 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3012201187 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "libcephsqlite", 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.469 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-05-10T04:07:35.470 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:0", 2024-05-10T04:07:35.470 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 661419065 2024-05-10T04:07:35.470 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.470 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:07:35.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.470 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_support", 2024-05-10T04:07:35.470 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-05-10T04:07:35.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.470 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-05-10T04:07:35.470 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:0", 2024-05-10T04:07:35.470 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1054942358 2024-05-10T04:07:35.470 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "volumes", 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:0", 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 36425353 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:07:35.471 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:07:35.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:34 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2976080329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:07:35.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:34 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2307634280' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T04:07:35.473 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:34 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2976080329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:07:35.473 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:34 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2307634280' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T04:07:35.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.187+0000 7fea11417700 1 -- 172.21.15.150:0/2307634280 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9f40a1120 msgr2=0x7fe9f40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:35.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.187+0000 7fea11417700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9f40a1120 0x7fe9f40a3630 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fea0c199120 tx=0x7fe9fc009260 comp rx=0 tx=0).stop 2024-05-10T04:07:35.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.187+0000 7fea11417700 1 -- 172.21.15.150:0/2307634280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0c0fff40 msgr2=0x7fea0c198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:35.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.187+0000 7fea11417700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0c0fff40 0x7fea0c198080 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fea0000d980 tx=0x7fea0000dd40 comp rx=0 tx=0).stop 2024-05-10T04:07:35.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.187+0000 7fea11417700 1 -- 172.21.15.150:0/2307634280 shutdown_connections 2024-05-10T04:07:35.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.187+0000 7fea11417700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9f40a1120 0x7fe9f40a3630 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:35.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.187+0000 7fea11417700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea0c107a10 0x7fea0c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:35.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.187+0000 7fea11417700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0c0fff40 0x7fea0c198080 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:35.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.187+0000 7fea11417700 1 --2- 172.21.15.150:0/2307634280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea0c1050b0 0x7fea0c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:35.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.187+0000 7fea11417700 1 -- 172.21.15.150:0/2307634280 >> 172.21.15.150:0/2307634280 conn(0x7fea0c0fb8b0 msgr2=0x7fea0c105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:35.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.187+0000 7fea11417700 1 -- 172.21.15.150:0/2307634280 shutdown_connections 2024-05-10T04:07:35.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:34.187+0000 7fea11417700 1 -- 172.21.15.150:0/2307634280 wait complete. 2024-05-10T04:07:35.481 DEBUG:tasks.check_counter:Getting stats from a 2024-05-10T04:07:35.481 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-10T04:07:35.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:34 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2976080329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:07:35.481 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:34 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2307634280' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T04:07:35.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:35 smithi150 ceph-mon[42477]: pgmap v380: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:35.716 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:07:35.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:35 smithi181 ceph-mon[46602]: pgmap v380: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:35.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:35 smithi161 ceph-mon[46918]: pgmap v380: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:36.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:36 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:07:36.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:36 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:07:36.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:36 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:07:36.767 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:07:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:07:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-05-10T04:07:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-05-10T04:07:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-05-10T04:07:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-05-10T04:07:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-10T04:07:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-10T04:07:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001709789, 2024-05-10T04:07:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T04:07:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000290023, 2024-05-10T04:07:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 5, 2024-05-10T04:07:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 68, 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3152, 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 88192, 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 3, 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.006505010, 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.004036549, 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001772347, 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 67, 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.004430227, 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000066122 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 88192 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 90, 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 87, 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 435056, 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 48848, 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.015876934, 2024-05-10T04:07:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.005351879, 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.010996000, 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.001841062, 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 86, 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.006988830, 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000081265 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 435056, 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 48848 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-10T04:07:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:07:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:07:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 149854, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 28, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-10T04:07:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-10T04:07:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 91, 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 422363, 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 91, 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 91, 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 422363, 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:07:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:36.777 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:07:37.164 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-10T04:07:37.164 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-10T04:07:37.164 DEBUG:tasks.check_counter:Getting stats from d 2024-05-10T04:07:37.164 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-10T04:07:37.396 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:07:37.792 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:07:37.792 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:07:37.792 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:37 smithi150 ceph-mon[42477]: pgmap v381: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:37.792 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:07:37.792 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:07:37.792 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:37 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:07:37.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:07:37.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:07:37.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:37 smithi181 ceph-mon[46602]: pgmap v381: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:37.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:07:37.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:07:37.987 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:37 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:07:37.991 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:07:37.991 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:07:37.991 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:37 smithi161 ceph-mon[46918]: pgmap v381: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:37.991 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:07:37.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:07:37.992 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:37 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' 2024-05-10T04:07:38.378 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001412550, 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000452543, 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:38.379 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1, 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 69, 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 160, 2024-05-10T04:07:38.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 88272, 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 3, 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.006615257, 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.004261305, 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001545674, 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 67, 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.015210460, 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000227021 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:38.381 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 88272 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 97, 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 92, 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 438864, 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 51088, 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.017407325, 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.005736483, 2024-05-10T04:07:38.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.012295367, 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.001901251, 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 92, 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.007290507, 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000079244 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 438864, 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 51088 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:38.383 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-10T04:07:38.384 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-10T04:07:38.384 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-10T04:07:38.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:38.384 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-10T04:07:38.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:07:38.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-10T04:07:38.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:07:38.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:07:38.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:07:38.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:07:38.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148840, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 28, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-10T04:07:38.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:07:38.388 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 94, 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 422753, 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 94, 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 94, 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 422753, 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:38.389 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:07:38.761 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-10T04:07:38.761 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-10T04:07:38.761 DEBUG:tasks.check_counter:Getting stats from g 2024-05-10T04:07:38.762 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-10T04:07:38.984 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:07:40.002 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:07:40.002 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:07:40.002 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-05-10T04:07:40.002 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-05-10T04:07:40.002 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-05-10T04:07:40.002 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-05-10T04:07:40.002 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-10T04:07:40.002 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001417913, 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000390558, 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:40.003 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 5, 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 69, 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3152, 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 88368, 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 3, 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.007129814, 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.004056510, 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.002344421, 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:07:40.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 68, 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.016394178, 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000241090 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 88368 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:07:40.005 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 91, 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 88, 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 435328, 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 49456, 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.015450854, 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.005191540, 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.010578456, 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.001880018, 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 87, 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.012470428, 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000143338 2024-05-10T04:07:40.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 435328, 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 49456 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:07:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-10T04:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148823, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 27, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-10T04:07:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 92, 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 422493, 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 92, 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 92, 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 422493, 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:40.013 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:07:40.370 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-10T04:07:40.370 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-10T04:07:40.370 DEBUG:tasks.check_counter:Getting stats from j 2024-05-10T04:07:40.370 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-10T04:07:40.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:40 smithi150 ceph-mon[42477]: pgmap v382: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:07:40.594 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:07:40.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:40 smithi181 ceph-mon[46602]: pgmap v382: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:07:40.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:40 smithi161 ceph-mon[46918]: pgmap v382: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:07:41.618 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:07:41.618 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:07:41.618 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-05-10T04:07:41.618 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-05-10T04:07:41.618 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-05-10T04:07:41.618 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-05-10T04:07:41.618 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-10T04:07:41.618 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-10T04:07:41.618 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001246286, 2024-05-10T04:07:41.618 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000325416, 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:07:41.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1, 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 70, 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 160, 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 88448, 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 3, 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.006456769, 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.004113085, 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001538568, 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 68, 2024-05-10T04:07:41.620 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.024245539, 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000356552 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 88448 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 98, 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 93, 2024-05-10T04:07:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 439136, 2024-05-10T04:07:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 51680, 2024-05-10T04:07:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-10T04:07:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-10T04:07:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.017201235, 2024-05-10T04:07:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.005740845, 2024-05-10T04:07:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.012206952, 2024-05-10T04:07:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.001997040, 2024-05-10T04:07:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 93, 2024-05-10T04:07:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.017103430, 2024-05-10T04:07:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000183907 2024-05-10T04:07:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 439136, 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 51680 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-10T04:07:41.623 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:07:41.624 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:07:41.624 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:07:41.624 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:07:41.624 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:07:41.624 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:07:41.624 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:07:41.624 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:07:41.624 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:07:41.624 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:07:41.624 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:07:41.624 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:07:41.624 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:07:41.624 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-10T04:07:41.625 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:07:41.625 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-10T04:07:41.625 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:07:41.625 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:07:41.625 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:07:41.625 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:07:41.625 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:07:41.625 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:07:41.625 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:07:41.625 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-10T04:07:41.625 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:07:41.625 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148823, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 27, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-10T04:07:41.626 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-10T04:07:41.627 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 95, 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 422883, 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 95, 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 95, 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 422883, 2024-05-10T04:07:41.628 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:07:41.629 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:07:41.629 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:07:41.629 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:07:41.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:41.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:07:41.629 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:07:42.002 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-10T04:07:42.003 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-10T04:07:42.003 DEBUG:tasks.check_counter:Getting stats from b 2024-05-10T04:07:42.003 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-10T04:07:42.253 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:07:42.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:42 smithi161 ceph-mon[46918]: pgmap v383: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:42.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:42 smithi150 ceph-mon[42477]: pgmap v383: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:42.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:42 smithi181 ceph-mon[46602]: pgmap v383: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:43.370 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-10T04:07:43.371 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:07:43.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 2179, 2024-05-10T04:07:43.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2116, 2024-05-10T04:07:43.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 608550, 2024-05-10T04:07:43.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1239445, 2024-05-10T04:07:43.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 12, 2024-05-10T04:07:43.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-05-10T04:07:43.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.172843910, 2024-05-10T04:07:43.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.043258231, 2024-05-10T04:07:43.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.049142531, 2024-05-10T04:07:43.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.062289661, 2024-05-10T04:07:43.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2114, 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.368433741, 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000647319 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 280, 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000074296, 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000265 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1970, 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1820, 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 974399, 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1289335, 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 77, 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 7, 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.176745902, 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.038692388, 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.081717024, 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.053709153, 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1819, 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.724866712, 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000398497 2024-05-10T04:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 340, 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000079287, 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000233 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 568560, 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 97072 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 2818, 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2710, 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 800566, 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1172352, 2024-05-10T04:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 15, 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.234754934, 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.064822467, 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.083882626, 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.077510563, 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2709, 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.242795657, 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000458765 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1224, 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000360949, 2024-05-10T04:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000294 2024-05-10T04:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-10T04:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 198208 2024-05-10T04:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-05-10T04:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-05-10T04:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-05-10T04:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-05-10T04:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-10T04:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-10T04:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 605, 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.128956933, 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000213151 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3989, 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.140622292, 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000035252 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-05-10T04:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 234, 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 234, 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 234, 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.132322447, 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000565480 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 75, 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-05-10T04:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 23, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 4, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 19, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 16, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 2, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 2, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 2, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 436, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 429, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-05-10T04:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 78, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 78, 2024-05-10T04:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 51, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 6, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 45, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 27, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 17, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 120, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 146, 2024-05-10T04:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 124, 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 4, 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 27 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 2, 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 5, 2024-05-10T04:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 5, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 6, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-05-10T04:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 196, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 187, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 187, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 9, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 3, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 13, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 12, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 12, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 1, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4685019, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4712896, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-05-10T04:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 177, 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.514844062, 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002908723 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 23, 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 34, 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 11, 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 18, 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 20, 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 2, 2024-05-10T04:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 23, 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 32, 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 9, 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 2, 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 19, 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 17, 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 123168, 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 207252 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 255, 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 234, 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 14, 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-05-10T04:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 7, 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.007193794, 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001027684 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 197, 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.067188595, 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000341058 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000443511, 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000443511 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000299079, 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000299079 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8, 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.012688256, 2024-05-10T04:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001586032 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.004494613, 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002247306 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.003079461, 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001539730 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.004997862, 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002498931 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.002477807, 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001238903 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 9, 2024-05-10T04:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.023607350, 2024-05-10T04:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002623038 2024-05-10T04:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-05-10T04:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-05-10T04:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-05-10T04:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.005852119, 2024-05-10T04:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001950706 2024-05-10T04:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 31, 2024-05-10T04:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 31, 2024-05-10T04:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 218, 2024-05-10T04:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-05-10T04:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-05-10T04:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-05-10T04:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 299549, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 130, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 616, 2024-05-10T04:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 7, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8696, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 166, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 91754, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 587, 2024-05-10T04:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 5606, 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 604206, 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 37, 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 5606, 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 5606, 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 89.768171461, 2024-05-10T04:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.016012873 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 5606, 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 5817 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 5606, 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 95, 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 5511, 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 55, 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 38, 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 11, 2024-05-10T04:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 182, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 56, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 5127, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 84, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 69, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-05-10T04:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 87, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 7, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2024-05-10T04:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 181, 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 14 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-05-10T04:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 3585, 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 1, 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 5, 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 6963, 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1736567, 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 6963, 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 6963, 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1736567, 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 5606, 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 604735, 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 5606, 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 604735, 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 5606, 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 5606, 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 5606, 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 5606, 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 5, 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 529, 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 5, 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 5, 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 529, 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x556ad675e4e0": { 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 196, 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 518592, 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 196, 2024-05-10T04:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 177, 2024-05-10T04:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 518592, 2024-05-10T04:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-10T04:07:43.738 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-10T04:07:43.738 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-10T04:07:43.738 DEBUG:tasks.check_counter:Getting stats from e 2024-05-10T04:07:43.738 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-10T04:07:43.961 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:07:44.439 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:44 smithi161 ceph-mon[46918]: pgmap v384: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:44.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:44 smithi150 ceph-mon[42477]: pgmap v384: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:44.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:44 smithi181 ceph-mon[46602]: pgmap v384: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:44.962 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-10T04:07:44.962 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:07:44.962 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 794, 2024-05-10T04:07:44.963 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 796, 2024-05-10T04:07:44.963 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 227863, 2024-05-10T04:07:44.963 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 246941, 2024-05-10T04:07:44.963 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-05-10T04:07:44.963 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-05-10T04:07:44.963 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.071128222, 2024-05-10T04:07:44.963 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.021559621, 2024-05-10T04:07:44.963 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.018211720, 2024-05-10T04:07:44.963 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.024760157, 2024-05-10T04:07:44.963 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:44.963 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 794, 2024-05-10T04:07:44.963 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.028269569, 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000035603 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 104, 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 100, 2024-05-10T04:07:44.964 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 455036, 2024-05-10T04:07:44.965 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 53100, 2024-05-10T04:07:44.965 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 3, 2024-05-10T04:07:44.965 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-05-10T04:07:44.965 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.018698378, 2024-05-10T04:07:44.965 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.005846879, 2024-05-10T04:07:44.965 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.012263013, 2024-05-10T04:07:44.965 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.002400550, 2024-05-10T04:07:44.965 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:44.965 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 99, 2024-05-10T04:07:44.965 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.006647064, 2024-05-10T04:07:44.965 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000067142 2024-05-10T04:07:44.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.966 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:44.966 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.966 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.966 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.966 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 454480, 2024-05-10T04:07:44.966 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 52416 2024-05-10T04:07:44.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.966 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:07:44.966 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 6, 2024-05-10T04:07:44.966 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 73, 2024-05-10T04:07:44.966 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 3420, 2024-05-10T04:07:44.966 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 189095, 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.005521891, 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.002759011, 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.001928367, 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000025969, 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 72, 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.006644309, 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000092282 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.967 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 188784 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1370, 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.038755867, 2024-05-10T04:07:44.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000028288 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-05-10T04:07:44.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.970 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.970 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-05-10T04:07:44.970 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-05-10T04:07:44.970 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-05-10T04:07:44.970 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-05-10T04:07:44.970 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-05-10T04:07:44.970 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-05-10T04:07:44.970 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-05-10T04:07:44.970 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 0, 2024-05-10T04:07:44.970 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-05-10T04:07:44.970 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 1, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-05-10T04:07:44.971 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:07:44.972 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-05-10T04:07:44.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.974 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-05-10T04:07:44.974 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-05-10T04:07:44.974 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-05-10T04:07:44.974 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-05-10T04:07:44.974 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-05-10T04:07:44.974 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-05-10T04:07:44.974 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-05-10T04:07:44.974 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-05-10T04:07:44.974 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:07:44.974 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-05-10T04:07:44.974 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-05-10T04:07:44.974 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-05-10T04:07:44.975 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 4, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 1, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4195692, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 4195551, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.976 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 4 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 3, 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 3, 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 0, 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 1, 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 1, 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 0, 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 0, 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 0, 2024-05-10T04:07:44.977 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 0, 2024-05-10T04:07:44.978 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-05-10T04:07:44.978 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-05-10T04:07:44.978 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-05-10T04:07:44.978 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 115920, 2024-05-10T04:07:44.978 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 207252 2024-05-10T04:07:44.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.978 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-05-10T04:07:44.978 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-05-10T04:07:44.978 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-05-10T04:07:44.978 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-05-10T04:07:44.978 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-05-10T04:07:44.978 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-05-10T04:07:44.978 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-05-10T04:07:44.979 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.980 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.982 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.983 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-05-10T04:07:44.984 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.985 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.985 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.985 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-05-10T04:07:44.985 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.985 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.985 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.985 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-05-10T04:07:44.985 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.985 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.985 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.986 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.987 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:44.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 2, 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 2, 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 3000, 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-05-10T04:07:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-05-10T04:07:44.990 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-05-10T04:07:44.991 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:07:44.992 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:07:44.992 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:07:44.992 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:07:44.992 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 182190, 2024-05-10T04:07:44.992 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 45, 2024-05-10T04:07:44.992 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:07:44.992 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-05-10T04:07:44.992 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-05-10T04:07:44.992 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-05-10T04:07:44.992 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:07:44.992 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-05-10T04:07:44.992 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:07:44.992 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8696, 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 166, 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 8320, 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 13, 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-05-10T04:07:44.993 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 797, 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 797, 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 797, 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.486990064, 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000611028 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 797, 2024-05-10T04:07:44.994 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 798 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 797, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 797, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 500, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 295, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-05-10T04:07:44.995 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:07:44.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-05-10T04:07:44.997 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 87, 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 2, 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 4, 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 4, 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.998 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-05-10T04:07:44.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 900, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 597078, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 900, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 900, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 597078, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:45.000 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 797, 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 1393, 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 797, 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1393, 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:07:45.001 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:45.002 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:45.002 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:45.002 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:45.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:45.002 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-05-10T04:07:45.002 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:07:45.002 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-05-10T04:07:45.002 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:07:45.002 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:07:45.002 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:07:45.002 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:45.002 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 797, 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 797, 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 797, 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 797, 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:07:45.003 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55ed3863a8f0": { 2024-05-10T04:07:45.004 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:45.005 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:45.006 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:45.006 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-10T04:07:45.305 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-10T04:07:45.305 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-10T04:07:45.305 DEBUG:tasks.check_counter:Getting stats from h 2024-05-10T04:07:45.305 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-10T04:07:45.527 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:07:46.518 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-10T04:07:46.519 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:07:46.519 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 4, 2024-05-10T04:07:46.519 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 4, 2024-05-10T04:07:46.519 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 2992, 2024-05-10T04:07:46.519 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 448, 2024-05-10T04:07:46.519 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-05-10T04:07:46.519 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2024-05-10T04:07:46.519 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.001875887, 2024-05-10T04:07:46.519 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000031845, 2024-05-10T04:07:46.519 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000895852, 2024-05-10T04:07:46.519 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:07:46.519 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000908819, 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000302939 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:07:46.520 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 93, 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 90, 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 435872, 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 50640, 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.018292569, 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.004656104, 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.012290539, 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.003037387, 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 89, 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.006308022, 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000070876 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:46.521 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 435872, 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 50640 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1, 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 69, 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 160, 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 88272, 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 3, 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-05-10T04:07:46.522 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.004922527, 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.002919395, 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.001145996, 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 67, 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.003660372, 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000054632 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:46.523 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 88272 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:07:46.524 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:07:46.525 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:07:46.526 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 149854, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 28, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:07:46.527 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 94, 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 422753, 2024-05-10T04:07:46.528 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:46.529 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 94, 2024-05-10T04:07:46.529 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:07:46.529 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:07:46.529 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 94, 2024-05-10T04:07:46.529 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 422753, 2024-05-10T04:07:46.529 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:07:46.529 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:46.529 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:46.529 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:46.529 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:46.529 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:46.529 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-10T04:07:46.543 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:46 smithi161 ceph-mon[46918]: pgmap v385: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:46.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:46 smithi150 ceph-mon[42477]: pgmap v385: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:46.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:46 smithi181 ceph-mon[46602]: pgmap v385: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:46.913 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.h 2024-05-10T04:07:46.913 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.h 2024-05-10T04:07:46.913 DEBUG:tasks.check_counter:Getting stats from k 2024-05-10T04:07:46.914 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-10T04:07:47.136 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:07:48.103 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-10T04:07:48.104 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:07:48.104 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 4, 2024-05-10T04:07:48.104 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 4, 2024-05-10T04:07:48.104 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 2992, 2024-05-10T04:07:48.104 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 448, 2024-05-10T04:07:48.104 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-05-10T04:07:48.104 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2024-05-10T04:07:48.104 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.001822051, 2024-05-10T04:07:48.104 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000033507, 2024-05-10T04:07:48.104 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000772287, 2024-05-10T04:07:48.104 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:07:48.104 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000910199, 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000303399 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1, 2024-05-10T04:07:48.105 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 71, 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 160, 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 88624, 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 3, 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.005133148, 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.003099421, 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.001032783, 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 69, 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.004688448, 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000067948 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:48.106 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 88624 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 96, 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 93, 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 436688, 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 52416, 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.018599835, 2024-05-10T04:07:48.107 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.005520485, 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.012210886, 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.002608994, 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 92, 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.005983562, 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000065038 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 436688, 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 52416 2024-05-10T04:07:48.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:07:48.109 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:07:48.110 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 148863, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 28, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-05-10T04:07:48.111 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-05-10T04:07:48.112 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 97, 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 423143, 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 97, 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 97, 2024-05-10T04:07:48.113 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 423143, 2024-05-10T04:07:48.114 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:07:48.114 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:07:48.114 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:07:48.114 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:07:48.114 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:48.114 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:07:48.114 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-10T04:07:48.472 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-10T04:07:48.472 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-10T04:07:48.472 DEBUG:tasks.check_counter:Getting stats from c 2024-05-10T04:07:48.472 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-10T04:07:48.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:48 smithi150 ceph-mon[42477]: pgmap v386: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:48.722 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:07:48.736 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:48 smithi181 ceph-mon[46602]: pgmap v386: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:48.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:48 smithi161 ceph-mon[46918]: pgmap v386: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:49.812 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 169, 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 161, 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 474120, 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 144244, 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 15, 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 6, 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.028879776, 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.006763727, 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.019944362, 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.002915916, 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 161, 2024-05-10T04:07:49.813 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.018279659, 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000113538 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 23, 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000004303, 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000187 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 469264, 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 60000 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 23, 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 92, 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 5353, 2024-05-10T04:07:49.814 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 241645, 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 11, 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.011177615, 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.004015140, 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.005662780, 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000399080, 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 90, 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.023264758, 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000258497 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 11, 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000003633, 2024-05-10T04:07:49.815 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000330 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 189904 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 125, 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 194, 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 31419, 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 135954, 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 10, 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.022886545, 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.007683562, 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.011447660, 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.002351628, 2024-05-10T04:07:49.816 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 192, 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.024018754, 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000125097 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 287, 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000111353, 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000387 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-05-10T04:07:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-05-10T04:07:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-05-10T04:07:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:07:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:07:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 50, 2024-05-10T04:07:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001788387, 2024-05-10T04:07:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000035767 2024-05-10T04:07:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-05-10T04:07:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:07:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:07:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000018382, 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000009191 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 4, 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 1, 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000335985, 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000335985 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-05-10T04:07:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 3, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 11, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 10, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 0, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 10, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 10, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 0, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 0, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 1, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 5, 2024-05-10T04:07:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 2, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 2, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-05-10T04:07:49.821 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 4, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 234, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:07:49.822 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 1 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 1, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-05-10T04:07:49.823 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-05-10T04:07:49.824 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 4, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 4, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 1, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 1, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-05-10T04:07:49.825 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 4195692, 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4, 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.007861363, 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001965340 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 13, 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 13, 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 0, 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 11, 2024-05-10T04:07:49.826 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 12, 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 1, 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 10, 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 10, 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 0, 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 1, 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 1, 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 116020, 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 199060 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 4, 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 4, 2024-05-10T04:07:49.827 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 8, 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000335985, 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000335985 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.828 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.829 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.830 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-05-10T04:07:49.831 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-05-10T04:07:49.832 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.833 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.834 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.836 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:07:49.837 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 2, 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 1, 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 1, 2024-05-10T04:07:49.838 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 1, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 224, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:07:49.839 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-05-10T04:07:49.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 231323, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 82, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-05-10T04:07:49.841 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8696, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 166, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 50056, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 200, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-05-10T04:07:49.842 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 40, 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 11690, 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 33, 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 40, 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 40, 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.245995323, 2024-05-10T04:07:49.843 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006149883 2024-05-10T04:07:49.844 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.844 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-05-10T04:07:49.844 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-05-10T04:07:49.844 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 40, 2024-05-10T04:07:49.844 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 62 2024-05-10T04:07:49.844 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.844 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 40, 2024-05-10T04:07:49.844 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 0, 2024-05-10T04:07:49.844 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 40, 2024-05-10T04:07:49.844 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-05-10T04:07:49.844 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-05-10T04:07:49.844 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 10, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 0, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 4, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 7, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 6, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 0, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 20, 2024-05-10T04:07:49.845 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-05-10T04:07:49.846 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 87, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 2, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 8, 2024-05-10T04:07:49.847 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 8, 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 14, 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 1 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-05-10T04:07:49.848 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 313, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 472613, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 313, 2024-05-10T04:07:49.849 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 313, 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 472613, 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:49.850 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:49.851 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:49.851 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:49.851 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 40, 2024-05-10T04:07:49.851 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 11690, 2024-05-10T04:07:49.851 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 40, 2024-05-10T04:07:49.851 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 11690, 2024-05-10T04:07:49.851 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:49.851 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.851 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.851 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.851 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:49.851 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.851 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 40, 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 40, 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 40, 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 40, 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.852 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:49.853 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55e37aec0270": { 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 4, 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 1388, 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 4, 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:07:49.854 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:07:49.855 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 4, 2024-05-10T04:07:49.855 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1388, 2024-05-10T04:07:49.855 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:49.855 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:49.855 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:49.855 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:49.855 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:49.855 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:49.855 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-10T04:07:50.151 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-10T04:07:50.151 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-10T04:07:50.151 DEBUG:tasks.check_counter:Getting stats from f 2024-05-10T04:07:50.151 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-10T04:07:50.377 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:07:50.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:50 smithi181 ceph-mon[46602]: pgmap v387: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:07:50.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:50 smithi150 ceph-mon[42477]: pgmap v387: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:07:50.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:50 smithi161 ceph-mon[46918]: pgmap v387: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T04:07:51.431 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-10T04:07:51.431 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:07:51.431 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 644, 2024-05-10T04:07:51.431 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 646, 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 708801, 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 200294, 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 6, 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 2, 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.067900792, 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.019737833, 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.019087003, 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.021850975, 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 644, 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.034390402, 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000053401 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.432 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 5, 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 75, 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1816, 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 191390, 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 5, 2024-05-10T04:07:51.433 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2024-05-10T04:07:51.434 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.005871922, 2024-05-10T04:07:51.434 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.002972906, 2024-05-10T04:07:51.434 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.001917798, 2024-05-10T04:07:51.434 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000108905, 2024-05-10T04:07:51.434 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:51.434 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 73, 2024-05-10T04:07:51.434 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.008352179, 2024-05-10T04:07:51.434 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000114413 2024-05-10T04:07:51.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.434 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:51.435 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.435 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.435 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.435 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:07:51.435 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 190080 2024-05-10T04:07:51.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.435 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:07:51.435 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 404, 2024-05-10T04:07:51.435 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 398, 2024-05-10T04:07:51.435 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 530817, 2024-05-10T04:07:51.435 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 145938, 2024-05-10T04:07:51.435 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 3, 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 2, 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.048987092, 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.016210223, 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.019013383, 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.012871608, 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 398, 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.023836698, 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000059891 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.436 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 458176, 2024-05-10T04:07:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 54192 2024-05-10T04:07:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-05-10T04:07:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-05-10T04:07:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-05-10T04:07:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-05-10T04:07:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:07:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:07:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1555, 2024-05-10T04:07:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.048675355, 2024-05-10T04:07:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000031302 2024-05-10T04:07:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-05-10T04:07:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:07:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:07:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-05-10T04:07:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 0, 2024-05-10T04:07:51.439 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 0, 2024-05-10T04:07:51.439 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-05-10T04:07:51.439 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.439 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.439 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.439 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 0, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 21, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 6, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 15, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 12, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 0, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 0, 2024-05-10T04:07:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 2, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-05-10T04:07:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 4, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 234, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:07:51.442 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:07:51.443 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:07:51.443 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:07:51.443 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:07:51.443 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:07:51.443 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2024-05-10T04:07:51.443 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:07:51.443 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2024-05-10T04:07:51.443 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2024-05-10T04:07:51.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.443 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-05-10T04:07:51.443 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:07:51.444 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:07:51.445 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-05-10T04:07:51.445 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-05-10T04:07:51.445 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:07:51.445 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-05-10T04:07:51.445 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-05-10T04:07:51.445 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:07:51.445 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 0, 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 196, 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 0, 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 2, 2024-05-10T04:07:51.446 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-05-10T04:07:51.447 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-05-10T04:07:51.447 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4685019, 2024-05-10T04:07:51.447 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 4712896, 2024-05-10T04:07:51.447 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 4711758, 2024-05-10T04:07:51.447 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-05-10T04:07:51.447 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.447 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.447 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.448 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 196 2024-05-10T04:07:51.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.448 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-05-10T04:07:51.448 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 16, 2024-05-10T04:07:51.448 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 21, 2024-05-10T04:07:51.448 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 5, 2024-05-10T04:07:51.448 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 11, 2024-05-10T04:07:51.448 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 13, 2024-05-10T04:07:51.448 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 2, 2024-05-10T04:07:51.448 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 21, 2024-05-10T04:07:51.448 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 25, 2024-05-10T04:07:51.448 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 4, 2024-05-10T04:07:51.449 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2024-05-10T04:07:51.449 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 0, 2024-05-10T04:07:51.449 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2024-05-10T04:07:51.449 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 119560, 2024-05-10T04:07:51.449 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 207252 2024-05-10T04:07:51.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.449 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-05-10T04:07:51.449 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 0, 2024-05-10T04:07:51.449 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-05-10T04:07:51.449 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 0, 2024-05-10T04:07:51.449 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 0, 2024-05-10T04:07:51.449 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2024-05-10T04:07:51.449 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.451 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.452 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.453 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.454 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.454 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-05-10T04:07:51.454 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.454 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.454 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.454 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-05-10T04:07:51.454 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.454 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.454 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.454 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-05-10T04:07:51.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-05-10T04:07:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-05-10T04:07:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.457 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-05-10T04:07:51.457 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.457 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.457 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.457 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-05-10T04:07:51.457 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.457 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.457 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.457 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-05-10T04:07:51.457 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.457 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.458 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.458 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-05-10T04:07:51.458 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.458 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.458 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.458 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-05-10T04:07:51.458 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.458 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.458 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.458 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.460 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 3, 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 2, 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 2, 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 0, 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 0, 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 3017, 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:07:51.461 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:07:51.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-05-10T04:07:51.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 189239, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 48, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8696, 2024-05-10T04:07:51.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 166, 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 63778, 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 299, 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-05-10T04:07:51.465 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 943, 2024-05-10T04:07:51.466 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 26, 2024-05-10T04:07:51.466 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 4, 2024-05-10T04:07:51.466 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 943, 2024-05-10T04:07:51.466 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-05-10T04:07:51.466 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 943, 2024-05-10T04:07:51.466 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.495990553, 2024-05-10T04:07:51.466 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000525970 2024-05-10T04:07:51.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.466 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-05-10T04:07:51.466 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-05-10T04:07:51.466 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 943, 2024-05-10T04:07:51.466 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 944 2024-05-10T04:07:51.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 943, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 943, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 0, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 588, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 0, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 352, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 0, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 0, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-05-10T04:07:51.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 2, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-05-10T04:07:51.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-05-10T04:07:51.469 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 87, 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 2, 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 5, 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 5, 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 0, 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 2, 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 0 2024-05-10T04:07:51.470 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 0, 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-05-10T04:07:51.471 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-05-10T04:07:51.472 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-05-10T04:07:51.472 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.472 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:07:51.472 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:51.472 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:07:51.472 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:51.472 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 1049, 2024-05-10T04:07:51.472 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 1138633, 2024-05-10T04:07:51.472 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 1049, 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1049, 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1138633, 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:51.473 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:07:51.474 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:51.474 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:51.474 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:51.474 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:51.474 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:51.474 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 943, 2024-05-10T04:07:51.474 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 518602, 2024-05-10T04:07:51.474 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 943, 2024-05-10T04:07:51.474 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 518602, 2024-05-10T04:07:51.474 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:51.474 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.474 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.474 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.475 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:51.475 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.475 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-05-10T04:07:51.475 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:51.475 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-05-10T04:07:51.475 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:51.475 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:51.475 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:51.475 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 943, 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 943, 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 943, 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 943, 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:51.477 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:51.478 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55e4981364e0": { 2024-05-10T04:07:51.478 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:51.478 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:07:51.478 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:51.478 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:51.478 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:51.478 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:51.478 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:51.478 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:07:51.478 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:07:51.478 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-05-10T04:07:51.478 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-05-10T04:07:51.478 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:51.479 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:51.479 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:51.479 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:51.479 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:51.479 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:51.479 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-10T04:07:51.833 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-10T04:07:51.833 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-10T04:07:51.833 DEBUG:tasks.check_counter:Getting stats from i 2024-05-10T04:07:51.833 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-10T04:07:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:07:52.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:52 smithi181 ceph-mon[46602]: pgmap v388: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:52.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:52 smithi150 ceph-mon[42477]: pgmap v388: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:52.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:52 smithi161 ceph-mon[46918]: pgmap v388: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:53.103 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-10T04:07:53.103 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:07:53.103 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 135, 2024-05-10T04:07:53.103 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 207, 2024-05-10T04:07:53.103 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 31696, 2024-05-10T04:07:53.103 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 142139, 2024-05-10T04:07:53.103 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 12, 2024-05-10T04:07:53.103 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.026624280, 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.008469148, 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.011622723, 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.002543744, 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 205, 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.029940565, 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000146051 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 288, 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000125643, 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000436 2024-05-10T04:07:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 164, 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 154, 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 475840, 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 141188, 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 14, 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 8, 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.026634210, 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.006829321, 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.019540959, 2024-05-10T04:07:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.002973661, 2024-05-10T04:07:53.106 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:53.106 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 154, 2024-05-10T04:07:53.106 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.019935402, 2024-05-10T04:07:53.106 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000129450 2024-05-10T04:07:53.106 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.106 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:53.106 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 17, 2024-05-10T04:07:53.106 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000003358, 2024-05-10T04:07:53.106 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000197 2024-05-10T04:07:53.106 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.106 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 471888, 2024-05-10T04:07:53.106 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 59152 2024-05-10T04:07:53.106 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 17, 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 86, 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 3738, 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 240609, 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 11, 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.010779102, 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.003899468, 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.005517160, 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000202334, 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 84, 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.014327260, 2024-05-10T04:07:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000170562 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 11, 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000003915, 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000355 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 191472 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 39, 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001868471, 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000047909 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000014750, 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000007375 2024-05-10T04:07:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 1, 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 1, 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000134129, 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000134129 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-05-10T04:07:53.110 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 11, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 10, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 0, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 10, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 10, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 1, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 1, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 1, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 0, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 0, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2024-05-10T04:07:53.111 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 4, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 234, 2024-05-10T04:07:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:07:53.113 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 1, 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:07:53.114 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-05-10T04:07:53.115 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 4, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 4, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 1, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 1, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-05-10T04:07:53.116 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 4195692, 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4, 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.009131974, 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002282993 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 13, 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 13, 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 0, 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 11, 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 12, 2024-05-10T04:07:53.117 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 1, 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 10, 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 10, 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 0, 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 1, 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 1, 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 115996, 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 199060 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 1, 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 1, 2024-05-10T04:07:53.118 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 8, 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.119 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.120 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000134129, 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000134129 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-05-10T04:07:53.121 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.122 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.124 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.125 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.127 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-05-10T04:07:53.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 2, 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 1, 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 1, 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 0, 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 218, 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-05-10T04:07:53.129 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:07:53.130 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:07:53.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 246503, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 83, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-05-10T04:07:53.132 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8696, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 166, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 50408, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 200, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-05-10T04:07:53.133 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 40, 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 11690, 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 33, 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 40, 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 40, 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.249995284, 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006249882 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-05-10T04:07:53.134 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 40, 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 62 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 40, 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 0, 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 40, 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 10, 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 0, 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 4, 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 7, 2024-05-10T04:07:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 6, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 0, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 20, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-05-10T04:07:53.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-05-10T04:07:53.137 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 87, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 2, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 9, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 9, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 14, 2024-05-10T04:07:53.138 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 1 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:07:53.139 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 312, 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 473115, 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 312, 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:07:53.140 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 312, 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 473115, 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:53.141 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 40, 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 11690, 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 40, 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 11690, 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-05-10T04:07:53.142 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 40, 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 40, 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 40, 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 40, 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.143 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.144 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x557cccaaa270": { 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 4, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 1388, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 4, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 4, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1388, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:53.145 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:53.146 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:53.146 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:53.146 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-10T04:07:53.483 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-10T04:07:53.483 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-10T04:07:53.483 DEBUG:tasks.check_counter:Getting stats from l 2024-05-10T04:07:53.483 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-10T04:07:53.708 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:07:54.528 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:54 smithi181 ceph-mon[46602]: pgmap v389: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:54.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:54 smithi150 ceph-mon[42477]: pgmap v389: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:54.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:54 smithi161 ceph-mon[46918]: pgmap v389: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:54.753 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-10T04:07:54.753 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:07:54.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 112, 2024-05-10T04:07:54.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 106, 2024-05-10T04:07:54.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 459666, 2024-05-10T04:07:54.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 56141, 2024-05-10T04:07:54.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 5, 2024-05-10T04:07:54.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 1, 2024-05-10T04:07:54.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.020929389, 2024-05-10T04:07:54.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.006055328, 2024-05-10T04:07:54.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.013611163, 2024-05-10T04:07:54.753 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.002589376, 2024-05-10T04:07:54.754 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:54.754 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 106, 2024-05-10T04:07:54.754 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.007910576, 2024-05-10T04:07:54.754 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000074628 2024-05-10T04:07:54.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.754 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:54.754 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.754 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.754 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.754 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 458832, 2024-05-10T04:07:54.754 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 55824 2024-05-10T04:07:54.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 234, 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 306, 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 57500, 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 262900, 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 5, 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.031808651, 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.009138215, 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.008488572, 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.010949110, 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 304, 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.017699624, 2024-05-10T04:07:54.755 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000058222 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 190432 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 600, 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 602, 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 180569, 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 186658, 2024-05-10T04:07:54.756 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.066762097, 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.019618682, 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.018446152, 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.022173346, 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 600, 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.031433228, 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000052388 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.757 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1388, 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.047781700, 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000034424 2024-05-10T04:07:54.758 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 0, 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 0, 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-05-10T04:07:54.759 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 0, 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 0, 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 0, 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 0, 2024-05-10T04:07:54.760 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 1, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-05-10T04:07:54.761 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:07:54.762 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.763 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 0, 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-05-10T04:07:54.764 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 0, 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-05-10T04:07:54.765 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 4, 2024-05-10T04:07:54.766 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-05-10T04:07:54.766 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-05-10T04:07:54.766 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 0, 2024-05-10T04:07:54.766 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-05-10T04:07:54.766 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-05-10T04:07:54.766 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 1, 2024-05-10T04:07:54.766 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-05-10T04:07:54.766 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-05-10T04:07:54.766 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-05-10T04:07:54.766 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 4195692, 2024-05-10T04:07:54.766 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 4195551, 2024-05-10T04:07:54.766 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 4 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 3, 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 3, 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 0, 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 1, 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 1, 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 0, 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 0, 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 0, 2024-05-10T04:07:54.767 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 0, 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 0, 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 115896, 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 207252 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 0, 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 0, 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 0, 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-05-10T04:07:54.768 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-05-10T04:07:54.769 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.770 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.770 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.770 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.770 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-05-10T04:07:54.770 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.770 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.770 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.770 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.770 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-05-10T04:07:54.770 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.770 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.770 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.770 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.771 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.772 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-05-10T04:07:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.774 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.774 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.774 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.774 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-05-10T04:07:54.774 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.774 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.774 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.774 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.774 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-05-10T04:07:54.774 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.774 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.774 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.774 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.775 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.776 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.777 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.778 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 2, 2024-05-10T04:07:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 1, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 2, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 0, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 0, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 3018, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:07:54.780 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:07:54.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 189256, 2024-05-10T04:07:54.782 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 49, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8696, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 166, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-05-10T04:07:54.783 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 8320, 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 13, 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 834, 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 21, 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 3, 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 834, 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-05-10T04:07:54.784 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 834, 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1.393973395, 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001671430 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 834, 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 835 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 834, 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 834, 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 0, 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 528, 2024-05-10T04:07:54.785 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 0, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 304, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 0, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 0, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 0, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 1, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 0, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:07:54.786 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-05-10T04:07:54.787 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 87, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 2, 2024-05-10T04:07:54.788 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 5, 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 5, 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 0, 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 2, 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 0 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 0, 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.789 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-05-10T04:07:54.790 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-05-10T04:07:54.790 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.790 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-05-10T04:07:54.790 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-05-10T04:07:54.790 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:07:54.790 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-05-10T04:07:54.790 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-05-10T04:07:54.790 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-05-10T04:07:54.790 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-05-10T04:07:54.790 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.790 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:07:54.790 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:54.790 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 942, 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 604454, 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 942, 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 942, 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 604454, 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:54.791 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.792 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-05-10T04:07:54.792 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:54.792 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:07:54.792 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:54.792 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:54.792 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:54.792 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:54.792 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:54.792 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 834, 2024-05-10T04:07:54.792 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 1393, 2024-05-10T04:07:54.792 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 834, 2024-05-10T04:07:54.792 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1393, 2024-05-10T04:07:54.792 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 834, 2024-05-10T04:07:54.793 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 834, 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 834, 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 834, 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:54.794 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x5555f69a68f0": { 2024-05-10T04:07:54.795 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:54.796 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:07:54.797 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-10T04:07:55.133 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-10T04:07:55.134 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-10T04:07:55.134 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-10T04:07:55.147 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-10T04:07:55.147 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:07:55.147 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T04:07:55.172 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:07:55.172 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T04:07:55.234 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:07:55.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.418+0000 7f39068fd700 1 -- 172.21.15.150:0/78831976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3900102580 msgr2=0x7f3900108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:55.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.418+0000 7f39068fd700 1 --2- 172.21.15.150:0/78831976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3900102580 0x7f3900108e30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f38f800b0a0 tx=0x7f38f800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.418+0000 7f39068fd700 1 -- 172.21.15.150:0/78831976 shutdown_connections 2024-05-10T04:07:55.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.418+0000 7f39068fd700 1 --2- 172.21.15.150:0/78831976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3900102580 0x7f3900108e30 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.418+0000 7f39068fd700 1 --2- 172.21.15.150:0/78831976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3900100ac0 0x7f3900100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.418+0000 7f39068fd700 1 --2- 172.21.15.150:0/78831976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3900101be0 0x7f3900101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.418+0000 7f39068fd700 1 -- 172.21.15.150:0/78831976 >> 172.21.15.150:0/78831976 conn(0x7f39000fc0d0 msgr2=0x7f39000fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:55.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.418+0000 7f39068fd700 1 -- 172.21.15.150:0/78831976 shutdown_connections 2024-05-10T04:07:55.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.418+0000 7f39068fd700 1 -- 172.21.15.150:0/78831976 wait complete. 2024-05-10T04:07:55.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.419+0000 7f39068fd700 1 Processor -- start 2024-05-10T04:07:55.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.419+0000 7f39068fd700 1 -- start start 2024-05-10T04:07:55.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f39068fd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3900100ac0 0x7f3900196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f39068fd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3900101be0 0x7f39001974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f39068fd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3900102580 0x7f390019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f39068fd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390019ea10 con 0x7f3900100ac0 2024-05-10T04:07:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f39068fd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390019eb50 con 0x7f3900101be0 2024-05-10T04:07:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f39068fd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390019ece0 con 0x7f3900102580 2024-05-10T04:07:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f38fffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3900100ac0 0x7f3900196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f38fffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3900100ac0 0x7f3900196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59238/0 (socket says 172.21.15.150:59238) 2024-05-10T04:07:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f38fffff700 1 -- 172.21.15.150:0/597767733 learned_addr learned my addr 172.21.15.150:0/597767733 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f3904e9a700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3900102580 0x7f390019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f38ff7fe700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3900101be0 0x7f39001974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f38fffff700 1 -- 172.21.15.150:0/597767733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3900102580 msgr2=0x7f390019e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f38fffff700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3900102580 0x7f390019e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f38fffff700 1 -- 172.21.15.150:0/597767733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3900101be0 msgr2=0x7f39001974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:55.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f38fffff700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3900101be0 0x7f39001974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f38fffff700 1 -- 172.21.15.150:0/597767733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f8009d00 con 0x7f3900100ac0 2024-05-10T04:07:55.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f38ff7fe700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3900101be0 0x7f39001974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:07:55.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.420+0000 7f38fffff700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3900100ac0 0x7f3900196f70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c9a0 tx=0x7f38e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:55.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.421+0000 7f38fd7fa700 1 -- 172.21.15.150:0/597767733 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8007ac0 con 0x7f3900100ac0 2024-05-10T04:07:55.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.421+0000 7f38fd7fa700 1 -- 172.21.15.150:0/597767733 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e8007c20 con 0x7f3900100ac0 2024-05-10T04:07:55.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.421+0000 7f39068fd700 1 -- 172.21.15.150:0/597767733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390019efc0 con 0x7f3900100ac0 2024-05-10T04:07:55.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.421+0000 7f38fd7fa700 1 -- 172.21.15.150:0/597767733 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8018660 con 0x7f3900100ac0 2024-05-10T04:07:55.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.421+0000 7f39068fd700 1 -- 172.21.15.150:0/597767733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390019f3f0 con 0x7f3900100ac0 2024-05-10T04:07:55.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.422+0000 7f39068fd700 1 -- 172.21.15.150:0/597767733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3900107cd0 con 0x7f3900100ac0 2024-05-10T04:07:55.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.423+0000 7f38fd7fa700 1 -- 172.21.15.150:0/597767733 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f38e800f450 con 0x7f3900100ac0 2024-05-10T04:07:55.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.423+0000 7f38fd7fa700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38ec0a1120 0x7f38ec0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:55.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.424+0000 7f38fd7fa700 1 -- 172.21.15.150:0/597767733 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f38e80cccb0 con 0x7f3900100ac0 2024-05-10T04:07:55.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.424+0000 7f38ff7fe700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38ec0a1120 0x7f38ec0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:55.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.424+0000 7f38ff7fe700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38ec0a1120 0x7f38ec0a3630 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f38f0005950 tx=0x7f38f00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:55.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.428+0000 7f38fd7fa700 1 -- 172.21.15.150:0/597767733 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f38e80961c0 con 0x7f3900100ac0 2024-05-10T04:07:55.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.631+0000 7f39068fd700 1 -- 172.21.15.150:0/597767733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f390010c2d0 con 0x7f3900100ac0 2024-05-10T04:07:55.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.633+0000 7f38fd7fa700 1 -- 172.21.15.150:0/597767733 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24554 (secure 0 0 0) 0x7f38e80997e0 con 0x7f3900100ac0 2024-05-10T04:07:55.634 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:07:55.634 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":87,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:03:30.185138+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:07:55.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.635+0000 7f39068fd700 1 -- 172.21.15.150:0/597767733 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38ec0a1120 msgr2=0x7f38ec0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:55.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.635+0000 7f39068fd700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38ec0a1120 0x7f38ec0a3630 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f38f0005950 tx=0x7f38f00058e0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.635+0000 7f39068fd700 1 -- 172.21.15.150:0/597767733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3900100ac0 msgr2=0x7f3900196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:55.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.635+0000 7f39068fd700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3900100ac0 0x7f3900196f70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c9a0 tx=0x7f38e800ccb0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.635+0000 7f39068fd700 1 -- 172.21.15.150:0/597767733 shutdown_connections 2024-05-10T04:07:55.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.635+0000 7f39068fd700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f38ec0a1120 0x7f38ec0a3630 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.635+0000 7f39068fd700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3900102580 0x7f390019e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.635+0000 7f39068fd700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3900100ac0 0x7f3900196f70 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.635+0000 7f39068fd700 1 --2- 172.21.15.150:0/597767733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3900101be0 0x7f39001974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.635+0000 7f39068fd700 1 -- 172.21.15.150:0/597767733 >> 172.21.15.150:0/597767733 conn(0x7f39000fc0d0 msgr2=0x7f3900104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:55.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.635+0000 7f39068fd700 1 -- 172.21.15.150:0/597767733 shutdown_connections 2024-05-10T04:07:55.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.635+0000 7f39068fd700 1 -- 172.21.15.150:0/597767733 wait complete. 2024-05-10T04:07:55.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:59:19.332343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:02:30.076694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:02:30.760710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:02:33.850714+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:07:55.650 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:07:55.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.788+0000 7f3fe1eed700 1 -- 172.21.15.150:0/2508731325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fdc1050f0 msgr2=0x7f3fdc107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:55.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.788+0000 7f3fe1eed700 1 --2- 172.21.15.150:0/2508731325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fdc1050f0 0x7f3fdc107510 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009990 tx=0x7f3fd0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.788+0000 7f3fe1eed700 1 -- 172.21.15.150:0/2508731325 shutdown_connections 2024-05-10T04:07:55.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.788+0000 7f3fe1eed700 1 --2- 172.21.15.150:0/2508731325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fdc107a50 0x7f3fdc109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.788+0000 7f3fe1eed700 1 --2- 172.21.15.150:0/2508731325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fdc0fff80 0x7f3fdc100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.788+0000 7f3fe1eed700 1 --2- 172.21.15.150:0/2508731325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fdc1050f0 0x7f3fdc107510 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.788+0000 7f3fe1eed700 1 -- 172.21.15.150:0/2508731325 >> 172.21.15.150:0/2508731325 conn(0x7f3fdc0fb8f0 msgr2=0x7f3fdc0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:55.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.788+0000 7f3fe1eed700 1 -- 172.21.15.150:0/2508731325 shutdown_connections 2024-05-10T04:07:55.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.788+0000 7f3fe1eed700 1 -- 172.21.15.150:0/2508731325 wait complete. 2024-05-10T04:07:55.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.788+0000 7f3fe1eed700 1 Processor -- start 2024-05-10T04:07:55.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fe1eed700 1 -- start start 2024-05-10T04:07:55.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fe1eed700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fdc0fff80 0x7f3fdc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:55.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fe1eed700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fdc1050f0 0x7f3fdc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:55.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fe1eed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fdc107a50 0x7f3fdc19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:55.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fe1eed700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc19f980 con 0x7f3fdc0fff80 2024-05-10T04:07:55.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fe1eed700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc19faf0 con 0x7f3fdc1050f0 2024-05-10T04:07:55.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fe1eed700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc19fcb0 con 0x7f3fdc107a50 2024-05-10T04:07:55.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fdb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fdc0fff80 0x7f3fdc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:55.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fdb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fdc0fff80 0x7f3fdc197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59256/0 (socket says 172.21.15.150:59256) 2024-05-10T04:07:55.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fdb7fe700 1 -- 172.21.15.150:0/3374436398 learned_addr learned my addr 172.21.15.150:0/3374436398 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:55.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fdaffd700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fdc1050f0 0x7f3fdc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:55.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fdbfff700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fdc107a50 0x7f3fdc19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:55.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fdb7fe700 1 -- 172.21.15.150:0/3374436398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fdc107a50 msgr2=0x7f3fdc19f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:55.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fdb7fe700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fdc107a50 0x7f3fdc19f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fdb7fe700 1 -- 172.21.15.150:0/3374436398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fdc1050f0 msgr2=0x7f3fdc1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:55.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fdb7fe700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fdc1050f0 0x7f3fdc1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fdb7fe700 1 -- 172.21.15.150:0/3374436398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd00095f0 con 0x7f3fdc0fff80 2024-05-10T04:07:55.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.789+0000 7f3fdaffd700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fdc1050f0 0x7f3fdc1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:07:55.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.790+0000 7f3fdb7fe700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fdc0fff80 0x7f3fdc197ea0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00e970 tx=0x7f3fcc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:55.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.790+0000 7f3fd8ff9700 1 -- 172.21.15.150:0/3374436398 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc009790 con 0x7f3fdc0fff80 2024-05-10T04:07:55.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.790+0000 7f3fd8ff9700 1 -- 172.21.15.150:0/3374436398 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fcc004cf0 con 0x7f3fdc0fff80 2024-05-10T04:07:55.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.790+0000 7f3fe1eed700 1 -- 172.21.15.150:0/3374436398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc19ff90 con 0x7f3fdc0fff80 2024-05-10T04:07:55.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.790+0000 7f3fd8ff9700 1 -- 172.21.15.150:0/3374436398 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc005680 con 0x7f3fdc0fff80 2024-05-10T04:07:55.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.790+0000 7f3fe1eed700 1 -- 172.21.15.150:0/3374436398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc1a03c0 con 0x7f3fdc0fff80 2024-05-10T04:07:55.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.791+0000 7f3fe1eed700 1 -- 172.21.15.150:0/3374436398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fdc0fd4a0 con 0x7f3fdc0fff80 2024-05-10T04:07:55.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.792+0000 7f3fd8ff9700 1 -- 172.21.15.150:0/3374436398 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3fcc004750 con 0x7f3fdc0fff80 2024-05-10T04:07:55.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.793+0000 7f3fd8ff9700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3fc40a11c0 0x7f3fc40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:55.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.793+0000 7f3fdaffd700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3fc40a11c0 0x7f3fc40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:55.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.793+0000 7f3fd8ff9700 1 -- 172.21.15.150:0/3374436398 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3fcc014070 con 0x7f3fdc0fff80 2024-05-10T04:07:55.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.793+0000 7f3fdaffd700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3fc40a11c0 0x7f3fc40a36d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000b2e0 tx=0x7f3fd0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:55.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.796+0000 7f3fd8ff9700 1 -- 172.21.15.150:0/3374436398 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3fcc0965d0 con 0x7f3fdc0fff80 2024-05-10T04:07:55.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.993+0000 7f3fe1eed700 1 -- 172.21.15.150:0/3374436398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fdc0678b0 con 0x7f3fdc0fff80 2024-05-10T04:07:55.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.994+0000 7f3fd8ff9700 1 -- 172.21.15.150:0/3374436398 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fcc099bf0 con 0x7f3fdc0fff80 2024-05-10T04:07:55.995 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:07:55.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.996+0000 7f3fe1eed700 1 -- 172.21.15.150:0/3374436398 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3fc40a11c0 msgr2=0x7f3fc40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:55.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.996+0000 7f3fe1eed700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3fc40a11c0 0x7f3fc40a36d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000b2e0 tx=0x7f3fd0005fd0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.996+0000 7f3fe1eed700 1 -- 172.21.15.150:0/3374436398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fdc0fff80 msgr2=0x7f3fdc197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:55.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.996+0000 7f3fe1eed700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fdc0fff80 0x7f3fdc197ea0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00e970 tx=0x7f3fcc00ed30 comp rx=0 tx=0).stop 2024-05-10T04:07:55.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.996+0000 7f3fe1eed700 1 -- 172.21.15.150:0/3374436398 shutdown_connections 2024-05-10T04:07:55.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.996+0000 7f3fe1eed700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f3fc40a11c0 0x7f3fc40a36d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.996+0000 7f3fe1eed700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fdc107a50 0x7f3fdc19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.996+0000 7f3fe1eed700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fdc0fff80 0x7f3fdc197ea0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.996+0000 7f3fe1eed700 1 --2- 172.21.15.150:0/3374436398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fdc1050f0 0x7f3fdc1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:55.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.996+0000 7f3fe1eed700 1 -- 172.21.15.150:0/3374436398 >> 172.21.15.150:0/3374436398 conn(0x7f3fdc0fb8f0 msgr2=0x7f3fdc106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:55.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.996+0000 7f3fe1eed700 1 -- 172.21.15.150:0/3374436398 shutdown_connections 2024-05-10T04:07:55.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:55.996+0000 7f3fe1eed700 1 -- 172.21.15.150:0/3374436398 wait complete. 2024-05-10T04:07:56.010 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:07:56.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.147+0000 7f5689c66700 1 -- 172.21.15.150:0/3102233008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5684100ac0 msgr2=0x7f5684100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:56.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.147+0000 7f5689c66700 1 --2- 172.21.15.150:0/3102233008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5684100ac0 0x7f5684100ed0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5678009990 tx=0x7f5678009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.147+0000 7f5689c66700 1 -- 172.21.15.150:0/3102233008 shutdown_connections 2024-05-10T04:07:56.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.147+0000 7f5689c66700 1 --2- 172.21.15.150:0/3102233008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5684102580 0x7f5684108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.147+0000 7f5689c66700 1 --2- 172.21.15.150:0/3102233008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5684100ac0 0x7f5684100ed0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.147+0000 7f5689c66700 1 --2- 172.21.15.150:0/3102233008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5684101be0 0x7f5684101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.147+0000 7f5689c66700 1 -- 172.21.15.150:0/3102233008 >> 172.21.15.150:0/3102233008 conn(0x7f56840fc0d0 msgr2=0x7f56840fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:56.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.147+0000 7f5689c66700 1 -- 172.21.15.150:0/3102233008 shutdown_connections 2024-05-10T04:07:56.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.147+0000 7f5689c66700 1 -- 172.21.15.150:0/3102233008 wait complete. 2024-05-10T04:07:56.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f5689c66700 1 Processor -- start 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f5689c66700 1 -- start start 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f5689c66700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5684100ac0 0x7f5684068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f5689c66700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5684101be0 0x7f568419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f5689c66700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5684102580 0x7f568419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f5689c66700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568419fa10 con 0x7f5684100ac0 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f5689c66700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568419fb80 con 0x7f5684102580 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f5689c66700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568419fd40 con 0x7f5684101be0 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f56837fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5684100ac0 0x7f5684068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f56837fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5684100ac0 0x7f5684068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59274/0 (socket says 172.21.15.150:59274) 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f56837fe700 1 -- 172.21.15.150:0/2148074551 learned_addr learned my addr 172.21.15.150:0/2148074551 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f5682ffd700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5684101be0 0x7f568419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f5683fff700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5684102580 0x7f568419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f56837fe700 1 -- 172.21.15.150:0/2148074551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5684101be0 msgr2=0x7f568419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:56.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f56837fe700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5684101be0 0x7f568419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f56837fe700 1 -- 172.21.15.150:0/2148074551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5684102580 msgr2=0x7f568419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:56.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.148+0000 7f56837fe700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5684102580 0x7f568419f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.149+0000 7f56837fe700 1 -- 172.21.15.150:0/2148074551 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56780095f0 con 0x7f5684100ac0 2024-05-10T04:07:56.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.149+0000 7f56837fe700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5684100ac0 0x7f5684068e50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5678006010 tx=0x7f567800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:56.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.149+0000 7f5680ff9700 1 -- 172.21.15.150:0/2148074551 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567801d070 con 0x7f5684100ac0 2024-05-10T04:07:56.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.149+0000 7f5689c66700 1 -- 172.21.15.150:0/2148074551 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568419ff60 con 0x7f5684100ac0 2024-05-10T04:07:56.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.149+0000 7f5680ff9700 1 -- 172.21.15.150:0/2148074551 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5678003960 con 0x7f5684100ac0 2024-05-10T04:07:56.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.149+0000 7f5680ff9700 1 -- 172.21.15.150:0/2148074551 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56780054f0 con 0x7f5684100ac0 2024-05-10T04:07:56.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.149+0000 7f5689c66700 1 -- 172.21.15.150:0/2148074551 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56841a0320 con 0x7f5684100ac0 2024-05-10T04:07:56.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.150+0000 7f5689c66700 1 -- 172.21.15.150:0/2148074551 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5684107cd0 con 0x7f5684100ac0 2024-05-10T04:07:56.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.151+0000 7f5680ff9700 1 -- 172.21.15.150:0/2148074551 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f567800fb40 con 0x7f5684100ac0 2024-05-10T04:07:56.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.152+0000 7f5680ff9700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f566c0a11c0 0x7f566c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:56.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.152+0000 7f5682ffd700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f566c0a11c0 0x7f566c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:56.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.152+0000 7f5680ff9700 1 -- 172.21.15.150:0/2148074551 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f56780ce1d0 con 0x7f5684100ac0 2024-05-10T04:07:56.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.152+0000 7f5682ffd700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f566c0a11c0 0x7f566c0a36d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56840687f0 tx=0x7f567400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:56.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.155+0000 7f5680ff9700 1 -- 172.21.15.150:0/2148074551 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f56780976e0 con 0x7f5684100ac0 2024-05-10T04:07:56.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.355+0000 7f5689c66700 1 -- 172.21.15.150:0/2148074551 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f568410c2a0 con 0x7f5684100ac0 2024-05-10T04:07:56.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.356+0000 7f5680ff9700 1 -- 172.21.15.150:0/2148074551 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f567809ad00 con 0x7f5684100ac0 2024-05-10T04:07:56.357 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:07:56.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.358+0000 7f5689c66700 1 -- 172.21.15.150:0/2148074551 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f566c0a11c0 msgr2=0x7f566c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:56.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.358+0000 7f5689c66700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f566c0a11c0 0x7f566c0a36d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56840687f0 tx=0x7f567400b410 comp rx=0 tx=0).stop 2024-05-10T04:07:56.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.358+0000 7f5689c66700 1 -- 172.21.15.150:0/2148074551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5684100ac0 msgr2=0x7f5684068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:56.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.358+0000 7f5689c66700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5684100ac0 0x7f5684068e50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5678006010 tx=0x7f567800f690 comp rx=0 tx=0).stop 2024-05-10T04:07:56.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.358+0000 7f5689c66700 1 -- 172.21.15.150:0/2148074551 shutdown_connections 2024-05-10T04:07:56.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.358+0000 7f5689c66700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f566c0a11c0 0x7f566c0a36d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.358+0000 7f5689c66700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5684102580 0x7f568419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.358+0000 7f5689c66700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5684100ac0 0x7f5684068e50 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.358+0000 7f5689c66700 1 --2- 172.21.15.150:0/2148074551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5684101be0 0x7f568419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.358+0000 7f5689c66700 1 -- 172.21.15.150:0/2148074551 >> 172.21.15.150:0/2148074551 conn(0x7f56840fc0d0 msgr2=0x7f5684104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:56.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.359+0000 7f5689c66700 1 -- 172.21.15.150:0/2148074551 shutdown_connections 2024-05-10T04:07:56.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.359+0000 7f5689c66700 1 -- 172.21.15.150:0/2148074551 wait complete. 2024-05-10T04:07:56.372 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:07:56.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.507+0000 7f2a51b2b700 1 -- 172.21.15.150:0/3861455737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4c101be0 msgr2=0x7f2a4c101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:56.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.507+0000 7f2a51b2b700 1 --2- 172.21.15.150:0/3861455737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4c101be0 0x7f2a4c101ff0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009990 tx=0x7f2a34009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.507+0000 7f2a51b2b700 1 -- 172.21.15.150:0/3861455737 shutdown_connections 2024-05-10T04:07:56.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.507+0000 7f2a51b2b700 1 --2- 172.21.15.150:0/3861455737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a4c102580 0x7f2a4c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.507+0000 7f2a51b2b700 1 --2- 172.21.15.150:0/3861455737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c100ac0 0x7f2a4c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.507+0000 7f2a51b2b700 1 --2- 172.21.15.150:0/3861455737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4c101be0 0x7f2a4c101ff0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.507+0000 7f2a51b2b700 1 -- 172.21.15.150:0/3861455737 >> 172.21.15.150:0/3861455737 conn(0x7f2a4c0fc0d0 msgr2=0x7f2a4c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:56.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.507+0000 7f2a51b2b700 1 -- 172.21.15.150:0/3861455737 shutdown_connections 2024-05-10T04:07:56.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.507+0000 7f2a51b2b700 1 -- 172.21.15.150:0/3861455737 wait complete. 2024-05-10T04:07:56.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.508+0000 7f2a51b2b700 1 Processor -- start 2024-05-10T04:07:56.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.508+0000 7f2a51b2b700 1 -- start start 2024-05-10T04:07:56.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.508+0000 7f2a51b2b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c100ac0 0x7f2a4c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:56.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.508+0000 7f2a51b2b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4c101be0 0x7f2a4c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:56.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.508+0000 7f2a51b2b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a4c102580 0x7f2a4c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.508+0000 7f2a51b2b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c19f990 con 0x7f2a4c101be0 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.508+0000 7f2a51b2b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c19fb00 con 0x7f2a4c100ac0 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.508+0000 7f2a51b2b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c19fcc0 con 0x7f2a4c102580 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a4affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4c101be0 0x7f2a4c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a4affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4c101be0 0x7f2a4c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59288/0 (socket says 172.21.15.150:59288) 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a4affd700 1 -- 172.21.15.150:0/543102732 learned_addr learned my addr 172.21.15.150:0/543102732 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a4bfff700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a4c102580 0x7f2a4c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a4b7fe700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c100ac0 0x7f2a4c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a4affd700 1 -- 172.21.15.150:0/543102732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a4c102580 msgr2=0x7f2a4c19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a4affd700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a4c102580 0x7f2a4c19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a4affd700 1 -- 172.21.15.150:0/543102732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c100ac0 msgr2=0x7f2a4c197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a4affd700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c100ac0 0x7f2a4c197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a4affd700 1 -- 172.21.15.150:0/543102732 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a340095f0 con 0x7f2a4c101be0 2024-05-10T04:07:56.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a4affd700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4c101be0 0x7f2a4c1983f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005b40 tx=0x7f2a3400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:56.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a48ff9700 1 -- 172.21.15.150:0/543102732 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3401d070 con 0x7f2a4c101be0 2024-05-10T04:07:56.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a51b2b700 1 -- 172.21.15.150:0/543102732 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4c19ff40 con 0x7f2a4c101be0 2024-05-10T04:07:56.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a48ff9700 1 -- 172.21.15.150:0/543102732 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a3400fb80 con 0x7f2a4c101be0 2024-05-10T04:07:56.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a48ff9700 1 -- 172.21.15.150:0/543102732 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340173f0 con 0x7f2a4c101be0 2024-05-10T04:07:56.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.509+0000 7f2a51b2b700 1 -- 172.21.15.150:0/543102732 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4c1a02b0 con 0x7f2a4c101be0 2024-05-10T04:07:56.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.510+0000 7f2a51b2b700 1 -- 172.21.15.150:0/543102732 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a4c107cd0 con 0x7f2a4c101be0 2024-05-10T04:07:56.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.511+0000 7f2a48ff9700 1 -- 172.21.15.150:0/543102732 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2a3400fcf0 con 0x7f2a4c101be0 2024-05-10T04:07:56.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.512+0000 7f2a48ff9700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2a380a11c0 0x7f2a380a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:56.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.512+0000 7f2a48ff9700 1 -- 172.21.15.150:0/543102732 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2a340ce250 con 0x7f2a4c101be0 2024-05-10T04:07:56.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.512+0000 7f2a4b7fe700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2a380a11c0 0x7f2a380a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:56.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.512+0000 7f2a4b7fe700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2a380a11c0 0x7f2a380a36d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c101ae0 tx=0x7f2a44009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:56.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.516+0000 7f2a48ff9700 1 -- 172.21.15.150:0/543102732 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2a34097760 con 0x7f2a4c101be0 2024-05-10T04:07:56.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:56 smithi150 ceph-mon[42477]: pgmap v390: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:56.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:56 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/597767733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:07:56.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:56 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3374436398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:07:56.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.716+0000 7f2a51b2b700 1 -- 172.21.15.150:0/543102732 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a4c10d410 con 0x7f2a4c101be0 2024-05-10T04:07:56.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.716+0000 7f2a48ff9700 1 -- 172.21.15.150:0/543102732 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a340a1070 con 0x7f2a4c101be0 2024-05-10T04:07:56.717 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:07:56.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.719+0000 7f2a51b2b700 1 -- 172.21.15.150:0/543102732 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2a380a11c0 msgr2=0x7f2a380a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:56.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.719+0000 7f2a51b2b700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2a380a11c0 0x7f2a380a36d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c101ae0 tx=0x7f2a44009040 comp rx=0 tx=0).stop 2024-05-10T04:07:56.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.719+0000 7f2a51b2b700 1 -- 172.21.15.150:0/543102732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4c101be0 msgr2=0x7f2a4c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:56.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.719+0000 7f2a51b2b700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4c101be0 0x7f2a4c1983f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005b40 tx=0x7f2a3400f690 comp rx=0 tx=0).stop 2024-05-10T04:07:56.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.719+0000 7f2a51b2b700 1 -- 172.21.15.150:0/543102732 shutdown_connections 2024-05-10T04:07:56.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.719+0000 7f2a51b2b700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2a380a11c0 0x7f2a380a36d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.719+0000 7f2a51b2b700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a4c102580 0x7f2a4c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.719+0000 7f2a51b2b700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c100ac0 0x7f2a4c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.719+0000 7f2a51b2b700 1 --2- 172.21.15.150:0/543102732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4c101be0 0x7f2a4c1983f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.719+0000 7f2a51b2b700 1 -- 172.21.15.150:0/543102732 >> 172.21.15.150:0/543102732 conn(0x7f2a4c0fc0d0 msgr2=0x7f2a4c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:56.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.719+0000 7f2a51b2b700 1 -- 172.21.15.150:0/543102732 shutdown_connections 2024-05-10T04:07:56.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.719+0000 7f2a51b2b700 1 -- 172.21.15.150:0/543102732 wait complete. 2024-05-10T04:07:56.733 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:07:56.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:56 smithi181 ceph-mon[46602]: pgmap v390: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:56.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:56 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/597767733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:07:56.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:56 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3374436398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:07:56.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:56 smithi161 ceph-mon[46918]: pgmap v390: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T04:07:56.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:56 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/597767733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:07:56.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:56 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3374436398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:07:56.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.867+0000 7f660a8fe700 1 -- 172.21.15.150:0/3108548568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6604101060 msgr2=0x7f6604101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:56.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.867+0000 7f660a8fe700 1 --2- 172.21.15.150:0/3108548568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6604101060 0x7f6604101470 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009990 tx=0x7f65f4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.867+0000 7f660a8fe700 1 -- 172.21.15.150:0/3108548568 shutdown_connections 2024-05-10T04:07:56.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.867+0000 7f660a8fe700 1 --2- 172.21.15.150:0/3108548568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66041028a0 0x7f66041091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.867+0000 7f660a8fe700 1 --2- 172.21.15.150:0/3108548568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6604101060 0x7f6604101470 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.867+0000 7f660a8fe700 1 --2- 172.21.15.150:0/3108548568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6604101f50 0x7f6604102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.867+0000 7f660a8fe700 1 -- 172.21.15.150:0/3108548568 >> 172.21.15.150:0/3108548568 conn(0x7f66040fb800 msgr2=0x7f66040fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:56.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.867+0000 7f660a8fe700 1 -- 172.21.15.150:0/3108548568 shutdown_connections 2024-05-10T04:07:56.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.867+0000 7f660a8fe700 1 -- 172.21.15.150:0/3108548568 wait complete. 2024-05-10T04:07:56.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.867+0000 7f660a8fe700 1 Processor -- start 2024-05-10T04:07:56.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.867+0000 7f660a8fe700 1 -- start start 2024-05-10T04:07:56.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f660a8fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6604101060 0x7f6604196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:56.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f660a8fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6604101f50 0x7f6604197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:56.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f660a8fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66041028a0 0x7f660419e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:56.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f660a8fe700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660419ea70 con 0x7f66041028a0 2024-05-10T04:07:56.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f660a8fe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660419ebb0 con 0x7f6604101060 2024-05-10T04:07:56.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f660a8fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660419ed40 con 0x7f6604101f50 2024-05-10T04:07:56.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f6608e9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66041028a0 0x7f660419e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:56.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f6608e9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66041028a0 0x7f660419e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59296/0 (socket says 172.21.15.150:59296) 2024-05-10T04:07:56.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f6608e9b700 1 -- 172.21.15.150:0/4014463385 learned_addr learned my addr 172.21.15.150:0/4014463385 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:56.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f66037fe700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6604101f50 0x7f6604197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:56.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f6603fff700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6604101060 0x7f6604196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:56.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f6608e9b700 1 -- 172.21.15.150:0/4014463385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6604101f50 msgr2=0x7f6604197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:56.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f6608e9b700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6604101f50 0x7f6604197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f6608e9b700 1 -- 172.21.15.150:0/4014463385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6604101060 msgr2=0x7f6604196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:56.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f6608e9b700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6604101060 0x7f6604196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:56.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f6608e9b700 1 -- 172.21.15.150:0/4014463385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f40095f0 con 0x7f66041028a0 2024-05-10T04:07:56.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.868+0000 7f6608e9b700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66041028a0 0x7f660419e4a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f65fc007f10 tx=0x7f65fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:56.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.869+0000 7f66017fa700 1 -- 172.21.15.150:0/4014463385 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00ddd0 con 0x7f66041028a0 2024-05-10T04:07:56.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.869+0000 7f66017fa700 1 -- 172.21.15.150:0/4014463385 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65fc00f040 con 0x7f66041028a0 2024-05-10T04:07:56.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.869+0000 7f66017fa700 1 -- 172.21.15.150:0/4014463385 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc015400 con 0x7f66041028a0 2024-05-10T04:07:56.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.869+0000 7f660a8fe700 1 -- 172.21.15.150:0/4014463385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660419efc0 con 0x7f66041028a0 2024-05-10T04:07:56.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.869+0000 7f660a8fe700 1 -- 172.21.15.150:0/4014463385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660419f420 con 0x7f66041028a0 2024-05-10T04:07:56.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.870+0000 7f660a8fe700 1 -- 172.21.15.150:0/4014463385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6604108040 con 0x7f66041028a0 2024-05-10T04:07:56.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.871+0000 7f66017fa700 1 -- 172.21.15.150:0/4014463385 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f65fc004000 con 0x7f66041028a0 2024-05-10T04:07:56.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.871+0000 7f66017fa700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f65ec0a1240 0x7f65ec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:56.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.872+0000 7f6603fff700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f65ec0a1240 0x7f65ec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:56.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.872+0000 7f66017fa700 1 -- 172.21.15.150:0/4014463385 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f65fc0cccb0 con 0x7f66041028a0 2024-05-10T04:07:56.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.872+0000 7f6603fff700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f65ec0a1240 0x7f65ec0a3750 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f65f4006010 tx=0x7f65f401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:56.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:56.875+0000 7f66017fa700 1 -- 172.21.15.150:0/4014463385 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f65fc096130 con 0x7f66041028a0 2024-05-10T04:07:57.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.072+0000 7f660a8fe700 1 -- 172.21.15.150:0/4014463385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f660410c490 con 0x7f66041028a0 2024-05-10T04:07:57.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.073+0000 7f66017fa700 1 -- 172.21.15.150:0/4014463385 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f65fc023020 con 0x7f66041028a0 2024-05-10T04:07:57.074 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:07:57.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.075+0000 7f660a8fe700 1 -- 172.21.15.150:0/4014463385 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f65ec0a1240 msgr2=0x7f65ec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:57.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.075+0000 7f660a8fe700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f65ec0a1240 0x7f65ec0a3750 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f65f4006010 tx=0x7f65f401a040 comp rx=0 tx=0).stop 2024-05-10T04:07:57.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.075+0000 7f660a8fe700 1 -- 172.21.15.150:0/4014463385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66041028a0 msgr2=0x7f660419e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:57.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.075+0000 7f660a8fe700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66041028a0 0x7f660419e4a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f65fc007f10 tx=0x7f65fc00d3b0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.075+0000 7f660a8fe700 1 -- 172.21.15.150:0/4014463385 shutdown_connections 2024-05-10T04:07:57.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.075+0000 7f660a8fe700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f65ec0a1240 0x7f65ec0a3750 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.075+0000 7f660a8fe700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66041028a0 0x7f660419e4a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.075+0000 7f660a8fe700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6604101060 0x7f6604196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.075+0000 7f660a8fe700 1 --2- 172.21.15.150:0/4014463385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6604101f50 0x7f6604197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.075+0000 7f660a8fe700 1 -- 172.21.15.150:0/4014463385 >> 172.21.15.150:0/4014463385 conn(0x7f66040fb800 msgr2=0x7f6604105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:57.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.075+0000 7f660a8fe700 1 -- 172.21.15.150:0/4014463385 shutdown_connections 2024-05-10T04:07:57.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.075+0000 7f660a8fe700 1 -- 172.21.15.150:0/4014463385 wait complete. 2024-05-10T04:07:57.089 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T04:07:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.223+0000 7ff4eb26e700 1 -- 172.21.15.150:0/3354869537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4e41010d0 msgr2=0x7ff4e41014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.223+0000 7ff4eb26e700 1 --2- 172.21.15.150:0/3354869537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4e41010d0 0x7ff4e41014e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0009990 tx=0x7ff4e0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.223+0000 7ff4eb26e700 1 -- 172.21.15.150:0/3354869537 shutdown_connections 2024-05-10T04:07:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.223+0000 7ff4eb26e700 1 --2- 172.21.15.150:0/3354869537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e41025d0 0x7ff4e4109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.223+0000 7ff4eb26e700 1 --2- 172.21.15.150:0/3354869537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4e41010d0 0x7ff4e41014e0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.223+0000 7ff4eb26e700 1 --2- 172.21.15.150:0/3354869537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e4101ab0 0x7ff4e4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.223+0000 7ff4eb26e700 1 -- 172.21.15.150:0/3354869537 >> 172.21.15.150:0/3354869537 conn(0x7ff4e40fb8f0 msgr2=0x7ff4e40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.223+0000 7ff4eb26e700 1 -- 172.21.15.150:0/3354869537 shutdown_connections 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.223+0000 7ff4eb26e700 1 -- 172.21.15.150:0/3354869537 wait complete. 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4eb26e700 1 Processor -- start 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4eb26e700 1 -- start start 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4eb26e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4e41010d0 0x7ff4e4196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4eb26e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e4101ab0 0x7ff4e41973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4eb26e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e41025d0 0x7ff4e419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4eb26e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e419e940 con 0x7ff4e41010d0 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4eb26e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e419eab0 con 0x7ff4e41025d0 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4eb26e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e419ec70 con 0x7ff4e4101ab0 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4e900a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4e41010d0 0x7ff4e4196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4e900a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4e41010d0 0x7ff4e4196e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59316/0 (socket says 172.21.15.150:59316) 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4e900a700 1 -- 172.21.15.150:0/344394070 learned_addr learned my addr 172.21.15.150:0/344394070 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4e900a700 1 -- 172.21.15.150:0/344394070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e4101ab0 msgr2=0x7ff4e41973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4e8809700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e4101ab0 0x7ff4e41973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4e980b700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e41025d0 0x7ff4e419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4e900a700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e4101ab0 0x7ff4e41973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4e900a700 1 -- 172.21.15.150:0/344394070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e41025d0 msgr2=0x7ff4e419e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4e900a700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e41025d0 0x7ff4e419e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.224+0000 7ff4e900a700 1 -- 172.21.15.150:0/344394070 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e00095f0 con 0x7ff4e41010d0 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.225+0000 7ff4e900a700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4e41010d0 0x7ff4e4196e60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0005b40 tx=0x7ff4e000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.225+0000 7ff4da7fc700 1 -- 172.21.15.150:0/344394070 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e001d070 con 0x7ff4e41010d0 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.225+0000 7ff4da7fc700 1 -- 172.21.15.150:0/344394070 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4e00054f0 con 0x7ff4e41010d0 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.225+0000 7ff4da7fc700 1 -- 172.21.15.150:0/344394070 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0004ba0 con 0x7ff4e41010d0 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.225+0000 7ff4eb26e700 1 -- 172.21.15.150:0/344394070 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e419eef0 con 0x7ff4e41010d0 2024-05-10T04:07:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.225+0000 7ff4eb26e700 1 -- 172.21.15.150:0/344394070 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e419f260 con 0x7ff4e41010d0 2024-05-10T04:07:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.226+0000 7ff4eb26e700 1 -- 172.21.15.150:0/344394070 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e4107f30 con 0x7ff4e41010d0 2024-05-10T04:07:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.227+0000 7ff4da7fc700 1 -- 172.21.15.150:0/344394070 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff4e0004d00 con 0x7ff4e41010d0 2024-05-10T04:07:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.228+0000 7ff4da7fc700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff4d00a11c0 0x7ff4d00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:57.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.228+0000 7ff4e8809700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff4d00a11c0 0x7ff4d00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:57.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.228+0000 7ff4da7fc700 1 -- 172.21.15.150:0/344394070 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff4e00cdfa0 con 0x7ff4e41010d0 2024-05-10T04:07:57.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.228+0000 7ff4e8809700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff4d00a11c0 0x7ff4d00a36d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4005950 tx=0x7ff4d40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.231+0000 7ff4da7fc700 1 -- 172.21.15.150:0/344394070 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff4e0097570 con 0x7ff4e41010d0 2024-05-10T04:07:57.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.448+0000 7ff4eb26e700 1 -- 172.21.15.150:0/344394070 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff4e40678b0 con 0x7ff4e41010d0 2024-05-10T04:07:57.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.449+0000 7ff4da7fc700 1 -- 172.21.15.150:0/344394070 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+597 (secure 0 0 0) 0x7ff4e0026080 con 0x7ff4e41010d0 2024-05-10T04:07:57.450 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3457770606 2024-05-11T03:57:29.013448+0000 2024-05-10T04:07:57.450 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6801/999775795 2024-05-11T03:57:29.013448+0000 2024-05-10T04:07:57.450 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/977507934 2024-05-11T03:57:08.747290+0000 2024-05-10T04:07:57.450 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6800/3392737437 2024-05-11T03:57:08.747290+0000 2024-05-10T04:07:57.450 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3909466387 2024-05-11T03:57:29.013448+0000 2024-05-10T04:07:57.450 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/2393499570 2024-05-11T03:57:29.013448+0000 2024-05-10T04:07:57.451 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6800/999775795 2024-05-11T03:57:29.013448+0000 2024-05-10T04:07:57.451 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6801/3392737437 2024-05-11T03:57:08.747290+0000 2024-05-10T04:07:57.451 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/679926778 2024-05-11T03:57:08.747290+0000 2024-05-10T04:07:57.451 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/431676611 2024-05-11T03:57:08.747290+0000 2024-05-10T04:07:57.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.451+0000 7ff4eb26e700 1 -- 172.21.15.150:0/344394070 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff4d00a11c0 msgr2=0x7ff4d00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:57.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.451+0000 7ff4eb26e700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff4d00a11c0 0x7ff4d00a36d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4005950 tx=0x7ff4d40058e0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.451+0000 7ff4eb26e700 1 -- 172.21.15.150:0/344394070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4e41010d0 msgr2=0x7ff4e4196e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:57.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.451+0000 7ff4eb26e700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4e41010d0 0x7ff4e4196e60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0005b40 tx=0x7ff4e000f740 comp rx=0 tx=0).stop 2024-05-10T04:07:57.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.451+0000 7ff4eb26e700 1 -- 172.21.15.150:0/344394070 shutdown_connections 2024-05-10T04:07:57.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.451+0000 7ff4eb26e700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7ff4d00a11c0 0x7ff4d00a36d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.451+0000 7ff4eb26e700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e41025d0 0x7ff4e419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.451+0000 7ff4eb26e700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4e41010d0 0x7ff4e4196e60 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.451+0000 7ff4eb26e700 1 --2- 172.21.15.150:0/344394070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e4101ab0 0x7ff4e41973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.451+0000 7ff4eb26e700 1 -- 172.21.15.150:0/344394070 >> 172.21.15.150:0/344394070 conn(0x7ff4e40fb8f0 msgr2=0x7ff4e4105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:57.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.451+0000 7ff4eb26e700 1 -- 172.21.15.150:0/344394070 shutdown_connections 2024-05-10T04:07:57.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.451+0000 7ff4eb26e700 1 -- 172.21.15.150:0/344394070 wait complete. 2024-05-10T04:07:57.454 INFO:teuthology.orchestra.run.smithi150.stderr:listed 10 entries 2024-05-10T04:07:57.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:57 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2148074551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:07:57.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:57 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/543102732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:07:57.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:57 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/4014463385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:07:57.465 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T04:07:57.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:07:57.465 DEBUG:teuthology.orchestra.run.smithi150:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T04:07:57.527 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:07:57.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:07:57.527 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T04:07:57.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:07:57.562 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-05-10T04:07:57.620 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T04:07:57.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:07:57.620 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T04:07:57.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:07:57.699 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-05-10T04:07:57.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:57 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2148074551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:07:57.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:57 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/543102732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:07:57.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:57 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/4014463385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:07:57.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:57 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2148074551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:07:57.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:57 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/543102732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:07:57.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:57 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/4014463385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:07:57.797 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-10T04:07:57.808 INFO:tasks.cephadm:Teardown begin 2024-05-10T04:07:57.808 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T04:07:57.808 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:07:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.942+0000 7f824fd8a700 1 -- 172.21.15.150:0/2244806879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82481031a0 msgr2=0x7f82481035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.942+0000 7f824fd8a700 1 --2- 172.21.15.150:0/2244806879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82481031a0 0x7f82481035b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8244009990 tx=0x7f8244009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.942+0000 7f824fd8a700 1 -- 172.21.15.150:0/2244806879 shutdown_connections 2024-05-10T04:07:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.942+0000 7f824fd8a700 1 --2- 172.21.15.150:0/2244806879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8248100530 0x7f824810a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.942+0000 7f824fd8a700 1 --2- 172.21.15.150:0/2244806879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82481031a0 0x7f82481035b0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.942+0000 7f824fd8a700 1 --2- 172.21.15.150:0/2244806879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82480ffbe0 0x7f82480ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.942+0000 7f824fd8a700 1 -- 172.21.15.150:0/2244806879 >> 172.21.15.150:0/2244806879 conn(0x7f82480fb8b0 msgr2=0x7f82480fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:57.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.942+0000 7f824fd8a700 1 -- 172.21.15.150:0/2244806879 shutdown_connections 2024-05-10T04:07:57.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.942+0000 7f824fd8a700 1 -- 172.21.15.150:0/2244806879 wait complete. 2024-05-10T04:07:57.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824fd8a700 1 Processor -- start 2024-05-10T04:07:57.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824fd8a700 1 -- start start 2024-05-10T04:07:57.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824fd8a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82480ffbe0 0x7f8248197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:57.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824fd8a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8248100530 0x7f82481984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:57.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824fd8a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82481031a0 0x7f824819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:57.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824fd8a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824819f9c0 con 0x7f8248100530 2024-05-10T04:07:57.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824fd8a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824819fb00 con 0x7f82480ffbe0 2024-05-10T04:07:57.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824fd8a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824819fc90 con 0x7f82481031a0 2024-05-10T04:07:57.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824d325700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8248100530 0x7f82481984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:57.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824d325700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8248100530 0x7f82481984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59346/0 (socket says 172.21.15.150:59346) 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824d325700 1 -- 172.21.15.150:0/3492290157 learned_addr learned my addr 172.21.15.150:0/3492290157 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824db26700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82480ffbe0 0x7f8248197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824e327700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82481031a0 0x7f824819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824d325700 1 -- 172.21.15.150:0/3492290157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82481031a0 msgr2=0x7f824819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824d325700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82481031a0 0x7f824819f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824d325700 1 -- 172.21.15.150:0/3492290157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82480ffbe0 msgr2=0x7f8248197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824d325700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82480ffbe0 0x7f8248197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.943+0000 7f824d325700 1 -- 172.21.15.150:0/3492290157 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82440095f0 con 0x7f8248100530 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.944+0000 7f824d325700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8248100530 0x7f82481984f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f823800c9e0 tx=0x7f823800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.944+0000 7f823effd700 1 -- 172.21.15.150:0/3492290157 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238007ac0 con 0x7f8248100530 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.944+0000 7f823effd700 1 -- 172.21.15.150:0/3492290157 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8238007c20 con 0x7f8248100530 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.944+0000 7f824fd8a700 1 -- 172.21.15.150:0/3492290157 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824819ff70 con 0x7f8248100530 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.944+0000 7f823effd700 1 -- 172.21.15.150:0/3492290157 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238018660 con 0x7f8248100530 2024-05-10T04:07:57.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.944+0000 7f824fd8a700 1 -- 172.21.15.150:0/3492290157 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82481a03f0 con 0x7f8248100530 2024-05-10T04:07:57.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.945+0000 7f824fd8a700 1 -- 172.21.15.150:0/3492290157 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8248108ff0 con 0x7f8248100530 2024-05-10T04:07:57.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.946+0000 7f823effd700 1 -- 172.21.15.150:0/3492290157 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f823800f450 con 0x7f8248100530 2024-05-10T04:07:57.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.946+0000 7f823effd700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82340a11a0 0x7f82340a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:57.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.947+0000 7f823effd700 1 -- 172.21.15.150:0/3492290157 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f82380ccd00 con 0x7f8248100530 2024-05-10T04:07:57.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.947+0000 7f824db26700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82340a11a0 0x7f82340a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:57.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.947+0000 7f824db26700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82340a11a0 0x7f82340a36b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8244000c00 tx=0x7f824401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:57.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:57.950+0000 7f823effd700 1 -- 172.21.15.150:0/3492290157 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8238096200 con 0x7f8248100530 2024-05-10T04:07:58.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.195+0000 7f824fd8a700 1 -- 172.21.15.150:0/3492290157 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8248068730 con 0x7f8248100530 2024-05-10T04:07:58.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.196+0000 7f823effd700 1 -- 172.21.15.150:0/3492290157 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10886 (secure 0 0 0) 0x7f8238099820 con 0x7f8248100530 2024-05-10T04:07:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:07:58.197 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:04:04.157819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:07:58.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.198+0000 7f824fd8a700 1 -- 172.21.15.150:0/3492290157 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82340a11a0 msgr2=0x7f82340a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:58.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.198+0000 7f824fd8a700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82340a11a0 0x7f82340a36b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8244000c00 tx=0x7f824401a040 comp rx=0 tx=0).stop 2024-05-10T04:07:58.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.198+0000 7f824fd8a700 1 -- 172.21.15.150:0/3492290157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8248100530 msgr2=0x7f82481984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:58.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.198+0000 7f824fd8a700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8248100530 0x7f82481984f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f823800c9e0 tx=0x7f823800cda0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.198+0000 7f824fd8a700 1 -- 172.21.15.150:0/3492290157 shutdown_connections 2024-05-10T04:07:58.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.198+0000 7f824fd8a700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f82340a11a0 0x7f82340a36b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.198+0000 7f824fd8a700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82481031a0 0x7f824819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.198+0000 7f824fd8a700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82480ffbe0 0x7f8248197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.198+0000 7f824fd8a700 1 --2- 172.21.15.150:0/3492290157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8248100530 0x7f82481984f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.198+0000 7f824fd8a700 1 -- 172.21.15.150:0/3492290157 >> 172.21.15.150:0/3492290157 conn(0x7f82480fb8b0 msgr2=0x7f8248106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:58.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.198+0000 7f824fd8a700 1 -- 172.21.15.150:0/3492290157 shutdown_connections 2024-05-10T04:07:58.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.198+0000 7f824fd8a700 1 -- 172.21.15.150:0/3492290157 wait complete. 2024-05-10T04:07:58.200 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 25 2024-05-10T04:07:58.212 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:07:58.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.344+0000 7f52f1b8a700 1 -- 172.21.15.150:0/2492077556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ec100ad0 msgr2=0x7f52ec100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:58.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.344+0000 7f52f1b8a700 1 --2- 172.21.15.150:0/2492077556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ec100ad0 0x7f52ec100ee0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009990 tx=0x7f52dc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.344+0000 7f52f1b8a700 1 -- 172.21.15.150:0/2492077556 shutdown_connections 2024-05-10T04:07:58.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.344+0000 7f52f1b8a700 1 --2- 172.21.15.150:0/2492077556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ec102590 0x7f52ec108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.344+0000 7f52f1b8a700 1 --2- 172.21.15.150:0/2492077556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ec100ad0 0x7f52ec100ee0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.344+0000 7f52f1b8a700 1 --2- 172.21.15.150:0/2492077556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52ec101bf0 0x7f52ec102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.344+0000 7f52f1b8a700 1 -- 172.21.15.150:0/2492077556 >> 172.21.15.150:0/2492077556 conn(0x7f52ec0fc0e0 msgr2=0x7f52ec0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:58.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.344+0000 7f52f1b8a700 1 -- 172.21.15.150:0/2492077556 shutdown_connections 2024-05-10T04:07:58.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.345+0000 7f52f1b8a700 1 -- 172.21.15.150:0/2492077556 wait complete. 2024-05-10T04:07:58.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.345+0000 7f52f1b8a700 1 Processor -- start 2024-05-10T04:07:58.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.345+0000 7f52f1b8a700 1 -- start start 2024-05-10T04:07:58.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.345+0000 7f52f1b8a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52ec100ad0 0x7f52ec073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:58.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.345+0000 7f52f1b8a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ec101bf0 0x7f52ec19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:58.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.345+0000 7f52f1b8a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ec102590 0x7f52ec19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:58.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.345+0000 7f52f1b8a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec19fac0 con 0x7f52ec101bf0 2024-05-10T04:07:58.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52f1b8a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec19fc30 con 0x7f52ec100ad0 2024-05-10T04:07:58.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52f1b8a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec19fdf0 con 0x7f52ec102590 2024-05-10T04:07:58.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52eaffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ec101bf0 0x7f52ec19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:58.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52eaffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ec101bf0 0x7f52ec19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59364/0 (socket says 172.21.15.150:59364) 2024-05-10T04:07:58.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52ec100ad0 0x7f52ec073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:58.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52eaffd700 1 -- 172.21.15.150:0/793656257 learned_addr learned my addr 172.21.15.150:0/793656257 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:58.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52ebfff700 1 --2- 172.21.15.150:0/793656257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ec102590 0x7f52ec19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:58.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52eaffd700 1 -- 172.21.15.150:0/793656257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ec102590 msgr2=0x7f52ec19f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:58.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52eaffd700 1 --2- 172.21.15.150:0/793656257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ec102590 0x7f52ec19f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52eaffd700 1 -- 172.21.15.150:0/793656257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52ec100ad0 msgr2=0x7f52ec073e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:58.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52eaffd700 1 --2- 172.21.15.150:0/793656257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52ec100ad0 0x7f52ec073e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52eaffd700 1 -- 172.21.15.150:0/793656257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc0095f0 con 0x7f52ec101bf0 2024-05-10T04:07:58.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52eaffd700 1 --2- 172.21.15.150:0/793656257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ec101bf0 0x7f52ec19cde0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f52e000d950 tx=0x7f52e000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:58.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52e8ff9700 1 -- 172.21.15.150:0/793656257 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e000f890 con 0x7f52ec101bf0 2024-05-10T04:07:58.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52e8ff9700 1 -- 172.21.15.150:0/793656257 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52e0010460 con 0x7f52ec101bf0 2024-05-10T04:07:58.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52f1b8a700 1 -- 172.21.15.150:0/793656257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec1a00d0 con 0x7f52ec101bf0 2024-05-10T04:07:58.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52e8ff9700 1 -- 172.21.15.150:0/793656257 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e000e610 con 0x7f52ec101bf0 2024-05-10T04:07:58.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.346+0000 7f52f1b8a700 1 -- 172.21.15.150:0/793656257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec1a05e0 con 0x7f52ec101bf0 2024-05-10T04:07:58.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.347+0000 7f52f1b8a700 1 -- 172.21.15.150:0/793656257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ec107ce0 con 0x7f52ec101bf0 2024-05-10T04:07:58.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.349+0000 7f52e8ff9700 1 -- 172.21.15.150:0/793656257 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f52e00108d0 con 0x7f52ec101bf0 2024-05-10T04:07:58.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.349+0000 7f52e8ff9700 1 --2- 172.21.15.150:0/793656257 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f52d40a1120 0x7f52d40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:58.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.350+0000 7f52eb7fe700 1 --2- 172.21.15.150:0/793656257 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f52d40a1120 0x7f52d40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:58.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.350+0000 7f52e8ff9700 1 -- 172.21.15.150:0/793656257 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f52e00cc9b0 con 0x7f52ec101bf0 2024-05-10T04:07:58.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.350+0000 7f52eb7fe700 1 --2- 172.21.15.150:0/793656257 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f52d40a1120 0x7f52d40a3630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f52dc006010 tx=0x7f52dc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:58.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.353+0000 7f52e8ff9700 1 -- 172.21.15.150:0/793656257 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f52e0095e30 con 0x7f52ec101bf0 2024-05-10T04:07:58.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:58 smithi150 ceph-mon[42477]: pgmap v391: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:58.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:58 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/344394070' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:07:58.463 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:58 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3492290157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:07:58.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.600+0000 7f52f1b8a700 1 -- 172.21.15.150:0/793656257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52ec068730 con 0x7f52ec101bf0 2024-05-10T04:07:58.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.601+0000 7f52e8ff9700 1 -- 172.21.15.150:0/793656257 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10886 (secure 0 0 0) 0x7f52e0016090 con 0x7f52ec101bf0 2024-05-10T04:07:58.602 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:07:58.602 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14592,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1387516089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1387516089},{"type":"v1","addr":"172.21.15.150:6835","nonce":1387516089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3462983502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3462983502},{"type":"v1","addr":"172.21.15.150:6837","nonce":3462983502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1619036473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1619036473},{"type":"v1","addr":"172.21.15.161:6837","nonce":1619036473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2082606207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2082606207},{"type":"v1","addr":"172.21.15.150:6839","nonce":2082606207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/591989720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":591989720},{"type":"v1","addr":"172.21.15.150:6841","nonce":591989720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3107886390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3107886390},{"type":"v1","addr":"172.21.15.161:6833","nonce":3107886390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:02:31.605102+0000","modified":"2024-05-10T04:04:04.157819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24556,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2987418700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2987418700},{"type":"v1","addr":"172.21.15.181:6833","nonce":2987418700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1403383813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1403383813},{"type":"v1","addr":"172.21.15.161:6835","nonce":1403383813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/501599730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":501599730},{"type":"v1","addr":"172.21.15.181:6835","nonce":501599730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6837/1798187480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1798187480},{"type":"v1","addr":"172.21.15.181:6837","nonce":1798187480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24556":{"gid":24556,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/1415332890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1415332890},{"type":"v1","addr":"172.21.15.181:6839","nonce":1415332890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/267491715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":267491715},{"type":"v1","addr":"172.21.15.161:6839","nonce":267491715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:07:58.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.603+0000 7f52f1b8a700 1 -- 172.21.15.150:0/793656257 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f52d40a1120 msgr2=0x7f52d40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:58.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.603+0000 7f52f1b8a700 1 --2- 172.21.15.150:0/793656257 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f52d40a1120 0x7f52d40a3630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f52dc006010 tx=0x7f52dc01a040 comp rx=0 tx=0).stop 2024-05-10T04:07:58.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.603+0000 7f52f1b8a700 1 -- 172.21.15.150:0/793656257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ec101bf0 msgr2=0x7f52ec19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:58.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.603+0000 7f52f1b8a700 1 --2- 172.21.15.150:0/793656257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ec101bf0 0x7f52ec19cde0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f52e000d950 tx=0x7f52e000dc60 comp rx=0 tx=0).stop 2024-05-10T04:07:58.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.603+0000 7f52f1b8a700 1 -- 172.21.15.150:0/793656257 shutdown_connections 2024-05-10T04:07:58.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.603+0000 7f52f1b8a700 1 --2- 172.21.15.150:0/793656257 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f52d40a1120 0x7f52d40a3630 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.603+0000 7f52f1b8a700 1 --2- 172.21.15.150:0/793656257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ec102590 0x7f52ec19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.603+0000 7f52f1b8a700 1 --2- 172.21.15.150:0/793656257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52ec100ad0 0x7f52ec073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.603+0000 7f52f1b8a700 1 --2- 172.21.15.150:0/793656257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ec101bf0 0x7f52ec19cde0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.603+0000 7f52f1b8a700 1 -- 172.21.15.150:0/793656257 >> 172.21.15.150:0/793656257 conn(0x7f52ec0fc0e0 msgr2=0x7f52ec104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:58.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.603+0000 7f52f1b8a700 1 -- 172.21.15.150:0/793656257 shutdown_connections 2024-05-10T04:07:58.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.603+0000 7f52f1b8a700 1 -- 172.21.15.150:0/793656257 wait complete. 2024-05-10T04:07:58.605 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 25 2024-05-10T04:07:58.617 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:07:58.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:58 smithi181 ceph-mon[46602]: pgmap v391: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:58.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:58 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/344394070' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:07:58.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:58 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3492290157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:07:58.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:58 smithi161 ceph-mon[46918]: pgmap v391: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-10T04:07:58.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:58 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/344394070' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:07:58.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:58 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3492290157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:07:58.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.750+0000 7f272bcd8700 1 -- 172.21.15.150:0/375453844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27241013e0 msgr2=0x7f27241017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:58.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.750+0000 7f272bcd8700 1 --2- 172.21.15.150:0/375453844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27241013e0 0x7f27241017f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2714009990 tx=0x7f2714009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.750+0000 7f272bcd8700 1 -- 172.21.15.150:0/375453844 shutdown_connections 2024-05-10T04:07:58.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.750+0000 7f272bcd8700 1 --2- 172.21.15.150:0/375453844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2724101d80 0x7f2724108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.750+0000 7f272bcd8700 1 --2- 172.21.15.150:0/375453844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27241002c0 0x7f27241006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.750+0000 7f272bcd8700 1 --2- 172.21.15.150:0/375453844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27241013e0 0x7f27241017f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.750+0000 7f272bcd8700 1 -- 172.21.15.150:0/375453844 >> 172.21.15.150:0/375453844 conn(0x7f27240fb8f0 msgr2=0x7f27240fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:58.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.750+0000 7f272bcd8700 1 -- 172.21.15.150:0/375453844 shutdown_connections 2024-05-10T04:07:58.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.750+0000 7f272bcd8700 1 -- 172.21.15.150:0/375453844 wait complete. 2024-05-10T04:07:58.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.750+0000 7f272bcd8700 1 Processor -- start 2024-05-10T04:07:58.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272bcd8700 1 -- start start 2024-05-10T04:07:58.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272bcd8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27241002c0 0x7f2724197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:58.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272bcd8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27241013e0 0x7f2724198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:58.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272bcd8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2724101d80 0x7f272419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:58.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272bcd8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272419fac0 con 0x7f2724101d80 2024-05-10T04:07:58.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272bcd8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272419fc30 con 0x7f27241013e0 2024-05-10T04:07:58.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272bcd8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272419fdf0 con 0x7f27241002c0 2024-05-10T04:07:58.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272a275700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2724101d80 0x7f272419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:58.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272a275700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2724101d80 0x7f272419f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59392/0 (socket says 172.21.15.150:59392) 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272a275700 1 -- 172.21.15.150:0/2167623659 learned_addr learned my addr 172.21.15.150:0/2167623659 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f2729a74700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27241002c0 0x7f2724197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f2729273700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27241013e0 0x7f2724198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272a275700 1 -- 172.21.15.150:0/2167623659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27241002c0 msgr2=0x7f2724197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272a275700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27241002c0 0x7f2724197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272a275700 1 -- 172.21.15.150:0/2167623659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27241013e0 msgr2=0x7f2724198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272a275700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27241013e0 0x7f2724198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272a275700 1 -- 172.21.15.150:0/2167623659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27140095f0 con 0x7f2724101d80 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.751+0000 7f272a275700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2724101d80 0x7f272419f4b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f271c007ee0 tx=0x7f271c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.752+0000 7f271affd700 1 -- 172.21.15.150:0/2167623659 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c017070 con 0x7f2724101d80 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.752+0000 7f272bcd8700 1 -- 172.21.15.150:0/2167623659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27241a00d0 con 0x7f2724101d80 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.752+0000 7f271affd700 1 -- 172.21.15.150:0/2167623659 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f271c00f040 con 0x7f2724101d80 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.752+0000 7f271affd700 1 -- 172.21.15.150:0/2167623659 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c005580 con 0x7f2724101d80 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.752+0000 7f272bcd8700 1 -- 172.21.15.150:0/2167623659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27241a05e0 con 0x7f2724101d80 2024-05-10T04:07:58.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.753+0000 7f272bcd8700 1 -- 172.21.15.150:0/2167623659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27240678b0 con 0x7f2724101d80 2024-05-10T04:07:58.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.754+0000 7f271affd700 1 -- 172.21.15.150:0/2167623659 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f271c00db20 con 0x7f2724101d80 2024-05-10T04:07:58.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.754+0000 7f271affd700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f27100a1120 0x7f27100a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:58.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.755+0000 7f2729a74700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f27100a1120 0x7f27100a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:58.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.755+0000 7f271affd700 1 -- 172.21.15.150:0/2167623659 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f271c0cc900 con 0x7f2724101d80 2024-05-10T04:07:58.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.755+0000 7f2729a74700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f27100a1120 0x7f27100a3630 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f27241012e0 tx=0x7f2720009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:58.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.758+0000 7f271affd700 1 -- 172.21.15.150:0/2167623659 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f271c095d80 con 0x7f2724101d80 2024-05-10T04:07:58.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.957+0000 7f272bcd8700 1 -- 172.21.15.150:0/2167623659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27241a08a0 con 0x7f2724101d80 2024-05-10T04:07:58.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.959+0000 7f271affd700 1 -- 172.21.15.150:0/2167623659 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24554 (secure 0 0 0) 0x7f271c0993a0 con 0x7f2724101d80 2024-05-10T04:07:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:07:58.960 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":87,"fsid":"325b4c82-0e81-11ef-bc98-c7b262605968","created":"2024-05-10T03:56:39.689538+0000","modified":"2024-05-10T04:03:30.185138+0000","last_up_change":"2024-05-10T04:01:56.685968+0000","last_in_change":"2024-05-10T04:01:39.850373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:59:19.332343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:02:30.076694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:02:30.760710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:02:33.850714+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e6088d05-bf1a-4fd0-be86-fde95b461d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6803","nonce":2214000215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6805","nonce":2214000215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6809","nonce":2214000215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2214000215},{"type":"v1","addr":"172.21.15.150:6807","nonce":2214000215}]},"public_addr":"172.21.15.150:6803/2214000215","cluster_addr":"172.21.15.150:6805/2214000215","heartbeat_back_addr":"172.21.15.150:6809/2214000215","heartbeat_front_addr":"172.21.15.150:6807/2214000215","state":["exists","up"]},{"osd":1,"uuid":"cef6c663-3fc6-4ea4-9dd8-e95845b8bac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6801","nonce":1606950735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6803","nonce":1606950735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6807","nonce":1606950735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1606950735},{"type":"v1","addr":"172.21.15.161:6805","nonce":1606950735}]},"public_addr":"172.21.15.161:6801/1606950735","cluster_addr":"172.21.15.161:6803/1606950735","heartbeat_back_addr":"172.21.15.161:6807/1606950735","heartbeat_front_addr":"172.21.15.161:6805/1606950735","state":["exists","up"]},{"osd":2,"uuid":"27256076-4b91-40ac-a69b-9e20e0f37b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6801","nonce":4035594145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6803","nonce":4035594145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6807","nonce":4035594145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4035594145},{"type":"v1","addr":"172.21.15.181:6805","nonce":4035594145}]},"public_addr":"172.21.15.181:6801/4035594145","cluster_addr":"172.21.15.181:6803/4035594145","heartbeat_back_addr":"172.21.15.181:6807/4035594145","heartbeat_front_addr":"172.21.15.181:6805/4035594145","state":["exists","up"]},{"osd":3,"uuid":"116fe668-1fb0-478b-a7ea-fd1542885ec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6811","nonce":1770832004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6813","nonce":1770832004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6817","nonce":1770832004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1770832004},{"type":"v1","addr":"172.21.15.150:6815","nonce":1770832004}]},"public_addr":"172.21.15.150:6811/1770832004","cluster_addr":"172.21.15.150:6813/1770832004","heartbeat_back_addr":"172.21.15.150:6817/1770832004","heartbeat_front_addr":"172.21.15.150:6815/1770832004","state":["exists","up"]},{"osd":4,"uuid":"827ce60d-b43f-45e0-a1e3-d40de3a1b5e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6809","nonce":765138503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6811","nonce":765138503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6815","nonce":765138503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":765138503},{"type":"v1","addr":"172.21.15.161:6813","nonce":765138503}]},"public_addr":"172.21.15.161:6809/765138503","cluster_addr":"172.21.15.161:6811/765138503","heartbeat_back_addr":"172.21.15.161:6815/765138503","heartbeat_front_addr":"172.21.15.161:6813/765138503","state":["exists","up"]},{"osd":5,"uuid":"bb8856d9-daa0-4119-9da6-84cc2c9c5dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6809","nonce":1962584780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6811","nonce":1962584780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6815","nonce":1962584780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1962584780},{"type":"v1","addr":"172.21.15.181:6813","nonce":1962584780}]},"public_addr":"172.21.15.181:6809/1962584780","cluster_addr":"172.21.15.181:6811/1962584780","heartbeat_back_addr":"172.21.15.181:6815/1962584780","heartbeat_front_addr":"172.21.15.181:6813/1962584780","state":["exists","up"]},{"osd":6,"uuid":"8e09e7ad-cc1f-4475-b63c-7f91efc3a2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6819","nonce":3579837504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6821","nonce":3579837504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6825","nonce":3579837504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3579837504},{"type":"v1","addr":"172.21.15.150:6823","nonce":3579837504}]},"public_addr":"172.21.15.150:6819/3579837504","cluster_addr":"172.21.15.150:6821/3579837504","heartbeat_back_addr":"172.21.15.150:6825/3579837504","heartbeat_front_addr":"172.21.15.150:6823/3579837504","state":["exists","up"]},{"osd":7,"uuid":"4e7b2d25-da3d-4d56-966a-13e6d9c2278e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6817","nonce":1528703649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6819","nonce":1528703649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6823","nonce":1528703649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1528703649},{"type":"v1","addr":"172.21.15.161:6821","nonce":1528703649}]},"public_addr":"172.21.15.161:6817/1528703649","cluster_addr":"172.21.15.161:6819/1528703649","heartbeat_back_addr":"172.21.15.161:6823/1528703649","heartbeat_front_addr":"172.21.15.161:6821/1528703649","state":["exists","up"]},{"osd":8,"uuid":"259306a4-2c3c-4aff-9f6b-38cf3256b090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6817","nonce":3605634813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6819","nonce":3605634813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6823","nonce":3605634813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3605634813},{"type":"v1","addr":"172.21.15.181:6821","nonce":3605634813}]},"public_addr":"172.21.15.181:6817/3605634813","cluster_addr":"172.21.15.181:6819/3605634813","heartbeat_back_addr":"172.21.15.181:6823/3605634813","heartbeat_front_addr":"172.21.15.181:6821/3605634813","state":["exists","up"]},{"osd":9,"uuid":"db540925-be5f-4601-8cb9-392f6b547d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6827","nonce":383677934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6829","nonce":383677934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6833","nonce":383677934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":383677934},{"type":"v1","addr":"172.21.15.150:6831","nonce":383677934}]},"public_addr":"172.21.15.150:6827/383677934","cluster_addr":"172.21.15.150:6829/383677934","heartbeat_back_addr":"172.21.15.150:6833/383677934","heartbeat_front_addr":"172.21.15.150:6831/383677934","state":["exists","up"]},{"osd":10,"uuid":"c3281319-74d6-4de1-ba77-6671f6d2d5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6825","nonce":1965594933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6827","nonce":1965594933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6831","nonce":1965594933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1965594933},{"type":"v1","addr":"172.21.15.161:6829","nonce":1965594933}]},"public_addr":"172.21.15.161:6825/1965594933","cluster_addr":"172.21.15.161:6827/1965594933","heartbeat_back_addr":"172.21.15.161:6831/1965594933","heartbeat_front_addr":"172.21.15.161:6829/1965594933","state":["exists","up"]},{"osd":11,"uuid":"a60f21a5-86c6-43b9-aea1-fa8517d04b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6825","nonce":1224976921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6827","nonce":1224976921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6831","nonce":1224976921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1224976921},{"type":"v1","addr":"172.21.15.181:6829","nonce":1224976921}]},"public_addr":"172.21.15.181:6825/1224976921","cluster_addr":"172.21.15.181:6827/1224976921","heartbeat_back_addr":"172.21.15.181:6831/1224976921","heartbeat_front_addr":"172.21.15.181:6829/1224976921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:40.369292+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:58:58.053152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:16.087829+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:33.517924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:59:51.308768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:08.902315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:26.306440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:00:43.774661+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:01.937380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:19.330294+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:37.097064+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:01:54.645083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/3457770606":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/977507934":"2024-05-11T03:57:08.747290+0000","172.21.15.150:6800/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/3909466387":"2024-05-11T03:57:29.013448+0000","172.21.15.150:0/2393499570":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6800/999775795":"2024-05-11T03:57:29.013448+0000","172.21.15.150:6801/3392737437":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/679926778":"2024-05-11T03:57:08.747290+0000","172.21.15.150:0/431676611":"2024-05-11T03:57:08.747290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:07:58.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.961+0000 7f272bcd8700 1 -- 172.21.15.150:0/2167623659 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f27100a1120 msgr2=0x7f27100a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:58.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.961+0000 7f272bcd8700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f27100a1120 0x7f27100a3630 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f27241012e0 tx=0x7f2720009260 comp rx=0 tx=0).stop 2024-05-10T04:07:58.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.961+0000 7f272bcd8700 1 -- 172.21.15.150:0/2167623659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2724101d80 msgr2=0x7f272419f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:58.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.961+0000 7f272bcd8700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2724101d80 0x7f272419f4b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f271c007ee0 tx=0x7f271c00d3b0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.961+0000 7f272bcd8700 1 -- 172.21.15.150:0/2167623659 shutdown_connections 2024-05-10T04:07:58.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.961+0000 7f272bcd8700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f27100a1120 0x7f27100a3630 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.961+0000 7f272bcd8700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2724101d80 0x7f272419f4b0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.961+0000 7f272bcd8700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27241002c0 0x7f2724197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.961+0000 7f272bcd8700 1 --2- 172.21.15.150:0/2167623659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27241013e0 0x7f2724198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:58.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.961+0000 7f272bcd8700 1 -- 172.21.15.150:0/2167623659 >> 172.21.15.150:0/2167623659 conn(0x7f27240fb8f0 msgr2=0x7f27240fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:58.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.961+0000 7f272bcd8700 1 -- 172.21.15.150:0/2167623659 shutdown_connections 2024-05-10T04:07:58.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:58.961+0000 7f272bcd8700 1 -- 172.21.15.150:0/2167623659 wait complete. 2024-05-10T04:07:58.976 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T04:07:59.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.108+0000 7f24351e1700 1 -- 172.21.15.150:0/1007083244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2430101bf0 msgr2=0x7f2430102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:59.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.108+0000 7f24351e1700 1 --2- 172.21.15.150:0/1007083244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2430101bf0 0x7f2430102000 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2420009990 tx=0x7f2420009ca0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.108+0000 7f24351e1700 1 -- 172.21.15.150:0/1007083244 shutdown_connections 2024-05-10T04:07:59.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.108+0000 7f24351e1700 1 --2- 172.21.15.150:0/1007083244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2430102590 0x7f2430108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.108+0000 7f24351e1700 1 --2- 172.21.15.150:0/1007083244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2430100ad0 0x7f2430100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.108+0000 7f24351e1700 1 --2- 172.21.15.150:0/1007083244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2430101bf0 0x7f2430102000 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.108+0000 7f24351e1700 1 -- 172.21.15.150:0/1007083244 >> 172.21.15.150:0/1007083244 conn(0x7f24300fc0c0 msgr2=0x7f24300fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:59.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.108+0000 7f24351e1700 1 -- 172.21.15.150:0/1007083244 shutdown_connections 2024-05-10T04:07:59.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.108+0000 7f24351e1700 1 -- 172.21.15.150:0/1007083244 wait complete. 2024-05-10T04:07:59.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.109+0000 7f24351e1700 1 Processor -- start 2024-05-10T04:07:59.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.109+0000 7f24351e1700 1 -- start start 2024-05-10T04:07:59.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.109+0000 7f24351e1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2430100ad0 0x7f2430196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:59.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.109+0000 7f24351e1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2430101bf0 0x7f2430197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:59.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.109+0000 7f24351e1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2430102590 0x7f243019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:59.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.109+0000 7f24351e1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243019e8b0 con 0x7f2430100ad0 2024-05-10T04:07:59.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.109+0000 7f24351e1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243019ea20 con 0x7f2430102590 2024-05-10T04:07:59.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.109+0000 7f24351e1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243019ebe0 con 0x7f2430101bf0 2024-05-10T04:07:59.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.109+0000 7f242ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2430100ad0 0x7f2430196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:59.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.109+0000 7f242ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2430100ad0 0x7f2430196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59406/0 (socket says 172.21.15.150:59406) 2024-05-10T04:07:59.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.109+0000 7f242ed9d700 1 -- 172.21.15.150:0/3277505286 learned_addr learned my addr 172.21.15.150:0/3277505286 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:59.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.109+0000 7f242f59e700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2430102590 0x7f243019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:59.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.109+0000 7f242e59c700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2430101bf0 0x7f2430197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:59.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.110+0000 7f242ed9d700 1 -- 172.21.15.150:0/3277505286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2430101bf0 msgr2=0x7f2430197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:59.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.110+0000 7f242ed9d700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2430101bf0 0x7f2430197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.110+0000 7f242ed9d700 1 -- 172.21.15.150:0/3277505286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2430102590 msgr2=0x7f243019e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:59.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.110+0000 7f242ed9d700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2430102590 0x7f243019e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.110+0000 7f242ed9d700 1 -- 172.21.15.150:0/3277505286 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24200095f0 con 0x7f2430100ad0 2024-05-10T04:07:59.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.110+0000 7f242ed9d700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2430100ad0 0x7f2430196de0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f242400d950 tx=0x7f242400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:59.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.110+0000 7f2417fff700 1 -- 172.21.15.150:0/3277505286 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2424004220 con 0x7f2430100ad0 2024-05-10T04:07:59.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.110+0000 7f2417fff700 1 -- 172.21.15.150:0/3277505286 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2424004380 con 0x7f2430100ad0 2024-05-10T04:07:59.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.110+0000 7f24351e1700 1 -- 172.21.15.150:0/3277505286 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243019eec0 con 0x7f2430100ad0 2024-05-10T04:07:59.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.110+0000 7f2417fff700 1 -- 172.21.15.150:0/3277505286 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2424003bf0 con 0x7f2430100ad0 2024-05-10T04:07:59.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.110+0000 7f24351e1700 1 -- 172.21.15.150:0/3277505286 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243019f2f0 con 0x7f2430100ad0 2024-05-10T04:07:59.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.111+0000 7f24351e1700 1 -- 172.21.15.150:0/3277505286 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2430107ce0 con 0x7f2430100ad0 2024-05-10T04:07:59.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.112+0000 7f2417fff700 1 -- 172.21.15.150:0/3277505286 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2424003d50 con 0x7f2430100ad0 2024-05-10T04:07:59.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.113+0000 7f2417fff700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f24180a11c0 0x7f24180a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:59.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.113+0000 7f2417fff700 1 -- 172.21.15.150:0/3277505286 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f24240cce80 con 0x7f2430100ad0 2024-05-10T04:07:59.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.113+0000 7f242e59c700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f24180a11c0 0x7f24180a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:59.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.113+0000 7f242e59c700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f24180a11c0 0x7f24180a36d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f24200038c0 tx=0x7f242001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:59.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.116+0000 7f2417fff700 1 -- 172.21.15.150:0/3277505286 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2424096300 con 0x7f2430100ad0 2024-05-10T04:07:59.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.364+0000 7f24351e1700 1 -- 172.21.15.150:0/3277505286 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f243010c520 con 0x7f2430100ad0 2024-05-10T04:07:59.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.378+0000 7f2417fff700 1 -- 172.21.15.150:0/3277505286 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2424017020 con 0x7f2430100ad0 2024-05-10T04:07:59.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.381+0000 7f24351e1700 1 -- 172.21.15.150:0/3277505286 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f24180a11c0 msgr2=0x7f24180a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:59.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.381+0000 7f24351e1700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f24180a11c0 0x7f24180a36d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f24200038c0 tx=0x7f242001a040 comp rx=0 tx=0).stop 2024-05-10T04:07:59.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.381+0000 7f24351e1700 1 -- 172.21.15.150:0/3277505286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2430100ad0 msgr2=0x7f2430196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:59.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.381+0000 7f24351e1700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2430100ad0 0x7f2430196de0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f242400d950 tx=0x7f242400dc60 comp rx=0 tx=0).stop 2024-05-10T04:07:59.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.381+0000 7f24351e1700 1 -- 172.21.15.150:0/3277505286 shutdown_connections 2024-05-10T04:07:59.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.381+0000 7f24351e1700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f24180a11c0 0x7f24180a36d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.381+0000 7f24351e1700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2430102590 0x7f243019e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.381+0000 7f24351e1700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2430100ad0 0x7f2430196de0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.381+0000 7f24351e1700 1 --2- 172.21.15.150:0/3277505286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2430101bf0 0x7f2430197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.381+0000 7f24351e1700 1 -- 172.21.15.150:0/3277505286 >> 172.21.15.150:0/3277505286 conn(0x7f24300fc0c0 msgr2=0x7f2430104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:59.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.381+0000 7f24351e1700 1 -- 172.21.15.150:0/3277505286 shutdown_connections 2024-05-10T04:07:59.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.381+0000 7f24351e1700 1 -- 172.21.15.150:0/3277505286 wait complete. 2024-05-10T04:07:59.384 INFO:teuthology.orchestra.run.smithi150.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T04:07:59.396 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T04:07:59.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.534+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/2845505813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4107a20 msgr2=0x7fe9f4109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:59.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.534+0000 7fe9fbfb9700 1 --2- 172.21.15.150:0/2845505813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4107a20 0x7fe9f4109e40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00b0a0 tx=0x7fe9ec00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.534+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/2845505813 shutdown_connections 2024-05-10T04:07:59.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.534+0000 7fe9fbfb9700 1 --2- 172.21.15.150:0/2845505813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4107a20 0x7fe9f4109e40 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.534+0000 7fe9fbfb9700 1 --2- 172.21.15.150:0/2845505813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9f4102760 0x7fe9f4104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.534+0000 7fe9fbfb9700 1 --2- 172.21.15.150:0/2845505813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f41050c0 0x7fe9f41074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.534+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/2845505813 >> 172.21.15.150:0/2845505813 conn(0x7fe9f40fc180 msgr2=0x7fe9f40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.534+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/2845505813 shutdown_connections 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.534+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/2845505813 wait complete. 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.534+0000 7fe9fbfb9700 1 Processor -- start 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.534+0000 7fe9fbfb9700 1 -- start start 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fbfb9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9f4102760 0x7fe9f4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fbfb9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f41050c0 0x7fe9f4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fbfb9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4107a20 0x7fe9f419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fbfb9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f419fae0 con 0x7fe9f4107a20 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fbfb9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f419fc50 con 0x7fe9f4102760 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fbfb9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f419fe10 con 0x7fe9f41050c0 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fa556700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4107a20 0x7fe9f419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9f9d55700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9f4102760 0x7fe9f4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fa556700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4107a20 0x7fe9f419f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41540/0 (socket says 172.21.15.150:41540) 2024-05-10T04:07:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fa556700 1 -- 172.21.15.150:0/650489845 learned_addr learned my addr 172.21.15.150:0/650489845 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:07:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9f9554700 1 --2- 172.21.15.150:0/650489845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f41050c0 0x7fe9f4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fa556700 1 -- 172.21.15.150:0/650489845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f41050c0 msgr2=0x7fe9f4198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fa556700 1 --2- 172.21.15.150:0/650489845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f41050c0 0x7fe9f4198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fa556700 1 -- 172.21.15.150:0/650489845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9f4102760 msgr2=0x7fe9f4198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:07:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fa556700 1 --2- 172.21.15.150:0/650489845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9f4102760 0x7fe9f4198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:07:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fa556700 1 -- 172.21.15.150:0/650489845 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec009d00 con 0x7fe9f4107a20 2024-05-10T04:07:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fa556700 1 --2- 172.21.15.150:0/650489845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4107a20 0x7fe9f419f4d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec006130 tx=0x7fe9ec007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9eaffd700 1 -- 172.21.15.150:0/650489845 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00e040 con 0x7fe9f4107a20 2024-05-10T04:07:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9eaffd700 1 -- 172.21.15.150:0/650489845 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9ec00be40 con 0x7fe9f4107a20 2024-05-10T04:07:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9eaffd700 1 -- 172.21.15.150:0/650489845 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec0052a0 con 0x7fe9f4107a20 2024-05-10T04:07:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.535+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/650489845 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f41a0090 con 0x7fe9f4107a20 2024-05-10T04:07:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.536+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/650489845 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f41a04c0 con 0x7fe9f4107a20 2024-05-10T04:07:59.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.536+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/650489845 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f40fdd30 con 0x7fe9f4107a20 2024-05-10T04:07:59.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.538+0000 7fe9eaffd700 1 -- 172.21.15.150:0/650489845 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe9ec01d470 con 0x7fe9f4107a20 2024-05-10T04:07:59.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.539+0000 7fe9eaffd700 1 --2- 172.21.15.150:0/650489845 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9e00a11c0 0x7fe9e00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:07:59.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.539+0000 7fe9eaffd700 1 -- 172.21.15.150:0/650489845 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fe9ec0ce510 con 0x7fe9f4107a20 2024-05-10T04:07:59.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.539+0000 7fe9f9d55700 1 --2- 172.21.15.150:0/650489845 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9e00a11c0 0x7fe9e00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:07:59.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.540+0000 7fe9f9d55700 1 --2- 172.21.15.150:0/650489845 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9e00a11c0 0x7fe9e00a36d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0005950 tx=0x7fe9f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:07:59.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.543+0000 7fe9eaffd700 1 -- 172.21.15.150:0/650489845 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe9ec097920 con 0x7fe9f4107a20 2024-05-10T04:07:59.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:59 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/793656257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:07:59.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:59 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/2167623659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:07:59.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:07:59 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mon-a[42454]: 2024-05-10T04:07:59.365+0000 7f92b11e3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T04:07:59.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:59 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/793656257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:07:59.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:07:59 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/2167623659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:07:59.738 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:07:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-c[84023]: ignoring --setuser ceph since I am not root 2024-05-10T04:07:59.738 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:07:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-c[84023]: ignoring --setgroup ceph since I am not root 2024-05-10T04:07:59.738 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:07:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-c[84023]: starting mds.c at 2024-05-10T04:07:59.738 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:07:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-f[83681]: ignoring --setuser ceph since I am not root 2024-05-10T04:07:59.738 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:07:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-f[83681]: ignoring --setgroup ceph since I am not root 2024-05-10T04:07:59.738 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:07:59 smithi181 ceph-mds[83703]: -- 172.21.15.181:0/3879546809 <== mon.0 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5652c20ea1a0 con 0x5652c214a400 2024-05-10T04:07:59.738 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:07:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-f[83681]: starting mds.f at 2024-05-10T04:07:59.739 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:07:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-i[84368]: ignoring --setuser ceph since I am not root 2024-05-10T04:07:59.739 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:07:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-i[84368]: ignoring --setgroup ceph since I am not root 2024-05-10T04:07:59.739 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:07:59 smithi181 ceph-mds[84390]: -- 172.21.15.181:0/2943473708 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55873a2561a0 con 0x55873a2b6d00 2024-05-10T04:07:59.739 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:07:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-i[84368]: starting mds.i at 2024-05-10T04:07:59.739 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:07:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-l[83340]: ignoring --setuser ceph since I am not root 2024-05-10T04:07:59.739 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:07:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-l[83340]: ignoring --setgroup ceph since I am not root 2024-05-10T04:07:59.739 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:07:59 smithi181 ceph-mds[83363]: -- 172.21.15.181:0/4267401909 <== mon.0 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56142fd381a0 con 0x56142fd98400 2024-05-10T04:07:59.739 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:07:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-l[83340]: starting mds.l at 2024-05-10T04:07:59.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:59 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/793656257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:07:59.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:07:59 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/2167623659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:07:59.741 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:07:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-e[82824]: ignoring --setuser ceph since I am not root 2024-05-10T04:07:59.742 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:07:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-e[82824]: ignoring --setgroup ceph since I am not root 2024-05-10T04:07:59.742 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:07:59 smithi161 ceph-mds[82847]: -- 172.21.15.161:0/189248987 <== mon.0 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560531a561a0 con 0x560531ab6400 2024-05-10T04:07:59.742 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:07:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-e[82824]: starting mds.e at 2024-05-10T04:07:59.742 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:07:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-b[83518]: ignoring --setuser ceph since I am not root 2024-05-10T04:07:59.742 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:07:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-b[83518]: ignoring --setgroup ceph since I am not root 2024-05-10T04:07:59.742 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:07:59 smithi161 ceph-mds[83541]: -- 172.21.15.161:0/4097703408 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5558765061a0 con 0x555876566400 2024-05-10T04:07:59.742 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:07:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-b[83518]: starting mds.b at 2024-05-10T04:07:59.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:07:59.792+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/650489845 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe9f410d400 con 0x7fe9f4107a20 2024-05-10T04:08:00.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.381+0000 7fe9eaffd700 1 -- 172.21.15.150:0/650489845 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe9ec004c50 con 0x7fe9f4107a20 2024-05-10T04:08:00.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.385+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/650489845 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9e00a11c0 msgr2=0x7fe9e00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:00.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.385+0000 7fe9fbfb9700 1 --2- 172.21.15.150:0/650489845 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9e00a11c0 0x7fe9e00a36d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0005950 tx=0x7fe9f000b410 comp rx=0 tx=0).stop 2024-05-10T04:08:00.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.385+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/650489845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4107a20 msgr2=0x7fe9f419f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:00.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.385+0000 7fe9fbfb9700 1 --2- 172.21.15.150:0/650489845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4107a20 0x7fe9f419f4d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec006130 tx=0x7fe9ec007920 comp rx=0 tx=0).stop 2024-05-10T04:08:00.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.385+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/650489845 shutdown_connections 2024-05-10T04:08:00.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.385+0000 7fe9fbfb9700 1 --2- 172.21.15.150:0/650489845 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7fe9e00a11c0 0x7fe9e00a36d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:00.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.385+0000 7fe9fbfb9700 1 --2- 172.21.15.150:0/650489845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4107a20 0x7fe9f419f4d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:00.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.385+0000 7fe9fbfb9700 1 --2- 172.21.15.150:0/650489845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9f4102760 0x7fe9f4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:00.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.385+0000 7fe9fbfb9700 1 --2- 172.21.15.150:0/650489845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f41050c0 0x7fe9f4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:00.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.385+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/650489845 >> 172.21.15.150:0/650489845 conn(0x7fe9f40fc180 msgr2=0x7fe9f41035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:08:00.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.385+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/650489845 shutdown_connections 2024-05-10T04:08:00.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.385+0000 7fe9fbfb9700 1 -- 172.21.15.150:0/650489845 wait complete. 2024-05-10T04:08:00.403 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T04:08:00.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.545+0000 7f2ee7b15700 1 -- 172.21.15.150:0/544573927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee00ffc20 msgr2=0x7f2ee0100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:00.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.545+0000 7f2ee7b15700 1 --2- 172.21.15.150:0/544573927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee00ffc20 0x7f2ee0100030 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009990 tx=0x7f2edc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:08:00.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.545+0000 7f2ee7b15700 1 -- 172.21.15.150:0/544573927 shutdown_connections 2024-05-10T04:08:00.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.545+0000 7f2ee7b15700 1 --2- 172.21.15.150:0/544573927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ee0100570 0x7f2ee010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:00.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.545+0000 7f2ee7b15700 1 --2- 172.21.15.150:0/544573927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ee01031e0 0x7f2ee01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:00.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.545+0000 7f2ee7b15700 1 --2- 172.21.15.150:0/544573927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee00ffc20 0x7f2ee0100030 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:00.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.545+0000 7f2ee7b15700 1 -- 172.21.15.150:0/544573927 >> 172.21.15.150:0/544573927 conn(0x7f2ee00fb8f0 msgr2=0x7f2ee00fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:08:00.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.545+0000 7f2ee7b15700 1 -- 172.21.15.150:0/544573927 shutdown_connections 2024-05-10T04:08:00.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.545+0000 7f2ee7b15700 1 -- 172.21.15.150:0/544573927 wait complete. 2024-05-10T04:08:00.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee7b15700 1 Processor -- start 2024-05-10T04:08:00.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee7b15700 1 -- start start 2024-05-10T04:08:00.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee7b15700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ee00ffc20 0x7f2ee0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:08:00.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee7b15700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee0100570 0x7f2ee0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:08:00.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee7b15700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ee01031e0 0x7f2ee019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:08:00.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee7b15700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee019f9a0 con 0x7f2ee0100570 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee7b15700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee019fb10 con 0x7f2ee00ffc20 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee7b15700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee019fcd0 con 0x7f2ee01031e0 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee50b0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee0100570 0x7f2ee0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee58b1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ee00ffc20 0x7f2ee0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee50b0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee0100570 0x7f2ee0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41558/0 (socket says 172.21.15.150:41558) 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee50b0700 1 -- 172.21.15.150:0/1748576072 learned_addr learned my addr 172.21.15.150:0/1748576072 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee60b2700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ee01031e0 0x7f2ee019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee50b0700 1 -- 172.21.15.150:0/1748576072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ee01031e0 msgr2=0x7f2ee019f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee50b0700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ee01031e0 0x7f2ee019f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee50b0700 1 -- 172.21.15.150:0/1748576072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ee00ffc20 msgr2=0x7f2ee0197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee50b0700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ee00ffc20 0x7f2ee0197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee50b0700 1 -- 172.21.15.150:0/1748576072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc0095f0 con 0x7f2ee0100570 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.546+0000 7f2ee58b1700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ee00ffc20 0x7f2ee0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:08:00.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.547+0000 7f2ee50b0700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee0100570 0x7f2ee0198400 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0098e0 tx=0x7f2edc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:08:00.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.547+0000 7f2ed6ffd700 1 -- 172.21.15.150:0/1748576072 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc01d070 con 0x7f2ee0100570 2024-05-10T04:08:00.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.547+0000 7f2ed6ffd700 1 -- 172.21.15.150:0/1748576072 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2edc021410 con 0x7f2ee0100570 2024-05-10T04:08:00.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.547+0000 7f2ed6ffd700 1 -- 172.21.15.150:0/1748576072 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc017650 con 0x7f2ee0100570 2024-05-10T04:08:00.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.547+0000 7f2ee7b15700 1 -- 172.21.15.150:0/1748576072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee019ff50 con 0x7f2ee0100570 2024-05-10T04:08:00.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.547+0000 7f2ee7b15700 1 -- 172.21.15.150:0/1748576072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee01a02c0 con 0x7f2ee0100570 2024-05-10T04:08:00.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.548+0000 7f2ee7b15700 1 -- 172.21.15.150:0/1748576072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee0109030 con 0x7f2ee0100570 2024-05-10T04:08:00.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.549+0000 7f2ed6ffd700 1 -- 172.21.15.150:0/1748576072 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2edc0217f0 con 0x7f2ee0100570 2024-05-10T04:08:00.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.550+0000 7f2ed6ffd700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2ecc0a11c0 0x7f2ecc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:08:00.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.550+0000 7f2ee58b1700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2ecc0a11c0 0x7f2ecc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:08:00.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.550+0000 7f2ed6ffd700 1 -- 172.21.15.150:0/1748576072 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f2edc0ce800 con 0x7f2ee0100570 2024-05-10T04:08:00.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.550+0000 7f2ee58b1700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2ecc0a11c0 0x7f2ecc0a36d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0068120 tx=0x7f2ed0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:08:00.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.553+0000 7f2ed6ffd700 1 -- 172.21.15.150:0/1748576072 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2edc097bb0 con 0x7f2ee0100570 2024-05-10T04:08:00.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:00 smithi150 ceph-mon[42477]: pgmap v392: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T04:08:00.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:00 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3277505286' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T04:08:00.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:00 smithi150 ceph-mon[42477]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T04:08:00.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:00 smithi150 ceph-mon[42477]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T04:08:00.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:00 smithi150 ceph-mon[42477]: osdmap e88: 12 total, 12 up, 12 in 2024-05-10T04:08:00.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:00 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/3277505286' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T04:08:00.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:00 smithi150 ceph-mon[42477]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-10T04:08:00.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:00 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/650489845' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T04:08:00.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:00 smithi181 ceph-mon[46602]: pgmap v392: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T04:08:00.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:00 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3277505286' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T04:08:00.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:00 smithi181 ceph-mon[46602]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T04:08:00.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:00 smithi181 ceph-mon[46602]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T04:08:00.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:00 smithi181 ceph-mon[46602]: osdmap e88: 12 total, 12 up, 12 in 2024-05-10T04:08:00.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:00 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/3277505286' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T04:08:00.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:00 smithi181 ceph-mon[46602]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-10T04:08:00.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:00 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/650489845' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T04:08:00.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:00 smithi161 ceph-mon[46918]: pgmap v392: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T04:08:00.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:00 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3277505286' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T04:08:00.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:00 smithi161 ceph-mon[46918]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T04:08:00.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:00 smithi161 ceph-mon[46918]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T04:08:00.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:00 smithi161 ceph-mon[46918]: osdmap e88: 12 total, 12 up, 12 in 2024-05-10T04:08:00.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:00 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/3277505286' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T04:08:00.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:00 smithi161 ceph-mon[46918]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-10T04:08:00.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:00 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/650489845' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T04:08:00.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:00.752+0000 7f2ee7b15700 1 -- 172.21.15.150:0/1748576072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2ee00678b0 con 0x7f2ee0100570 2024-05-10T04:08:01.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.386+0000 7f2ed6ffd700 1 -- 172.21.15.150:0/1748576072 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2edc09b1d0 con 0x7f2ee0100570 2024-05-10T04:08:01.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.391+0000 7f2ee7b15700 1 -- 172.21.15.150:0/1748576072 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2ecc0a11c0 msgr2=0x7f2ecc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:01.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.391+0000 7f2ee7b15700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2ecc0a11c0 0x7f2ecc0a36d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0068120 tx=0x7f2ed0009260 comp rx=0 tx=0).stop 2024-05-10T04:08:01.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.391+0000 7f2ee7b15700 1 -- 172.21.15.150:0/1748576072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee0100570 msgr2=0x7f2ee0198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:01.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.391+0000 7f2ee7b15700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee0100570 0x7f2ee0198400 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0098e0 tx=0x7f2edc004730 comp rx=0 tx=0).stop 2024-05-10T04:08:01.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.391+0000 7f2ee7b15700 1 -- 172.21.15.150:0/1748576072 shutdown_connections 2024-05-10T04:08:01.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.391+0000 7f2ee7b15700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f2ecc0a11c0 0x7f2ecc0a36d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:01.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.391+0000 7f2ee7b15700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ee01031e0 0x7f2ee019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:01.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.391+0000 7f2ee7b15700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ee00ffc20 0x7f2ee0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:01.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.391+0000 7f2ee7b15700 1 --2- 172.21.15.150:0/1748576072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee0100570 0x7f2ee0198400 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:01.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.391+0000 7f2ee7b15700 1 -- 172.21.15.150:0/1748576072 >> 172.21.15.150:0/1748576072 conn(0x7f2ee00fb8f0 msgr2=0x7f2ee0106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:08:01.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.391+0000 7f2ee7b15700 1 -- 172.21.15.150:0/1748576072 shutdown_connections 2024-05-10T04:08:01.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.392+0000 7f2ee7b15700 1 -- 172.21.15.150:0/1748576072 wait complete. 2024-05-10T04:08:01.395 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_metadata' removed 2024-05-10T04:08:01.409 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T04:08:01.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.552+0000 7faed809c700 1 -- 172.21.15.150:0/1968056710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed0101be0 msgr2=0x7faed0101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:01.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.552+0000 7faed809c700 1 --2- 172.21.15.150:0/1968056710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed0101be0 0x7faed0101ff0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7faec4009990 tx=0x7faec4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:08:01.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.552+0000 7faed809c700 1 -- 172.21.15.150:0/1968056710 shutdown_connections 2024-05-10T04:08:01.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.552+0000 7faed809c700 1 --2- 172.21.15.150:0/1968056710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed0102580 0x7faed0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:01.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.552+0000 7faed809c700 1 --2- 172.21.15.150:0/1968056710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed0100ac0 0x7faed0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:01.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.552+0000 7faed809c700 1 --2- 172.21.15.150:0/1968056710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed0101be0 0x7faed0101ff0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:01.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.552+0000 7faed809c700 1 -- 172.21.15.150:0/1968056710 >> 172.21.15.150:0/1968056710 conn(0x7faed00fc0d0 msgr2=0x7faed00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:08:01.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.552+0000 7faed809c700 1 -- 172.21.15.150:0/1968056710 shutdown_connections 2024-05-10T04:08:01.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.552+0000 7faed809c700 1 -- 172.21.15.150:0/1968056710 wait complete. 2024-05-10T04:08:01.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.553+0000 7faed809c700 1 Processor -- start 2024-05-10T04:08:01.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.553+0000 7faed809c700 1 -- start start 2024-05-10T04:08:01.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.553+0000 7faed809c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed0100ac0 0x7faed0068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:08:01.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.553+0000 7faed809c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed0101be0 0x7faed019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:08:01.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.553+0000 7faed809c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed0102580 0x7faed019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:08:01.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.553+0000 7faed809c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed019f960 con 0x7faed0102580 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.553+0000 7faed809c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed019fad0 con 0x7faed0100ac0 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.553+0000 7faed809c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed019fc90 con 0x7faed0101be0 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.553+0000 7faed6639700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed0102580 0x7faed019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.553+0000 7faed6639700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed0102580 0x7faed019f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41566/0 (socket says 172.21.15.150:41566) 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.553+0000 7faed6639700 1 -- 172.21.15.150:0/313507984 learned_addr learned my addr 172.21.15.150:0/313507984 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.553+0000 7faed5e38700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed0100ac0 0x7faed0068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.553+0000 7faed5637700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed0101be0 0x7faed019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.554+0000 7faed6639700 1 -- 172.21.15.150:0/313507984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed0101be0 msgr2=0x7faed019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.554+0000 7faed6639700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed0101be0 0x7faed019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.554+0000 7faed6639700 1 -- 172.21.15.150:0/313507984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed0100ac0 msgr2=0x7faed0068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.554+0000 7faed6639700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed0100ac0 0x7faed0068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.554+0000 7faed6639700 1 -- 172.21.15.150:0/313507984 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec40095f0 con 0x7faed0102580 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.554+0000 7faed5e38700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed0100ac0 0x7faed0068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.554+0000 7faed6639700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed0102580 0x7faed019f260 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7faec800d980 tx=0x7faec800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:08:01.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.554+0000 7faeceffd700 1 -- 172.21.15.150:0/313507984 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8009940 con 0x7faed0102580 2024-05-10T04:08:01.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.554+0000 7faeceffd700 1 -- 172.21.15.150:0/313507984 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faec8004830 con 0x7faed0102580 2024-05-10T04:08:01.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.554+0000 7faed809c700 1 -- 172.21.15.150:0/313507984 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed019ff70 con 0x7faed0102580 2024-05-10T04:08:01.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.554+0000 7faeceffd700 1 -- 172.21.15.150:0/313507984 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec80054e0 con 0x7faed0102580 2024-05-10T04:08:01.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.554+0000 7faed809c700 1 -- 172.21.15.150:0/313507984 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed01a03f0 con 0x7faed0102580 2024-05-10T04:08:01.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.555+0000 7faed809c700 1 -- 172.21.15.150:0/313507984 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed0107cd0 con 0x7faed0102580 2024-05-10T04:08:01.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.557+0000 7faeceffd700 1 -- 172.21.15.150:0/313507984 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7faec8004ca0 con 0x7faed0102580 2024-05-10T04:08:01.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.557+0000 7faeceffd700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faeb80a1240 0x7faeb80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:08:01.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.557+0000 7faeceffd700 1 -- 172.21.15.150:0/313507984 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8753+0+0 (secure 0 0 0) 0x7faec80cccd0 con 0x7faed0102580 2024-05-10T04:08:01.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.557+0000 7faed5e38700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faeb80a1240 0x7faeb80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:08:01.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.558+0000 7faed5e38700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faeb80a1240 0x7faeb80a3750 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faed0101ae0 tx=0x7faec000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:08:01.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.561+0000 7faeceffd700 1 -- 172.21.15.150:0/313507984 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faec80962d0 con 0x7faed0102580 2024-05-10T04:08:01.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T04:08:01.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T04:08:01.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: Cluster is now healthy 2024-05-10T04:08:01.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.181:6837/3259652915,v1:172.21.15.181:6839/3259652915] up:boot 2024-05-10T04:08:01.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.181:6834/2531815465,v1:172.21.15.181:6838/2531815465] up:boot 2024-05-10T04:08:01.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.181:6833/2354517413,v1:172.21.15.181:6836/2354517413] up:boot 2024-05-10T04:08:01.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.181:6832/3089553128,v1:172.21.15.181:6835/3089553128] up:boot 2024-05-10T04:08:01.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.161:6834/1180830718,v1:172.21.15.161:6838/1180830718] up:boot 2024-05-10T04:08:01.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: mds.? [v2:172.21.15.161:6835/2092399552,v1:172.21.15.161:6839/2092399552] up:boot 2024-05-10T04:08:01.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/650489845' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T04:08:01.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: fsmap 12 up:standby 2024-05-10T04:08:01.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T04:08:01.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T04:08:01.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T04:08:01.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T04:08:01.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T04:08:01.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T04:08:01.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1748576072' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:08:01.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:01 smithi150 ceph-mon[42477]: pgmap v394: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-10T04:08:01.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T04:08:01.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T04:08:01.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: Cluster is now healthy 2024-05-10T04:08:01.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.181:6837/3259652915,v1:172.21.15.181:6839/3259652915] up:boot 2024-05-10T04:08:01.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.181:6834/2531815465,v1:172.21.15.181:6838/2531815465] up:boot 2024-05-10T04:08:01.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.181:6833/2354517413,v1:172.21.15.181:6836/2354517413] up:boot 2024-05-10T04:08:01.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.181:6832/3089553128,v1:172.21.15.181:6835/3089553128] up:boot 2024-05-10T04:08:01.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.161:6834/1180830718,v1:172.21.15.161:6838/1180830718] up:boot 2024-05-10T04:08:01.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: mds.? [v2:172.21.15.161:6835/2092399552,v1:172.21.15.161:6839/2092399552] up:boot 2024-05-10T04:08:01.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/650489845' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T04:08:01.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: fsmap 12 up:standby 2024-05-10T04:08:01.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T04:08:01.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T04:08:01.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T04:08:01.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T04:08:01.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T04:08:01.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T04:08:01.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1748576072' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:08:01.738 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:01 smithi181 ceph-mon[46602]: pgmap v394: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-10T04:08:01.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T04:08:01.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T04:08:01.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: Cluster is now healthy 2024-05-10T04:08:01.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.181:6837/3259652915,v1:172.21.15.181:6839/3259652915] up:boot 2024-05-10T04:08:01.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.181:6834/2531815465,v1:172.21.15.181:6838/2531815465] up:boot 2024-05-10T04:08:01.742 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.181:6833/2354517413,v1:172.21.15.181:6836/2354517413] up:boot 2024-05-10T04:08:01.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.181:6832/3089553128,v1:172.21.15.181:6835/3089553128] up:boot 2024-05-10T04:08:01.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.161:6834/1180830718,v1:172.21.15.161:6838/1180830718] up:boot 2024-05-10T04:08:01.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: mds.? [v2:172.21.15.161:6835/2092399552,v1:172.21.15.161:6839/2092399552] up:boot 2024-05-10T04:08:01.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/650489845' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T04:08:01.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: fsmap 12 up:standby 2024-05-10T04:08:01.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T04:08:01.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T04:08:01.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T04:08:01.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T04:08:01.743 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T04:08:01.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: from='mgr.14152 172.21.15.150:0/984282551' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T04:08:01.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1748576072' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:08:01.744 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:01 smithi161 ceph-mon[46918]: pgmap v394: 129 pgs: 129 active+clean; 1.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-10T04:08:01.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:01.760+0000 7faed809c700 1 -- 172.21.15.150:0/313507984 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faed00694d0 con 0x7faed0102580 2024-05-10T04:08:02.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.391+0000 7faeceffd700 1 -- 172.21.15.150:0/313507984 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7faec8017070 con 0x7faed0102580 2024-05-10T04:08:02.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.398+0000 7faed809c700 1 -- 172.21.15.150:0/313507984 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faeb80a1240 msgr2=0x7faeb80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:02.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.398+0000 7faed809c700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faeb80a1240 0x7faeb80a3750 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faed0101ae0 tx=0x7faec000b040 comp rx=0 tx=0).stop 2024-05-10T04:08:02.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.398+0000 7faed809c700 1 -- 172.21.15.150:0/313507984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed0102580 msgr2=0x7faed019f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:02.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.398+0000 7faed809c700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed0102580 0x7faed019f260 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7faec800d980 tx=0x7faec800dd40 comp rx=0 tx=0).stop 2024-05-10T04:08:02.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.398+0000 7faed809c700 1 -- 172.21.15.150:0/313507984 shutdown_connections 2024-05-10T04:08:02.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.398+0000 7faed809c700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7faeb80a1240 0x7faeb80a3750 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:02.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.398+0000 7faed809c700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed0102580 0x7faed019f260 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:02.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.398+0000 7faed809c700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed0100ac0 0x7faed0068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:02.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.398+0000 7faed809c700 1 --2- 172.21.15.150:0/313507984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed0101be0 0x7faed019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:02.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.398+0000 7faed809c700 1 -- 172.21.15.150:0/313507984 >> 172.21.15.150:0/313507984 conn(0x7faed00fc0d0 msgr2=0x7faed0104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:08:02.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.398+0000 7faed809c700 1 -- 172.21.15.150:0/313507984 shutdown_connections 2024-05-10T04:08:02.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.399+0000 7faed809c700 1 -- 172.21.15.150:0/313507984 wait complete. 2024-05-10T04:08:02.401 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_data' removed 2024-05-10T04:08:02.418 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-10T04:08:02.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.560+0000 7f13eaacb700 1 -- 172.21.15.150:0/2953717331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e4102950 msgr2=0x7f13e4109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:02.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.560+0000 7f13eaacb700 1 --2- 172.21.15.150:0/2953717331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e4102950 0x7f13e4109250 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00b0a0 tx=0x7f13dc00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:08:02.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.560+0000 7f13eaacb700 1 -- 172.21.15.150:0/2953717331 shutdown_connections 2024-05-10T04:08:02.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.560+0000 7f13eaacb700 1 --2- 172.21.15.150:0/2953717331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e4102950 0x7f13e4109250 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:02.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.560+0000 7f13eaacb700 1 --2- 172.21.15.150:0/2953717331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e4101110 0x7f13e4101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:02.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.560+0000 7f13eaacb700 1 --2- 172.21.15.150:0/2953717331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e4102000 0x7f13e4102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:02.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.560+0000 7f13eaacb700 1 -- 172.21.15.150:0/2953717331 >> 172.21.15.150:0/2953717331 conn(0x7f13e40fb8f0 msgr2=0x7f13e40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:08:02.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.560+0000 7f13eaacb700 1 -- 172.21.15.150:0/2953717331 shutdown_connections 2024-05-10T04:08:02.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.560+0000 7f13eaacb700 1 -- 172.21.15.150:0/2953717331 wait complete. 2024-05-10T04:08:02.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13eaacb700 1 Processor -- start 2024-05-10T04:08:02.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13eaacb700 1 -- start start 2024-05-10T04:08:02.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13eaacb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e4101110 0x7f13e4198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:08:02.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13eaacb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e4102000 0x7f13e4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:08:02.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13eaacb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e4102950 0x7f13e419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:08:02.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13eaacb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e419fca0 con 0x7f13e4102000 2024-05-10T04:08:02.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13eaacb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e419fde0 con 0x7f13e4101110 2024-05-10T04:08:02.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13eaacb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e419ff70 con 0x7f13e4102950 2024-05-10T04:08:02.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13e3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e4102000 0x7f13e4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:08:02.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13e3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e4102000 0x7f13e4198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41582/0 (socket says 172.21.15.150:41582) 2024-05-10T04:08:02.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13e3fff700 1 -- 172.21.15.150:0/1113397318 learned_addr learned my addr 172.21.15.150:0/1113397318 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:08:02.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13e8867700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e4101110 0x7f13e4198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:08:02.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13e9068700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e4102950 0x7f13e419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:08:02.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13e3fff700 1 -- 172.21.15.150:0/1113397318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e4102950 msgr2=0x7f13e419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:02.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13e3fff700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e4102950 0x7f13e419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:02.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13e3fff700 1 -- 172.21.15.150:0/1113397318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e4101110 msgr2=0x7f13e4198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:02.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13e3fff700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e4101110 0x7f13e4198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:02.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.561+0000 7f13e3fff700 1 -- 172.21.15.150:0/1113397318 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc009d00 con 0x7f13e4102000 2024-05-10T04:08:02.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.562+0000 7f13e3fff700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e4102000 0x7f13e4198740 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f13d000c9e0 tx=0x7f13d000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:08:02.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.562+0000 7f13e1ffb700 1 -- 172.21.15.150:0/1113397318 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0007ac0 con 0x7f13e4102000 2024-05-10T04:08:02.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.562+0000 7f13e1ffb700 1 -- 172.21.15.150:0/1113397318 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13d0007c20 con 0x7f13e4102000 2024-05-10T04:08:02.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.562+0000 7f13eaacb700 1 -- 172.21.15.150:0/1113397318 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e41a0250 con 0x7f13e4102000 2024-05-10T04:08:02.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.562+0000 7f13e1ffb700 1 -- 172.21.15.150:0/1113397318 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0018660 con 0x7f13e4102000 2024-05-10T04:08:02.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.562+0000 7f13eaacb700 1 -- 172.21.15.150:0/1113397318 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e41a07b0 con 0x7f13e4102000 2024-05-10T04:08:02.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.563+0000 7f13eaacb700 1 -- 172.21.15.150:0/1113397318 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e40678b0 con 0x7f13e4102000 2024-05-10T04:08:02.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.564+0000 7f13e1ffb700 1 -- 172.21.15.150:0/1113397318 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f13d000f450 con 0x7f13e4102000 2024-05-10T04:08:02.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.565+0000 7f13e1ffb700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13d40a1240 0x7f13d40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:08:02.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.565+0000 7f13e8867700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13d40a1240 0x7f13d40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:08:02.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.565+0000 7f13e1ffb700 1 -- 172.21.15.150:0/1113397318 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8252+0+0 (secure 0 0 0) 0x7f13d00cc880 con 0x7f13e4102000 2024-05-10T04:08:02.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.565+0000 7f13e8867700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13d40a1240 0x7f13d40a3750 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f13d8005fd0 tx=0x7f13d800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:08:02.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.569+0000 7f13e1ffb700 1 -- 172.21.15.150:0/1113397318 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f13d0096000 con 0x7f13e4102000 2024-05-10T04:08:02.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:02 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1748576072' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T04:08:02.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:02 smithi150 ceph-mon[42477]: osdmap e89: 12 total, 12 up, 12 in 2024-05-10T04:08:02.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:02 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/313507984' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:08:02.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:02 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1748576072' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T04:08:02.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:02 smithi181 ceph-mon[46602]: osdmap e89: 12 total, 12 up, 12 in 2024-05-10T04:08:02.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:02 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/313507984' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:08:02.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:02 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1748576072' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T04:08:02.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:02 smithi161 ceph-mon[46918]: osdmap e89: 12 total, 12 up, 12 in 2024-05-10T04:08:02.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:02 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/313507984' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:08:02.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:02.766+0000 7f13eaacb700 1 -- 172.21.15.150:0/1113397318 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f13e410c700 con 0x7f13e4102000 2024-05-10T04:08:03.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:03.403+0000 7f13e1ffb700 1 -- 172.21.15.150:0/1113397318 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v91) v1 ==== 178+0+0 (secure 0 0 0) 0x7f13d0099620 con 0x7f13e4102000 2024-05-10T04:08:03.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:03.408+0000 7f13eaacb700 1 -- 172.21.15.150:0/1113397318 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13d40a1240 msgr2=0x7f13d40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:03.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:03.408+0000 7f13eaacb700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13d40a1240 0x7f13d40a3750 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f13d8005fd0 tx=0x7f13d800a560 comp rx=0 tx=0).stop 2024-05-10T04:08:03.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:03.408+0000 7f13eaacb700 1 -- 172.21.15.150:0/1113397318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e4102000 msgr2=0x7f13e4198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:08:03.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:03.408+0000 7f13eaacb700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e4102000 0x7f13e4198740 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f13d000c9e0 tx=0x7f13d000cda0 comp rx=0 tx=0).stop 2024-05-10T04:08:03.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:03.409+0000 7f13eaacb700 1 -- 172.21.15.150:0/1113397318 shutdown_connections 2024-05-10T04:08:03.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:03.409+0000 7f13eaacb700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.150:6800/1954703650,v1:172.21.15.150:6801/1954703650] conn(0x7f13d40a1240 0x7f13d40a3750 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:03.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:03.409+0000 7f13eaacb700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e4102950 0x7f13e419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:03.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:03.409+0000 7f13eaacb700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13e4101110 0x7f13e4198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:03.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:03.409+0000 7f13eaacb700 1 --2- 172.21.15.150:0/1113397318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13e4102000 0x7f13e4198740 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:08:03.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:03.409+0000 7f13eaacb700 1 -- 172.21.15.150:0/1113397318 >> 172.21.15.150:0/1113397318 conn(0x7f13e40fb8f0 msgr2=0x7f13e40ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:08:03.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:03.411+0000 7f13eaacb700 1 -- 172.21.15.150:0/1113397318 shutdown_connections 2024-05-10T04:08:03.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:08:03.411+0000 7f13eaacb700 1 -- 172.21.15.150:0/1113397318 wait complete. 2024-05-10T04:08:03.416 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_data_ec' removed 2024-05-10T04:08:03.432 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:08:03.459 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:08:03.495 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:08:03.534 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-10T04:08:03.534 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T04:08:03.551 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T04:08:03.569 DEBUG:teuthology.orchestra.run.smithi181:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T04:08:03.599 INFO:tasks.cephadm:Stopping all daemons... 2024-05-10T04:08:03.599 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-10T04:08:03.600 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.a 2024-05-10T04:08:03.713 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:03 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/313507984' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T04:08:03.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:03 smithi150 ceph-mon[42477]: osdmap e90: 12 total, 12 up, 12 in 2024-05-10T04:08:03.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:03 smithi150 ceph-mon[42477]: from='client.? 172.21.15.150:0/1113397318' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:08:03.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:03 smithi150 ceph-mon[42477]: pgmap v397: 33 pgs: 33 active+clean; 577 KiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:08:03.714 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:03 smithi150 systemd[1]: Stopping Ceph mon.a for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:03.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:03 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/313507984' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T04:08:03.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:03 smithi181 ceph-mon[46602]: osdmap e90: 12 total, 12 up, 12 in 2024-05-10T04:08:03.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:03 smithi181 ceph-mon[46602]: from='client.? 172.21.15.150:0/1113397318' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:08:03.737 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:03 smithi181 ceph-mon[46602]: pgmap v397: 33 pgs: 33 active+clean; 577 KiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:08:03.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:03 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/313507984' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T04:08:03.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:03 smithi161 ceph-mon[46918]: osdmap e90: 12 total, 12 up, 12 in 2024-05-10T04:08:03.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:03 smithi161 ceph-mon[46918]: from='client.? 172.21.15.150:0/1113397318' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:08:03.741 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:03 smithi161 ceph-mon[46918]: pgmap v397: 33 pgs: 33 active+clean; 577 KiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:08:04.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:03 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mon-a[42454]: 2024-05-10T04:08:03.855+0000 7f92b69ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:08:04.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:03 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mon-a[42454]: 2024-05-10T04:08:03.855+0000 7f92b69ee700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-10T04:08:04.213 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:08:04 smithi150 bash[115692]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mon-a 2024-05-10T04:08:04.412 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.a.service' 2024-05-10T04:08:04.480 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:08:04.480 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-10T04:08:04.480 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-10T04:08:04.481 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.b 2024-05-10T04:08:04.823 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:04 smithi161 systemd[1]: Stopping Ceph mon.b for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:04.823 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:04 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mon-b[46895]: 2024-05-10T04:08:04.708+0000 7f1aecb0f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:08:04.823 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:04 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mon-b[46895]: 2024-05-10T04:08:04.708+0000 7f1aecb0f700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-10T04:08:05.124 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:08:04 smithi161 bash[86817]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mon-b 2024-05-10T04:08:05.131 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.b.service' 2024-05-10T04:08:05.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:08:05.182 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-10T04:08:05.182 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-10T04:08:05.182 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.c 2024-05-10T04:08:05.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:05 smithi181 systemd[1]: Stopping Ceph mon.c for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:05.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:05 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mon-c[46579]: 2024-05-10T04:08:05.426+0000 7f53698d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:08:05.487 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:05 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mon-c[46579]: 2024-05-10T04:08:05.426+0000 7f53698d4700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-10T04:08:06.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:08:05 smithi181 bash[87789]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mon-c 2024-05-10T04:08:06.155 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mon.c.service' 2024-05-10T04:08:06.219 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:08:06.220 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-10T04:08:06.220 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-10T04:08:06.220 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.x 2024-05-10T04:08:06.500 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:08:06 smithi150 systemd[1]: Stopping Ceph mgr.x for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:06.500 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:08:06 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T04:08:06.471+0000 7f4e5e8dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:08:06.500 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:08:06 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x[42727]: 2024-05-10T04:08:06.471+0000 7f4e5e8dc700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-10T04:08:06.957 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:08:06 smithi150 bash[115831]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-x 2024-05-10T04:08:06.963 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-10T04:08:07.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:08:07.003 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-10T04:08:07.003 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-10T04:08:07.003 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.y 2024-05-10T04:08:07.364 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:08:07 smithi161 systemd[1]: Stopping Ceph mgr.y for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:07.674 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:08:07 smithi161 bash[86970]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-y 2024-05-10T04:08:07.674 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:08:07 smithi161 systemd[1]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-10T04:08:07.680 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-10T04:08:07.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:08:07.717 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-10T04:08:07.717 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-10T04:08:07.717 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.z 2024-05-10T04:08:08.092 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:08:07 smithi181 systemd[1]: Stopping Ceph mgr.z for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:08.487 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:08:08 smithi181 bash[87941]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mgr-z 2024-05-10T04:08:08.487 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:08:08 smithi181 systemd[1]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-10T04:08:08.526 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-10T04:08:08.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:08:08.564 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-10T04:08:08.564 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-10T04:08:08.564 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.0 2024-05-10T04:08:08.963 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:08:08 smithi150 systemd[1]: Stopping Ceph osd.0 for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:08.963 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:08:08 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-0[60933]: 2024-05-10T04:08:08.804+0000 7f8daf1eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:08:08.963 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:08:08 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-0[60933]: 2024-05-10T04:08:08.804+0000 7f8daf1eb700 -1 osd.0 91 *** Got signal Terminated *** 2024-05-10T04:08:08.963 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:08:08 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-0[60933]: 2024-05-10T04:08:08.804+0000 7f8daf1eb700 -1 osd.0 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:08:14.713 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:08:14 smithi150 bash[115972]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-0 2024-05-10T04:08:15.726 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.0.service' 2024-05-10T04:08:15.768 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:08:15.768 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-10T04:08:15.769 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-10T04:08:15.769 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.1 2024-05-10T04:08:16.241 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:08:15 smithi161 systemd[1]: Stopping Ceph osd.1 for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:16.242 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:08:16 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-1[51890]: 2024-05-10T04:08:16.015+0000 7f262561d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:08:16.242 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:08:16 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-1[51890]: 2024-05-10T04:08:16.015+0000 7f262561d700 -1 osd.1 91 *** Got signal Terminated *** 2024-05-10T04:08:16.242 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:08:16 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-1[51890]: 2024-05-10T04:08:16.015+0000 7f262561d700 -1 osd.1 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:08:21.991 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:08:21 smithi161 bash[87109]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-1 2024-05-10T04:08:22.812 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.1.service' 2024-05-10T04:08:22.853 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:08:22.854 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-10T04:08:22.854 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-10T04:08:22.854 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.2 2024-05-10T04:08:23.237 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:08:22 smithi181 systemd[1]: Stopping Ceph osd.2 for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:23.238 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:08:23 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-2[52428]: 2024-05-10T04:08:23.090+0000 7fedc6ee5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:08:23.238 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:08:23 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-2[52428]: 2024-05-10T04:08:23.090+0000 7fedc6ee5700 -1 osd.2 91 *** Got signal Terminated *** 2024-05-10T04:08:23.238 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:08:23 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-2[52428]: 2024-05-10T04:08:23.090+0000 7fedc6ee5700 -1 osd.2 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:08:28.987 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:08:28 smithi181 bash[88093]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-2 2024-05-10T04:08:29.963 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.2.service' 2024-05-10T04:08:30.005 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:08:30.005 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-10T04:08:30.005 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-10T04:08:30.005 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.3 2024-05-10T04:08:30.463 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 04:08:30 smithi150 systemd[1]: Stopping Ceph osd.3 for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:30.463 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 04:08:30 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-3[68825]: 2024-05-10T04:08:30.251+0000 7f39275a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:08:30.463 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 04:08:30 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-3[68825]: 2024-05-10T04:08:30.251+0000 7f39275a8700 -1 osd.3 91 *** Got signal Terminated *** 2024-05-10T04:08:30.463 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 04:08:30 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-3[68825]: 2024-05-10T04:08:30.251+0000 7f39275a8700 -1 osd.3 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:08:35.991 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:35 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:35.491+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:36.213 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 04:08:35 smithi150 bash[116231]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-3 2024-05-10T04:08:36.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:36 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:36.462+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:37.190 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.3.service' 2024-05-10T04:08:37.231 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:08:37.231 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-10T04:08:37.231 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-10T04:08:37.231 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.4 2024-05-10T04:08:37.737 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:37 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:37.402+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.614437+0000 front 2024-05-10T04:08:13.614535+0000 (oldest deadline 2024-05-10T04:08:36.514583+0000) 2024-05-10T04:08:37.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:37 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:37.301+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:37.741 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 04:08:37 smithi161 systemd[1]: Stopping Ceph osd.4 for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:37.742 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 04:08:37 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-4[59786]: 2024-05-10T04:08:37.478+0000 7f40fe1ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:08:37.742 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 04:08:37 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-4[59786]: 2024-05-10T04:08:37.478+0000 7f40fe1ab700 -1 osd.4 91 *** Got signal Terminated *** 2024-05-10T04:08:37.742 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 04:08:37 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-4[59786]: 2024-05-10T04:08:37.478+0000 7f40fe1ab700 -1 osd.4 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:08:37.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:37 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:37.440+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:38.737 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:38 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:38.415+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.614437+0000 front 2024-05-10T04:08:13.614535+0000 (oldest deadline 2024-05-10T04:08:36.514583+0000) 2024-05-10T04:08:38.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:38 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:38.330+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:38.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:38 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:38.421+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:39.737 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:39 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:39.405+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.614437+0000 front 2024-05-10T04:08:13.614535+0000 (oldest deadline 2024-05-10T04:08:36.514583+0000) 2024-05-10T04:08:39.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:39 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:39.357+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:39.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:39 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:39.391+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:40.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:40 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:40.044+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:40.737 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:40 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:40.368+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.614437+0000 front 2024-05-10T04:08:13.614535+0000 (oldest deadline 2024-05-10T04:08:36.514583+0000) 2024-05-10T04:08:40.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:40 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:40.342+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:40.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:40 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:40.356+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:41.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:41 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:41.023+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:41.609 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:41 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:41.349+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:41.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:41 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:41.351+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:41.737 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:41 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:41.365+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.614437+0000 front 2024-05-10T04:08:13.614535+0000 (oldest deadline 2024-05-10T04:08:36.514583+0000) 2024-05-10T04:08:41.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:41 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:41.607+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:42.213 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:42 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:42.010+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:42.213 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:41 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:41.780+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:42.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:42 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:42.326+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:42.737 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:42 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:42.377+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.614437+0000 front 2024-05-10T04:08:13.614535+0000 (oldest deadline 2024-05-10T04:08:36.514583+0000) 2024-05-10T04:08:42.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:42 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:42.448+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:42.741 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:42 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:42.603+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:42.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:42 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:42.353+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:42.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:42 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:42.353+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:43.213 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:42 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:42.802+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:43.213 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:42 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:42.986+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:43.491 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 04:08:43 smithi161 bash[87381]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-4 2024-05-10T04:08:43.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:43 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:43.317+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:43.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:43 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:43.317+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:43.737 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:43 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:43.413+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.614437+0000 front 2024-05-10T04:08:13.614535+0000 (oldest deadline 2024-05-10T04:08:36.514583+0000) 2024-05-10T04:08:43.737 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:43 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:43.413+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:16.515130+0000 front 2024-05-10T04:08:16.515191+0000 (oldest deadline 2024-05-10T04:08:42.915822+0000) 2024-05-10T04:08:43.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:43 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:43.480+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:43.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:43 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:43.284+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:43.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:43 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:43.621+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:44.213 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:43 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:43.777+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:44.213 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:44 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:44.010+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:44.392 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.4.service' 2024-05-10T04:08:44.435 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:08:44.435 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-10T04:08:44.435 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-10T04:08:44.436 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.5 2024-05-10T04:08:44.613 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:44 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:44.357+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:44.614 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:44 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:44.357+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:44.672 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:44 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:44.473+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:44.673 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:44 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:44.391+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.614437+0000 front 2024-05-10T04:08:13.614535+0000 (oldest deadline 2024-05-10T04:08:36.514583+0000) 2024-05-10T04:08:44.673 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:44 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:44.391+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:16.515130+0000 front 2024-05-10T04:08:16.515191+0000 (oldest deadline 2024-05-10T04:08:42.915822+0000) 2024-05-10T04:08:44.673 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:44 smithi181 systemd[1]: Stopping Ceph osd.5 for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:44.673 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:44 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:44.249+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:44.673 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:44 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:44.249+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:44.987 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:44 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:44.670+0000 7fbfebbb5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:08:44.987 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:44 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:44.670+0000 7fbfebbb5700 -1 osd.5 91 *** Got signal Terminated *** 2024-05-10T04:08:44.987 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:44 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:44.670+0000 7fbfebbb5700 -1 osd.5 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:08:44.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:44 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:44.612+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:45.049 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:44 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:44.742+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:45.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:45 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:45.048+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:45.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:45 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:45.048+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:45.661 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:45 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:45.385+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:45.662 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:45 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:45.385+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:45.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:45 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:45.461+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:45.738 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:45 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:45.420+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.614437+0000 front 2024-05-10T04:08:13.614535+0000 (oldest deadline 2024-05-10T04:08:36.514583+0000) 2024-05-10T04:08:45.738 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:45 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:45.420+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:16.515130+0000 front 2024-05-10T04:08:16.515191+0000 (oldest deadline 2024-05-10T04:08:42.915822+0000) 2024-05-10T04:08:45.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:45 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:45.288+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:45.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:45 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:45.288+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:45.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:45 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:45.700+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:45.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:45 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:45.660+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:46.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:46 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:46.082+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:46.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:46 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:46.082+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:46.679 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:46 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:46.371+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:46.679 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:46 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:46.371+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:46.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:46 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:46.460+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:46.737 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:46 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:46.410+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.614437+0000 front 2024-05-10T04:08:13.614535+0000 (oldest deadline 2024-05-10T04:08:36.514583+0000) 2024-05-10T04:08:46.737 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:46 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:46.410+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:16.515130+0000 front 2024-05-10T04:08:16.515191+0000 (oldest deadline 2024-05-10T04:08:42.915822+0000) 2024-05-10T04:08:46.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:46 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:46.288+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:46.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:46 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:46.288+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:46.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:46 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:46.662+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:46.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:46 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:46.678+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:47.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:47 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:47.058+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:47.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:47 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:47.058+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:47.506 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:47 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:47.370+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.614437+0000 front 2024-05-10T04:08:13.614535+0000 (oldest deadline 2024-05-10T04:08:36.514583+0000) 2024-05-10T04:08:47.506 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:47 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:47.370+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:16.515130+0000 front 2024-05-10T04:08:16.515191+0000 (oldest deadline 2024-05-10T04:08:42.915822+0000) 2024-05-10T04:08:47.506 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:47 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:47.245+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:47.506 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:47 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:47.245+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:47.696 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:47 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:47.407+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:47.696 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:47 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:47.407+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:47.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:47 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:47.615+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:47.987 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:47 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:47.502+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:47.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:47 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:47.695+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:48.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:48 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:48.012+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:48.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:48 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:48.012+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:48.467 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:48 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:48.329+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.614437+0000 front 2024-05-10T04:08:13.614535+0000 (oldest deadline 2024-05-10T04:08:36.514583+0000) 2024-05-10T04:08:48.467 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:48 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:48.329+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:16.515130+0000 front 2024-05-10T04:08:16.515191+0000 (oldest deadline 2024-05-10T04:08:42.915822+0000) 2024-05-10T04:08:48.467 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:48 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:48.204+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:48.467 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:48 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:48.204+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:48.669 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:48 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:48.402+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:48.669 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:48 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:48.402+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:48.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:48 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:48.465+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:48.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:48 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:48.572+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:48.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:48 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:48.572+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.363869+0000 front 2024-05-10T04:08:27.363808+0000 (oldest deadline 2024-05-10T04:08:48.463883+0000) 2024-05-10T04:08:48.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:48 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:48.667+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:49.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:49 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:49.009+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:49.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:49 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:49.009+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:49.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:49 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:49.009+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:08:49.678 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:49 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:49.374+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:49.678 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:49 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:49.374+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:49.737 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:49 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:49.301+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.614437+0000 front 2024-05-10T04:08:13.614535+0000 (oldest deadline 2024-05-10T04:08:36.514583+0000) 2024-05-10T04:08:49.737 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:49 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5[60404]: 2024-05-10T04:08:49.301+0000 7fbfe4165700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:16.515130+0000 front 2024-05-10T04:08:16.515191+0000 (oldest deadline 2024-05-10T04:08:42.915822+0000) 2024-05-10T04:08:49.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:49 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:49.495+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:49.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:49 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:49.495+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:08:49.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:49 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:49.249+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:49.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:49 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:49.250+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:49.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:49 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:49.568+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:49.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:49 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:49.568+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.363869+0000 front 2024-05-10T04:08:27.363808+0000 (oldest deadline 2024-05-10T04:08:48.463883+0000) 2024-05-10T04:08:49.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:49 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:49.677+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:50.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:50 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:50.001+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:50.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:50 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:50.001+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:50.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:50 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:50.001+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:08:50.484 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:08:50 smithi181 bash[88348]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-5 2024-05-10T04:08:50.484 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:50 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:50.206+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:50.485 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:50 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:50.206+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:50.485 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:50 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:50.206+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:08:50.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:50 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:50.481+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:50.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:50 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:50.481+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:08:50.741 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:50 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:50.641+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:50.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:50 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:50.402+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:50.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:50 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:50.402+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:50.962 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:50 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:50.599+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:50.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:50 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:50.599+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.363869+0000 front 2024-05-10T04:08:27.363808+0000 (oldest deadline 2024-05-10T04:08:48.463883+0000) 2024-05-10T04:08:51.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:51 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:51.040+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:51.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:51 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:51.040+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:51.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:51 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:51.040+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:08:51.514 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:51 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:51.238+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:51.514 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:51 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:51.238+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:51.514 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:51 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:51.238+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:08:51.618 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.5.service' 2024-05-10T04:08:51.658 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:08:51.658 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-10T04:08:51.658 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-10T04:08:51.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.6 2024-05-10T04:08:51.741 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:51 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:51.642+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:51.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:51 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:51.394+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:51.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:51 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:51.394+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:51.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:51 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:51.642+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:51.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:51 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:51.642+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.363869+0000 front 2024-05-10T04:08:27.363808+0000 (oldest deadline 2024-05-10T04:08:48.463883+0000) 2024-05-10T04:08:51.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:51 smithi150 systemd[1]: Stopping Ceph osd.6 for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:51.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:51 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:51.891+0000 7faee050d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:08:51.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:51 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:51.891+0000 7faee050d700 -1 osd.6 91 *** Got signal Terminated *** 2024-05-10T04:08:51.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:51 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:51.891+0000 7faee050d700 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:08:51.986 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:51 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:51.511+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:51.987 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:51 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:51.511+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:08:52.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:52 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:52.035+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:52.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:52 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:52.035+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:52.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:52 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:52.035+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:08:52.487 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:52 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:52.223+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:52.487 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:52 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:52.223+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:52.487 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:52 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:52.223+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:08:52.684 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:52 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:52.348+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:52.684 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:52 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:52.348+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:52.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:52 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:52.616+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:52.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:52 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:52.616+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.363869+0000 front 2024-05-10T04:08:27.363808+0000 (oldest deadline 2024-05-10T04:08:48.463883+0000) 2024-05-10T04:08:52.987 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:52 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:52.561+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:52.987 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:52 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:52.561+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:08:52.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:52 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:52.683+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:53.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:53 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:53.085+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:53.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:53 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:53.085+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:53.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:53 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:53.085+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:08:53.556 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:53 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:53.267+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:53.556 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:53 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:53.267+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:53.556 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:53 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:53.267+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:08:53.707 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:53 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:53.354+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:53.707 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:53 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:53.354+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:53.707 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:53 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:53.354+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:08:53.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:53 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:53.600+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:53.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:53 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:53.600+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.363869+0000 front 2024-05-10T04:08:27.363808+0000 (oldest deadline 2024-05-10T04:08:48.463883+0000) 2024-05-10T04:08:53.987 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:53 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:53.555+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:53.987 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:53 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:53.555+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:08:53.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:53 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:53.706+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:54.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:54 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:54.060+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:54.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:54 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:54.060+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:54.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:54 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:54.060+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:08:54.678 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:54 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:54.320+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:54.679 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:54 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:54.320+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:54.679 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:54 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:54.320+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:08:54.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:54 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:54.287+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:54.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:54 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:54.287+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:54.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:54 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:54.287+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:08:54.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:54 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:54.536+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:54.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:54 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:54.536+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:08:54.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:54 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:54.554+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:54.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:54 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:54.554+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.363869+0000 front 2024-05-10T04:08:27.363808+0000 (oldest deadline 2024-05-10T04:08:48.463883+0000) 2024-05-10T04:08:54.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:54 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:54.676+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:54.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:54 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:54.676+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:23.085830+0000 front 2024-05-10T04:08:23.085795+0000 (oldest deadline 2024-05-10T04:08:54.286518+0000) 2024-05-10T04:08:55.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:55 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:55.030+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:55.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:55 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:55.030+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:55.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:55 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:55.030+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:08:55.674 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:55 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:55.367+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:55.674 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:55 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:55.367+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:55.674 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:55 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:55.367+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:08:55.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:55 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:55.490+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:55.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:55 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:55.490+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:08:55.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:55 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:55.311+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:55.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:55 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:55.311+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:55.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:55 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:55.311+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:08:55.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:55 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:55.509+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:55.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:55 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:55.509+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.363869+0000 front 2024-05-10T04:08:27.363808+0000 (oldest deadline 2024-05-10T04:08:48.463883+0000) 2024-05-10T04:08:55.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:55 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:55.672+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:55.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:55 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:55.672+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:23.085830+0000 front 2024-05-10T04:08:23.085795+0000 (oldest deadline 2024-05-10T04:08:54.286518+0000) 2024-05-10T04:08:56.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:56 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:56.022+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:56.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:56 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:56.022+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:56.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:56 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:56.022+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:08:56.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:56 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:56.022+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:08:56.688 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:56 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:56.391+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:56.688 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:56 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:56.391+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:56.688 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:56 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:56.391+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:08:56.688 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:56 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:56.391+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:08:56.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:56 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:56.354+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:56.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:56 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:56.354+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:56.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:56 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:56.354+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:08:56.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:56 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:56.488+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:56.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:56 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:56.488+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:08:56.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:56 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:56.541+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.962499+0000 front 2024-05-10T04:08:20.962543+0000 (oldest deadline 2024-05-10T04:08:41.462622+0000) 2024-05-10T04:08:56.963 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:56 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6[76778]: 2024-05-10T04:08:56.541+0000 7faed8abd700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.363869+0000 front 2024-05-10T04:08:27.363808+0000 (oldest deadline 2024-05-10T04:08:48.463883+0000) 2024-05-10T04:08:56.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:56 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:56.687+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:56.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:56 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:56.687+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:23.085830+0000 front 2024-05-10T04:08:23.085795+0000 (oldest deadline 2024-05-10T04:08:54.286518+0000) 2024-05-10T04:08:57.438 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:56 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:56.978+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:57.438 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:56 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:56.978+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:57.439 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:56 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:56.978+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:08:57.439 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:56 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:56.978+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:08:57.640 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:57 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:57.358+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:57.641 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:57 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:57.358+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:57.641 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:57 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:57.358+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:08:57.641 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:57 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:57.358+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:08:57.713 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:08:57 smithi150 bash[116488]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-6 2024-05-10T04:08:57.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:57 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:57.496+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:57.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:57 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:57.496+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:08:57.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:57 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:57.373+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:57.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:57 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:57.373+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:57.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:57 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:57.373+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:08:57.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:57 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:57.639+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:57.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:57 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:57.639+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:23.085830+0000 front 2024-05-10T04:08:23.085795+0000 (oldest deadline 2024-05-10T04:08:54.286518+0000) 2024-05-10T04:08:58.445 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:58 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:58.028+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:58.445 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:58 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:58.028+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:58.445 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:58 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:58.028+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:08:58.446 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:58 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:58.028+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:08:58.605 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:58 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:58.325+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:58.605 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:58 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:58.325+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:58.605 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:58 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:58.325+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:08:58.606 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:58 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:58.325+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:08:58.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:58 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:58.506+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:58.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:58 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:58.506+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:08:58.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:58 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:58.414+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:58.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:58 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:58.414+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:58.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:58 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:58.414+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:08:58.796 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.6.service' 2024-05-10T04:08:58.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:08:58.837 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-10T04:08:58.837 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-10T04:08:58.837 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.7 2024-05-10T04:08:58.868 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:58 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:58.604+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:58.868 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:58 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:58.604+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:23.085830+0000 front 2024-05-10T04:08:23.085795+0000 (oldest deadline 2024-05-10T04:08:54.286518+0000) 2024-05-10T04:08:59.241 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:58 smithi161 systemd[1]: Stopping Ceph osd.7 for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:08:59.241 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:59.107+0000 7f18ae497700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:08:59.241 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:59.107+0000 7f18ae497700 -1 osd.7 91 *** Got signal Terminated *** 2024-05-10T04:08:59.242 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:59.107+0000 7f18ae497700 -1 osd.7 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:08:59.462 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:59 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:59.068+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:08:59.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:59 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:59.068+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:08:59.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:59 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:59.068+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:08:59.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:08:59 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:08:59.068+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:08:59.564 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:59.308+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:08:59.565 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:59.308+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:08:59.565 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:59.308+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:08:59.565 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:08:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:08:59.308+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:08:59.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:59.523+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:08:59.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:08:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:08:59.523+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:08:59.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:59.373+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:08:59.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:59.373+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:08:59.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:08:59 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:08:59.373+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:08:59.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:59.563+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:08:59.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:59.563+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:23.085830+0000 front 2024-05-10T04:08:23.085795+0000 (oldest deadline 2024-05-10T04:08:54.286518+0000) 2024-05-10T04:08:59.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:08:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:08:59.563+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.287085+0000 front 2024-05-10T04:08:34.287099+0000 (oldest deadline 2024-05-10T04:08:58.986936+0000) 2024-05-10T04:09:00.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:00 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:00.108+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:00.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:00 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:00.108+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:00.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:00 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:00.108+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:00.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:00 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:00.108+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:00.559 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:00 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:00.289+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:00.559 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:00 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:00.289+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:00.559 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:00 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:00.289+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:00.559 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:00 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:00.289+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:00.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:00 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:00.477+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:09:00.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:00 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:00.477+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:09:00.739 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:00 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:00.477+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.317239+0000 front 2024-05-10T04:08:34.317167+0000 (oldest deadline 2024-05-10T04:09:00.217227+0000) 2024-05-10T04:09:00.739 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:00 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:00.401+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:00.739 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:00 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:00.401+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:00.739 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:00 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:00.401+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:00.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:00 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:09:00.557+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:09:00.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:00 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:09:00.557+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:23.085830+0000 front 2024-05-10T04:08:23.085795+0000 (oldest deadline 2024-05-10T04:08:54.286518+0000) 2024-05-10T04:09:00.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:00 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:09:00.557+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.287085+0000 front 2024-05-10T04:08:34.287099+0000 (oldest deadline 2024-05-10T04:08:58.986936+0000) 2024-05-10T04:09:01.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:01 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:01.088+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:01.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:01 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:01.088+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:01.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:01 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:01.088+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:01.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:01 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:01.088+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:01.586 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:01 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:01.334+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:01.586 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:01 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:01.334+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:01.586 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:01 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:01.334+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:01.587 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:01 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:01.334+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:01.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:01 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:01.526+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:09:01.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:01 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:01.526+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:09:01.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:01 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:01.526+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.317239+0000 front 2024-05-10T04:08:34.317167+0000 (oldest deadline 2024-05-10T04:09:00.217227+0000) 2024-05-10T04:09:01.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:01 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:01.393+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:01.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:01 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:01.393+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:01.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:01 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:01.393+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:01.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:01 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:09:01.585+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:09:01.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:01 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:09:01.585+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:23.085830+0000 front 2024-05-10T04:08:23.085795+0000 (oldest deadline 2024-05-10T04:08:54.286518+0000) 2024-05-10T04:09:01.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:01 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:09:01.585+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.287085+0000 front 2024-05-10T04:08:34.287099+0000 (oldest deadline 2024-05-10T04:08:58.986936+0000) 2024-05-10T04:09:02.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:02 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:02.083+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:02.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:02 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:02.083+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:02.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:02 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:02.083+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:02.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:02 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:02.083+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:02.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:02 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:02.558+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:09:02.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:02 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:02.558+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:09:02.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:02 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:02.558+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.317239+0000 front 2024-05-10T04:08:34.317167+0000 (oldest deadline 2024-05-10T04:09:00.217227+0000) 2024-05-10T04:09:02.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:02 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:02.424+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:02.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:02 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:02.424+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:02.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:02 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:02.424+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:02.741 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:02 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:09:02.542+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:09:02.741 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:02 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:09:02.542+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:23.085830+0000 front 2024-05-10T04:08:23.085795+0000 (oldest deadline 2024-05-10T04:08:54.286518+0000) 2024-05-10T04:09:02.741 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:02 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:09:02.542+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.287085+0000 front 2024-05-10T04:08:34.287099+0000 (oldest deadline 2024-05-10T04:08:58.986936+0000) 2024-05-10T04:09:02.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:02 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:02.355+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:02.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:02 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:02.355+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:02.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:02 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:02.355+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:02.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:02 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:02.355+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:03.462 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:03 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:03.043+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:03.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:03 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:03.043+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:03.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:03 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:03.043+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:03.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:03 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:03.043+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:03.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:03 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:03.388+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:03.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:03 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:03.388+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:03.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:03 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:03.388+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:03.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:03 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:03.582+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:09:03.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:03 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:03.582+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:09:03.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:03 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:03.582+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.317239+0000 front 2024-05-10T04:08:34.317167+0000 (oldest deadline 2024-05-10T04:09:00.217227+0000) 2024-05-10T04:09:03.741 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:03 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:09:03.559+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:15.583867+0000 front 2024-05-10T04:08:15.583843+0000 (oldest deadline 2024-05-10T04:08:41.484186+0000) 2024-05-10T04:09:03.742 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:03 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:09:03.559+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:23.085830+0000 front 2024-05-10T04:08:23.085795+0000 (oldest deadline 2024-05-10T04:08:54.286518+0000) 2024-05-10T04:09:03.742 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:03 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:09:03.559+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.287085+0000 front 2024-05-10T04:08:34.287099+0000 (oldest deadline 2024-05-10T04:08:58.986936+0000) 2024-05-10T04:09:03.742 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:03 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7[68335]: 2024-05-10T04:09:03.559+0000 7f18a6a47700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.987271+0000 front 2024-05-10T04:08:38.987326+0000 (oldest deadline 2024-05-10T04:09:03.087421+0000) 2024-05-10T04:09:03.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:03 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:03.354+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:03.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:03 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:03.354+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:03.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:03 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:03.354+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:03.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:03 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:03.354+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:04.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:04.008+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:04.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:04.008+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:04.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:04.008+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:04.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:04.008+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:04.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:04.008+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:04.614 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:04 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:04.314+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:04.614 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:04 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:04.314+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:04.614 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:04 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:04.314+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:04.614 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:04 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:04.314+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:04.624 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:04 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:04.352+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:04.624 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:04 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:04.352+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:04.624 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:04 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:04.352+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:04.624 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:04 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:04.352+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:04.987 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:04 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:04.622+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:09:04.987 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:04 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:04.622+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:09:04.987 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:04 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:04.622+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.317239+0000 front 2024-05-10T04:08:34.317167+0000 (oldest deadline 2024-05-10T04:09:00.217227+0000) 2024-05-10T04:09:04.991 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:09:04 smithi161 bash[87635]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-7 2024-05-10T04:09:05.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:04.995+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:05.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:04.995+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:05.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:04.995+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:05.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:04.995+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:05.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:04 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:04.995+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:05.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:05 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:05.305+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:05.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:05 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:05.305+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:05.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:05 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:05.305+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:05.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:05 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:05.305+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:05.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:05 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:05.305+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:05.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:05 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:05.385+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:05.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:05 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:05.385+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:05.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:05 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:05.385+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:05.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:05 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:05.385+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:05.923 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.7.service' 2024-05-10T04:09:05.988 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:05 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:05.652+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:09:05.988 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:05 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:05.652+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:09:05.988 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:05 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:05.652+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.317239+0000 front 2024-05-10T04:08:34.317167+0000 (oldest deadline 2024-05-10T04:09:00.217227+0000) 2024-05-10T04:09:06.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:09:06.023 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-10T04:09:06.023 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-10T04:09:06.024 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.8 2024-05-10T04:09:06.394 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:06 smithi181 systemd[1]: Stopping Ceph osd.8 for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:09:06.395 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:06 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:06.267+0000 7f5230a01700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:09:06.395 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:06 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:06.267+0000 7f5230a01700 -1 osd.8 91 *** Got signal Terminated *** 2024-05-10T04:09:06.395 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:06 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:06.267+0000 7f5230a01700 -1 osd.8 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:09:06.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:06 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:06.019+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:06.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:06 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:06.019+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:06.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:06 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:06.019+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:06.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:06 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:06.019+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:06.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:06 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:06.019+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:06.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:06 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:06.393+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:06.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:06 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:06.393+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:06.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:06 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:06.393+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:06.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:06 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:06.393+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:06.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:06 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:06.627+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:09:06.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:06 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:06.627+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:09:06.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:06 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:06.627+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.317239+0000 front 2024-05-10T04:08:34.317167+0000 (oldest deadline 2024-05-10T04:09:00.217227+0000) 2024-05-10T04:09:06.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:06 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:06.627+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:41.318375+0000 front 2024-05-10T04:08:41.318433+0000 (oldest deadline 2024-05-10T04:09:06.618341+0000) 2024-05-10T04:09:06.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:06 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:06.264+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:06.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:06 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:06.264+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:06.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:06 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:06.264+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:06.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:06 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:06.264+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:06.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:06 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:06.264+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:07.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:07 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:07.061+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:07.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:07 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:07.061+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:07.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:07 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:07.061+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:07.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:07 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:07.061+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:07.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:07 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:07.061+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:07.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:07 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:07.413+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:07.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:07 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:07.413+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:07.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:07 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:07.413+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:07.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:07 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:07.413+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:07.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:07 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:07.650+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:09:07.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:07 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:07.650+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:09:07.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:07 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:07.650+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.317239+0000 front 2024-05-10T04:08:34.317167+0000 (oldest deadline 2024-05-10T04:09:00.217227+0000) 2024-05-10T04:09:07.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:07 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:07.650+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:41.318375+0000 front 2024-05-10T04:08:41.318433+0000 (oldest deadline 2024-05-10T04:09:06.618341+0000) 2024-05-10T04:09:07.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:07 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:07.278+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:07.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:07 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:07.278+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:07.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:07 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:07.278+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:07.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:07 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:07.278+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:07.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:07 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:07.278+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:08.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:08 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:08.023+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:08.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:08 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:08.023+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:08.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:08 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:08.023+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:08.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:08 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:08.023+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:08.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:08 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:08.023+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:08.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:08 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:08.418+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:08.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:08 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:08.418+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:08.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:08 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:08.418+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:08.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:08 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:08.418+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:08.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:08 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:08.613+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:09:08.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:08 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:08.613+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:09:08.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:08 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:08.613+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.317239+0000 front 2024-05-10T04:08:34.317167+0000 (oldest deadline 2024-05-10T04:09:00.217227+0000) 2024-05-10T04:09:08.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:08 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:08.613+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:41.318375+0000 front 2024-05-10T04:08:41.318433+0000 (oldest deadline 2024-05-10T04:09:06.618341+0000) 2024-05-10T04:09:08.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:08 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:08.248+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:08.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:08 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:08.248+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:08.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:08 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:08.248+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:08.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:08 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:08.248+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:08.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:08 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:08.248+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:09.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:09 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:09.040+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:09.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:09 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:09.040+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:09.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:09 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:09.040+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:09.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:09 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:09.040+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:09.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:09 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:09.040+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:09.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:09 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:09.596+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:09:09.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:09 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:09.596+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:09:09.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:09 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:09.596+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.317239+0000 front 2024-05-10T04:08:34.317167+0000 (oldest deadline 2024-05-10T04:09:00.217227+0000) 2024-05-10T04:09:09.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:09 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:09.596+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:41.318375+0000 front 2024-05-10T04:08:41.318433+0000 (oldest deadline 2024-05-10T04:09:06.618341+0000) 2024-05-10T04:09:09.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:09 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:09.433+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:09.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:09 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:09.433+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:09.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:09 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:09.433+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:09.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:09 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:09.433+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:09.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:09 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:09.270+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:09.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:09 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:09.270+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:09.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:09 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:09.270+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:09.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:09 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:09.270+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:09.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:09 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:09.270+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:10.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:10 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:10.065+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:10.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:10 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:10.065+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:10.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:10 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:10.065+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:10.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:10 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:10.065+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:10.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:10 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:10.065+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:10.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:10 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:10.605+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.514854+0000 front 2024-05-10T04:08:18.514808+0000 (oldest deadline 2024-05-10T04:08:42.014880+0000) 2024-05-10T04:09:10.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:10 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:10.605+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:27.916167+0000 front 2024-05-10T04:08:27.916068+0000 (oldest deadline 2024-05-10T04:08:49.016215+0000) 2024-05-10T04:09:10.737 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:10 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:10.605+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:34.317239+0000 front 2024-05-10T04:08:34.317167+0000 (oldest deadline 2024-05-10T04:09:00.217227+0000) 2024-05-10T04:09:10.738 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:10 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8[68985]: 2024-05-10T04:09:10.605+0000 7f5228fb1700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:41.318375+0000 front 2024-05-10T04:08:41.318433+0000 (oldest deadline 2024-05-10T04:09:06.618341+0000) 2024-05-10T04:09:10.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:10 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:10.472+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:10.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:10 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:10.472+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:10.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:10 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:10.472+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:10.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:10 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:10.472+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:10.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:10 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:10.291+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:10.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:10 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:10.291+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:10.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:10 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:10.291+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:10.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:10 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:10.291+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:10.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:10 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:10.291+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:11.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:11 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:11.059+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:11.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:11 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:11.059+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:11.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:11 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:11.059+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:11.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:11 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:11.059+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:11.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:11 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:11.059+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:11.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:11 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:11.443+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:11.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:11 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:11.443+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:11.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:11 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:11.443+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:11.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:11 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:11.443+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:11.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:11 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:11.443+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:11.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:11 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:11.255+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:11.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:11 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:11.255+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:11.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:11 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:11.255+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:11.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:11 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:11.255+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:11.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:11 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:11.255+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:11.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:11 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:11.255+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:12.237 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:09:11 smithi181 bash[88602]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-8 2024-05-10T04:09:12.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:12.097+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:12.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:12.097+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:12.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:12.097+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:12.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:12.097+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:12.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:12 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:12.097+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:12.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:12 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:12.465+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:12.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:12 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:12.465+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:12.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:12 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:12.465+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:12.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:12 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:12.465+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:12.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:12 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:12.465+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:12.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:12 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:12.248+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:12.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:12 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:12.248+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:12.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:12 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:12.248+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:12.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:12 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:12.248+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:12.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:12 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:12.248+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:12.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:12 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:12.248+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:13.223 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.8.service' 2024-05-10T04:09:13.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:09:13.266 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-10T04:09:13.266 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-10T04:09:13.266 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.9 2024-05-10T04:09:13.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:13 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:13.108+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:13.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:13 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:13.108+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:13.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:13 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:13.108+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:13.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:13 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:13.108+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:13.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:13 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:13.108+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:13.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:13 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:13.108+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:49.193720+0000 front 2024-05-10T04:08:49.193664+0000 (oldest deadline 2024-05-10T04:09:12.594110+0000) 2024-05-10T04:09:13.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:13 smithi150 systemd[1]: Stopping Ceph osd.9 for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:09:13.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:13 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:13.262+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:13.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:13 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:13.262+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:13.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:13 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:13.262+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:13.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:13 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:13.262+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:13.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:13 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:13.262+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:13.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:13 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:13.262+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:13.963 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:13 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:13.496+0000 7f12c077d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:09:13.963 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:13 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:13.496+0000 7f12c077d700 -1 osd.9 91 *** Got signal Terminated *** 2024-05-10T04:09:13.963 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:13 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:13.496+0000 7f12c077d700 -1 osd.9 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:09:13.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:13 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:13.489+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:13.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:13 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:13.489+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:13.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:13 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:13.489+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:13.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:13 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:13.489+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:13.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:13 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:13.489+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:14.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:14 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:14.124+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:14.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:14 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:14.124+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:14.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:14 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:14.124+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:14.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:14 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:14.124+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:14.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:14 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:14.124+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:14.464 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:14 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:14.124+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:49.193720+0000 front 2024-05-10T04:08:49.193664+0000 (oldest deadline 2024-05-10T04:09:12.594110+0000) 2024-05-10T04:09:14.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:14 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:14.268+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:14.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:14 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:14.268+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:14.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:14 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:14.268+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:14.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:14 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:14.268+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:14.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:14 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:14.268+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:14.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:14 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:14.268+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:14.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:14 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:14.527+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:14.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:14 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:14.527+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:14.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:14 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:14.527+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:14.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:14 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:14.527+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:14.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:14 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:14.527+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:15.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:15 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:15.142+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:15.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:15 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:15.142+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:15.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:15 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:15.142+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:15.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:15 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:15.142+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:15.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:15 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:15.142+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:15.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:15 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:15.142+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:49.193720+0000 front 2024-05-10T04:08:49.193664+0000 (oldest deadline 2024-05-10T04:09:12.594110+0000) 2024-05-10T04:09:15.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:15 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:15.237+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:15.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:15 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:15.237+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:15.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:15 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:15.237+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:15.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:15 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:15.237+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:15.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:15 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:15.237+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:15.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:15 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:15.237+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:15.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:15 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:15.481+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:15.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:15 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:15.481+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:15.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:15 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:15.481+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:15.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:15 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:15.481+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:15.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:15 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:15.481+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:16.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:16 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:16.100+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:16.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:16 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:16.100+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:16.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:16 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:16.100+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:16.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:16 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:16.100+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:16.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:16 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:16.100+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:16.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:16 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:16.100+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:49.193720+0000 front 2024-05-10T04:08:49.193664+0000 (oldest deadline 2024-05-10T04:09:12.594110+0000) 2024-05-10T04:09:16.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:16 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:16.215+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:16.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:16 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:16.215+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:16.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:16 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:16.215+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:16.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:16 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:16.215+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:16.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:16 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:16.215+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:16.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:16 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:16.215+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:16.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:16 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:16.433+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:16.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:16 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:16.433+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:16.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:16 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:16.433+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:16.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:16 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:16.433+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:16.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:16 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:16.433+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:17.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:17 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:17.138+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:17.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:17 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:17.138+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:17.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:17 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:17.138+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:17.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:17 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:17.138+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:17.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:17 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:17.138+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:17.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:17 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:17.138+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:49.193720+0000 front 2024-05-10T04:08:49.193664+0000 (oldest deadline 2024-05-10T04:09:12.594110+0000) 2024-05-10T04:09:17.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:17 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:17.421+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:17.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:17 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:17.421+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:17.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:17 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:17.421+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:17.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:17 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:17.421+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:17.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:17 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:17.421+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:17.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:17 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:17.257+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:17.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:17 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:17.257+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:17.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:17 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:17.257+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:17.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:17 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:17.257+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:17.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:17 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:17.257+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:17.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:17 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:17.257+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:18.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:18 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:18.096+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.187210+0000 front 2024-05-10T04:08:10.187185+0000 (oldest deadline 2024-05-10T04:08:39.588171+0000) 2024-05-10T04:09:18.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:18 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:18.096+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:19.588884+0000 front 2024-05-10T04:08:19.588693+0000 (oldest deadline 2024-05-10T04:08:44.288781+0000) 2024-05-10T04:09:18.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:18 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:18.096+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.289370+0000 front 2024-05-10T04:08:24.289384+0000 (oldest deadline 2024-05-10T04:08:48.989373+0000) 2024-05-10T04:09:18.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:18 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:18.096+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:33.690276+0000 front 2024-05-10T04:08:33.690258+0000 (oldest deadline 2024-05-10T04:08:55.990380+0000) 2024-05-10T04:09:18.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:18 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:18.096+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:38.191786+0000 front 2024-05-10T04:08:38.191894+0000 (oldest deadline 2024-05-10T04:09:03.392000+0000) 2024-05-10T04:09:18.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:18 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9[84981]: 2024-05-10T04:09:18.096+0000 7f12b8d2d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:49.193720+0000 front 2024-05-10T04:08:49.193664+0000 (oldest deadline 2024-05-10T04:09:12.594110+0000) 2024-05-10T04:09:18.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:18 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:18.427+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:18.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:18 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:18.427+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:18.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:18 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:18.427+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:18.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:18 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:18.427+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:18.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:18 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:18.427+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:18.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:18 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:18.427+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:18.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:18 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:18.270+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:18.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:18 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:18.270+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:18.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:18 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:18.270+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:18.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:18 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:18.270+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:18.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:18 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:18.270+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:18.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:18 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:18.270+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:19.463 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:09:19 smithi150 bash[116750]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-9 2024-05-10T04:09:19.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:19 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:19.389+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:19.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:19 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:19.389+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:19.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:19 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:19.389+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:19.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:19 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:19.389+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:19.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:19 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:19.389+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:19.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:19 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:19.389+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:19.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:19 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:19.275+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:19.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:19 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:19.275+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:19.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:19 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:19.275+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:19.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:19 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:19.275+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:19.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:19 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:19.275+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:19.742 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:19 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:19.275+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:20.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.9.service' 2024-05-10T04:09:20.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:09:20.502 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-10T04:09:20.503 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-10T04:09:20.503 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.10 2024-05-10T04:09:20.532 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:20 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:20.245+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:20.532 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:20 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:20.245+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:20.532 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:20 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:20.245+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:20.533 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:20 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:20.245+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:20.533 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:20 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:20.245+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:20.533 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:20 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:20.245+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:20.533 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:20 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:20.245+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.585587+0000 front 2024-05-10T04:08:56.585547+0000 (oldest deadline 2024-05-10T04:09:20.085481+0000) 2024-05-10T04:09:20.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:20 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:20.392+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:20.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:20 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:20.392+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:20.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:20 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:20.392+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:20.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:20 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:20.392+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:20.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:20 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:20.392+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:20.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:20 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:20.392+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:20.991 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:20 smithi161 systemd[1]: Stopping Ceph osd.10 for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:09:20.991 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:20 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:20.755+0000 7fbf6c112700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:09:20.991 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:20 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:20.755+0000 7fbf6c112700 -1 osd.10 91 *** Got signal Terminated *** 2024-05-10T04:09:20.992 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:20 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:20.755+0000 7fbf6c112700 -1 osd.10 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:09:21.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:21 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:21.233+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:21.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:21 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:21.233+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:21.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:21 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:21.233+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:21.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:21 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:21.233+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:21.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:21 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:21.233+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:21.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:21 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:21.233+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:21.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:21 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:21.233+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.585587+0000 front 2024-05-10T04:08:56.585547+0000 (oldest deadline 2024-05-10T04:09:20.085481+0000) 2024-05-10T04:09:21.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:21 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:21.357+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:21.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:21 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:21.357+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:21.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:21 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:21.357+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:21.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:21 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:21.357+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:21.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:21 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:21.357+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:21.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:21 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:21.357+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:22.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:22 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:22.186+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:22.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:22 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:22.186+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:22.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:22 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:22.186+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:22.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:22 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:22.186+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:22.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:22 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:22.186+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:22.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:22 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:22.186+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:22.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:22 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:22.186+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.585587+0000 front 2024-05-10T04:08:56.585547+0000 (oldest deadline 2024-05-10T04:09:20.085481+0000) 2024-05-10T04:09:22.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:22 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:22.359+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:22.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:22 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:22.359+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:22.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:22 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:22.359+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:22.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:22 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:22.359+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:22.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:22 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:22.359+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:22.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:22 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:22.359+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:23.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:23 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:23.208+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:23.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:23 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:23.208+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:23.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:23 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:23.208+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:23.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:23 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:23.208+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:23.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:23 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:23.208+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:23.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:23 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:23.208+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:23.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:23 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:23.208+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.585587+0000 front 2024-05-10T04:08:56.585547+0000 (oldest deadline 2024-05-10T04:09:20.085481+0000) 2024-05-10T04:09:23.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:23 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:23.321+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:23.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:23 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:23.321+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:23.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:23 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:23.321+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:23.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:23 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:23.321+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:23.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:23 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:23.321+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:23.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:23 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:23.321+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:24.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:24 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:24.166+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:24.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:24 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:24.166+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:24.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:24 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:24.166+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:24.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:24 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:24.166+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:24.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:24 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:24.166+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:24.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:24 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:24.166+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:24.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:24 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:24.166+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.585587+0000 front 2024-05-10T04:08:56.585547+0000 (oldest deadline 2024-05-10T04:09:20.085481+0000) 2024-05-10T04:09:24.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:24 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:24.323+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:24.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:24 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:24.323+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:24.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:24 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:24.323+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:24.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:24 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:24.323+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:24.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:24 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:24.323+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:24.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:24 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:24.323+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:25.491 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:25 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:25.187+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:10.077469+0000 front 2024-05-10T04:08:10.077352+0000 (oldest deadline 2024-05-10T04:08:35.377429+0000) 2024-05-10T04:09:25.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:25 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:25.187+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:18.779200+0000 front 2024-05-10T04:08:18.779161+0000 (oldest deadline 2024-05-10T04:08:41.679296+0000) 2024-05-10T04:09:25.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:25 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:25.187+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:26.881006+0000 front 2024-05-10T04:08:26.881152+0000 (oldest deadline 2024-05-10T04:08:52.780886+0000) 2024-05-10T04:09:25.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:25 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:25.187+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:08:35.082201+0000 front 2024-05-10T04:08:35.082184+0000 (oldest deadline 2024-05-10T04:08:56.182042+0000) 2024-05-10T04:09:25.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:25 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:25.187+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:40.282849+0000 front 2024-05-10T04:08:40.282882+0000 (oldest deadline 2024-05-10T04:09:04.382929+0000) 2024-05-10T04:09:25.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:25 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:25.187+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:45.484192+0000 front 2024-05-10T04:08:45.484235+0000 (oldest deadline 2024-05-10T04:09:10.784044+0000) 2024-05-10T04:09:25.492 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:25 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10[76478]: 2024-05-10T04:09:25.187+0000 7fbf646c2700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.585587+0000 front 2024-05-10T04:08:56.585547+0000 (oldest deadline 2024-05-10T04:09:20.085481+0000) 2024-05-10T04:09:25.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:25 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:25.365+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:25.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:25 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:25.365+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:25.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:25 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:25.365+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:25.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:25 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:25.365+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:25.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:25 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:25.365+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:25.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:25 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:25.365+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:26.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:26 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:26.388+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:26.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:26 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:26.388+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:26.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:26 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:26.388+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:26.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:26 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:26.388+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:26.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:26 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:26.388+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:26.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:26 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:26.388+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:26.741 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:09:26 smithi161 bash[87890]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-10 2024-05-10T04:09:27.610 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.10.service' 2024-05-10T04:09:27.692 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:09:27.693 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-10T04:09:27.693 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-10T04:09:27.693 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.11 2024-05-10T04:09:27.724 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:27 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:27.405+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:27.724 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:27 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:27.405+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:27.724 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:27 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:27.405+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:27.724 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:27 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:27.405+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:27.724 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:27 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:27.405+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:27.724 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:27 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:27.405+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:27.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:27 smithi181 systemd[1]: Stopping Ceph osd.11 for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:09:27.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:27 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:27.956+0000 7f1a2f3d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:09:27.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:27 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:27.956+0000 7f1a2f3d4700 -1 osd.11 91 *** Got signal Terminated *** 2024-05-10T04:09:27.987 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:27 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:27.956+0000 7f1a2f3d4700 -1 osd.11 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:09:28.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:28 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:28.395+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:28.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:28 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:28.395+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:28.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:28 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:28.395+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:28.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:28 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:28.395+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:28.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:28 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:28.395+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:28.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:28 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:28.395+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:28.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:28 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:28.395+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-05-10T04:09:03.039214+0000 front 2024-05-10T04:09:03.039399+0000 (oldest deadline 2024-05-10T04:09:27.739006+0000) 2024-05-10T04:09:29.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:29 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:29.366+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:29.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:29 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:29.366+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:29.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:29 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:29.366+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:29.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:29 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:29.366+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:29.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:29 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:29.366+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:29.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:29 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:29.366+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:29.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:29 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:29.366+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-05-10T04:09:03.039214+0000 front 2024-05-10T04:09:03.039399+0000 (oldest deadline 2024-05-10T04:09:27.739006+0000) 2024-05-10T04:09:30.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:30 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:30.317+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:30.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:30 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:30.317+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:30.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:30 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:30.317+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:30.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:30 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:30.317+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:30.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:30 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:30.317+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:30.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:30 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:30.317+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:30.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:30 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:30.317+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-05-10T04:09:03.039214+0000 front 2024-05-10T04:09:03.039399+0000 (oldest deadline 2024-05-10T04:09:27.739006+0000) 2024-05-10T04:09:31.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:31 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:31.313+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:31.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:31 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:31.313+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:31.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:31 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:31.313+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:31.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:31 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:31.313+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:31.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:31 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:31.313+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:31.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:31 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:31.313+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:31.738 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:31 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:31.313+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-05-10T04:09:03.039214+0000 front 2024-05-10T04:09:03.039399+0000 (oldest deadline 2024-05-10T04:09:27.739006+0000) 2024-05-10T04:09:32.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:32 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:32.307+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:08:13.130486+0000 front 2024-05-10T04:08:13.130277+0000 (oldest deadline 2024-05-10T04:08:36.630327+0000) 2024-05-10T04:09:32.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:32 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:32.307+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:08:20.731718+0000 front 2024-05-10T04:08:20.731701+0000 (oldest deadline 2024-05-10T04:08:44.231724+0000) 2024-05-10T04:09:32.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:32 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:32.307+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:08:24.232085+0000 front 2024-05-10T04:08:24.232098+0000 (oldest deadline 2024-05-10T04:08:49.532204+0000) 2024-05-10T04:09:32.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:32 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:32.307+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:08:39.434121+0000 front 2024-05-10T04:08:39.434303+0000 (oldest deadline 2024-05-10T04:09:03.534090+0000) 2024-05-10T04:09:32.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:32 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:32.307+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:08:48.036121+0000 front 2024-05-10T04:08:48.036156+0000 (oldest deadline 2024-05-10T04:09:10.936379+0000) 2024-05-10T04:09:32.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:32 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:32.307+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:08:56.837442+0000 front 2024-05-10T04:08:56.837422+0000 (oldest deadline 2024-05-10T04:09:17.937363+0000) 2024-05-10T04:09:32.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:32 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:32.307+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-05-10T04:09:03.039214+0000 front 2024-05-10T04:09:03.039399+0000 (oldest deadline 2024-05-10T04:09:27.739006+0000) 2024-05-10T04:09:32.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:32 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11[77172]: 2024-05-10T04:09:32.307+0000 7f1a27984700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2024-05-10T04:09:07.739378+0000 front 2024-05-10T04:09:07.739397+0000 (oldest deadline 2024-05-10T04:09:31.839467+0000) 2024-05-10T04:09:33.737 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:09:33 smithi181 bash[88862]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-osd-11 2024-05-10T04:09:34.702 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@osd.11.service' 2024-05-10T04:09:34.784 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:09:34.784 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-10T04:09:34.784 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-10T04:09:34.784 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.a 2024-05-10T04:09:35.213 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:09:34 smithi150 systemd[1]: Stopping Ceph mds.a for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:09:35.213 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:09:35 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-a[93102]: 2024-05-10T04:09:35.018+0000 7feedaafa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:09:35.213 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:09:35 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-a[93102]: 2024-05-10T04:09:35.018+0000 7feedaafa700 -1 mds.a *** got signal Terminated *** 2024-05-10T04:09:36.822 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:09:36 smithi150 bash[117005]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-a 2024-05-10T04:09:36.828 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.a.service' 2024-05-10T04:09:36.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:09:36.863 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-10T04:09:36.863 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-10T04:09:36.863 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.d 2024-05-10T04:09:37.130 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:09:36 smithi150 systemd[1]: Stopping Ceph mds.d for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:09:37.463 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:09:37 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-d[92189]: 2024-05-10T04:09:37.126+0000 7f5567019700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:09:37.463 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:09:37 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-d[92189]: 2024-05-10T04:09:37.126+0000 7f5567019700 -1 mds.d *** got signal Terminated *** 2024-05-10T04:09:40.462 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:09:40 smithi150 bash[117144]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-d 2024-05-10T04:09:40.508 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.d.service' 2024-05-10T04:09:40.542 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:09:40.543 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-10T04:09:40.543 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-10T04:09:40.543 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.g 2024-05-10T04:09:40.802 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:09:40 smithi150 systemd[1]: Stopping Ceph mds.g for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:09:41.212 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:09:40 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-g[92715]: 2024-05-10T04:09:40.801+0000 7f996517a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:09:41.213 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:09:40 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-g[92715]: 2024-05-10T04:09:40.801+0000 7f996517a700 -1 mds.g *** got signal Terminated *** 2024-05-10T04:09:42.922 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.g.service' 2024-05-10T04:09:42.946 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:09:42 smithi150 bash[117283]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-g 2024-05-10T04:09:42.946 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:09:42 smithi150 systemd[1]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.g.service: Succeeded. 2024-05-10T04:09:42.946 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:09:42 smithi150 systemd[1]: Stopped Ceph mds.g for 325b4c82-0e81-11ef-bc98-c7b262605968. 2024-05-10T04:09:42.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:09:42.957 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-10T04:09:42.957 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-10T04:09:42.957 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.j 2024-05-10T04:09:43.210 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:09:43 smithi150 systemd[1]: Stopping Ceph mds.j for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:09:43.462 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:09:43 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-j[91801]: 2024-05-10T04:09:43.209+0000 7f6c37cff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:09:43.463 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:09:43 smithi150 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-j[91801]: 2024-05-10T04:09:43.209+0000 7f6c37cff700 -1 mds.j *** got signal Terminated *** 2024-05-10T04:09:46.692 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:09:46 smithi150 bash[117424]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-j 2024-05-10T04:09:46.697 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.j.service' 2024-05-10T04:09:46.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:09:46.731 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-10T04:09:46.731 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-10T04:09:46.731 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.b 2024-05-10T04:09:47.241 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:09:46 smithi161 systemd[1]: Stopping Ceph mds.b for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:09:47.241 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:09:46 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-b[83518]: 2024-05-10T04:09:46.966+0000 7fee527f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:09:47.241 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:09:46 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-b[83518]: 2024-05-10T04:09:46.966+0000 7fee527f0700 -1 mds.b *** got signal Terminated *** 2024-05-10T04:09:51.396 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:09:51 smithi161 bash[88154]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-b 2024-05-10T04:09:51.402 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.b.service' 2024-05-10T04:09:51.439 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:09:51.440 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-10T04:09:51.440 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-10T04:09:51.440 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.e 2024-05-10T04:09:51.700 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:09:51 smithi161 systemd[1]: Stopping Ceph mds.e for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:09:51.991 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:09:51 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-e[82824]: 2024-05-10T04:09:51.699+0000 7f7d14d57700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:09:51.991 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:09:51 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-e[82824]: 2024-05-10T04:09:51.699+0000 7f7d14d57700 -1 mds.e *** got signal Terminated *** 2024-05-10T04:09:55.991 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:09:55 smithi161 bash[88295]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-e 2024-05-10T04:09:55.998 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.e.service' 2024-05-10T04:09:56.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:09:56.033 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-10T04:09:56.033 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-10T04:09:56.033 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.h 2024-05-10T04:09:56.309 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:09:56 smithi161 systemd[1]: Stopping Ceph mds.h for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:09:56.741 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:09:56 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-h[83166]: 2024-05-10T04:09:56.308+0000 7f786c8c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:09:56.741 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:09:56 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-h[83166]: 2024-05-10T04:09:56.308+0000 7f786c8c8700 -1 mds.h *** got signal Terminated *** 2024-05-10T04:09:59.491 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:09:59 smithi161 bash[88434]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-h 2024-05-10T04:09:59.507 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.h.service' 2024-05-10T04:09:59.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:09:59.541 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-10T04:09:59.542 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-10T04:09:59.542 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.k 2024-05-10T04:09:59.799 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:09:59 smithi161 systemd[1]: Stopping Ceph mds.k for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:10:00.241 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:09:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-k[82486]: 2024-05-10T04:09:59.798+0000 7fba5ba11700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:10:00.241 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:09:59 smithi161 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-k[82486]: 2024-05-10T04:09:59.798+0000 7fba5ba11700 -1 mds.k *** got signal Terminated *** 2024-05-10T04:10:04.103 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:10:03 smithi161 bash[88574]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-k 2024-05-10T04:10:04.239 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.k.service' 2024-05-10T04:10:04.276 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:10:04.276 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-10T04:10:04.276 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-10T04:10:04.276 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.c 2024-05-10T04:10:04.737 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:10:04 smithi181 systemd[1]: Stopping Ceph mds.c for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:10:04.737 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:10:04 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-c[84023]: 2024-05-10T04:10:04.525+0000 7f0d3dd5d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:10:04.737 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:10:04 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-c[84023]: 2024-05-10T04:10:04.525+0000 7f0d3dd5d700 -1 mds.c *** got signal Terminated *** 2024-05-10T04:10:07.987 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:10:07 smithi181 bash[89116]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-c 2024-05-10T04:10:08.000 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.c.service' 2024-05-10T04:10:08.038 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:10:08.039 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-10T04:10:08.039 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-10T04:10:08.039 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.f 2024-05-10T04:10:08.301 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:10:08 smithi181 systemd[1]: Stopping Ceph mds.f for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:10:08.737 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:10:08 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-f[83681]: 2024-05-10T04:10:08.299+0000 7f244d89a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:10:08.737 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:10:08 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-f[83681]: 2024-05-10T04:10:08.299+0000 7f244d89a700 -1 mds.f *** got signal Terminated *** 2024-05-10T04:10:11.877 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:10:11 smithi181 bash[89256]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-f 2024-05-10T04:10:11.882 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.f.service' 2024-05-10T04:10:11.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:10:11.917 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-10T04:10:11.918 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-10T04:10:11.918 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.i 2024-05-10T04:10:12.183 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:10:11 smithi181 systemd[1]: Stopping Ceph mds.i for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:10:12.486 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:10:12 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-i[84368]: 2024-05-10T04:10:12.182+0000 7f631602a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:10:12.487 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:10:12 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-i[84368]: 2024-05-10T04:10:12.182+0000 7f631602a700 -1 mds.i *** got signal Terminated *** 2024-05-10T04:10:15.922 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:10:15 smithi181 bash[89396]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-i 2024-05-10T04:10:15.927 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.i.service' 2024-05-10T04:10:15.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:10:15.963 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-10T04:10:15.963 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-10T04:10:15.963 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.l 2024-05-10T04:10:16.224 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:10:16 smithi181 systemd[1]: Stopping Ceph mds.l for 325b4c82-0e81-11ef-bc98-c7b262605968... 2024-05-10T04:10:16.487 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:10:16 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-l[83340]: 2024-05-10T04:10:16.223+0000 7f4260dc9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:10:16.487 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:10:16 smithi181 ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-l[83340]: 2024-05-10T04:10:16.223+0000 7f4260dc9700 -1 mds.l *** got signal Terminated *** 2024-05-10T04:10:19.987 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:10:19 smithi181 bash[89535]: ceph-325b4c82-0e81-11ef-bc98-c7b262605968-mds-l 2024-05-10T04:10:20.028 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-325b4c82-0e81-11ef-bc98-c7b262605968@mds.l.service' 2024-05-10T04:10:20.062 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:10:20.062 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-10T04:10:20.062 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T04:10:20.247 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T04:10:21.639 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T04:10:21.829 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T04:10:23.220 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T04:10:23.415 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T04:10:24.813 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:10:24.850 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:10:24.888 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:10:24.918 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-10T04:10:24.920 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505/remote/smithi150/crash 2024-05-10T04:10:24.921 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/crash -- . 2024-05-10T04:10:24.962 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T04:10:24.962 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-05-10T04:10:24.964 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505/remote/smithi161/crash 2024-05-10T04:10:24.965 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/crash -- . 2024-05-10T04:10:25.014 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T04:10:25.014 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-05-10T04:10:25.016 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505/remote/smithi181/crash 2024-05-10T04:10:25.017 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/crash -- . 2024-05-10T04:10:25.066 INFO:teuthology.orchestra.run.smithi181.stderr:tar: /var/lib/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T04:10:25.067 INFO:teuthology.orchestra.run.smithi181.stderr:tar: Error is not recoverable: exiting now 2024-05-10T04:10:25.067 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-10T04:10:25.068 DEBUG:teuthology.orchestra.run.smithi150:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-10T04:10:25.103 INFO:teuthology.orchestra.run.smithi150.stderr:grep: /var/log/ceph/325b4c82-0e81-11ef-bc98-c7b262605968/ceph.log: No such file or directory 2024-05-10T04:10:25.105 INFO:tasks.cephadm:Compressing logs... 2024-05-10T04:10:25.105 DEBUG:teuthology.orchestra.run.smithi150:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T04:10:25.148 DEBUG:teuthology.orchestra.run.smithi161:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T04:10:25.151 DEBUG:teuthology.orchestra.run.smithi181:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T04:10:25.173 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-10T04:10:25.177 INFO:teuthology.orchestra.run.smithi181.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-10T04:10:25.178 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-10T04:10:49.633 INFO:tasks.cephadm:Archiving logs... 2024-05-10T04:10:49.634 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505/remote/smithi150/log 2024-05-10T04:10:49.635 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T04:10:52.786 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505/remote/smithi161/log 2024-05-10T04:10:52.787 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T04:10:55.634 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505/remote/smithi181/log 2024-05-10T04:10:55.635 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T04:10:58.455 INFO:tasks.cephadm:Removing cluster... 2024-05-10T04:10:58.455 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 --force 2024-05-10T04:10:58.644 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T04:10:58.913 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 --force 2024-05-10T04:10:59.102 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T04:10:59.370 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 325b4c82-0e81-11ef-bc98-c7b262605968 --force 2024-05-10T04:10:59.556 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: 325b4c82-0e81-11ef-bc98-c7b262605968 2024-05-10T04:10:59.813 INFO:tasks.cephadm:Removing cephadm ... 2024-05-10T04:10:59.814 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T04:10:59.838 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T04:10:59.859 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T04:10:59.877 INFO:tasks.cephadm:Teardown complete 2024-05-10T04:10:59.878 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-10T04:10:59.915 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-10T04:10:59.915 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T04:10:59.918 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T04:10:59.921 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T04:11:00.037 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-10T04:11:00.081 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-10T04:11:00.081 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T04:11:00.084 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T04:11:00.088 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T04:11:00.100 INFO:teuthology.orchestra.run.smithi150.stderr:bash: ntpq: command not found 2024-05-10T04:11:00.103 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2024-05-10T04:11:00.104 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2024-05-10T04:11:00.106 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T04:11:00.106 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-05-10T04:11:00.106 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv01.front.sepia.ceph.com 3 7 377 98 +2275us[+2276us] +/- 55ms 2024-05-10T04:11:00.106 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 31 -2078us[-2078us] +/- 59ms 2024-05-10T04:11:00.106 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 -770us[ -770us] +/- 60ms 2024-05-10T04:11:00.106 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:11:00.108 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T04:11:00.108 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-05-10T04:11:00.108 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 +2176us[+2372us] +/- 40ms 2024-05-10T04:11:00.108 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 -2262us[-2067us] +/- 59ms 2024-05-10T04:11:00.108 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 -884us[ -884us] +/- 60ms 2024-05-10T04:11:00.108 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:11:00.109 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T04:11:00.109 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-05-10T04:11:00.109 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 +2224us[+2428us] +/- 40ms 2024-05-10T04:11:00.109 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 98 -2213us[-2026us] +/- 58ms 2024-05-10T04:11:00.109 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -826us[ -622us] +/- 60ms 2024-05-10T04:11:00.109 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:11:00.110 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-10T04:11:00.134 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-10T04:11:00.135 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-10T04:11:00.212 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T04:11:00.294 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T04:11:00.369 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T04:11:00.437 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T04:11:00.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:11:00.472 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T04:11:00.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:11:00.504 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T04:11:00.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:11:00.536 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-10T04:11:00.548 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-10T04:11:00.587 INFO:teuthology.task.internal:Duration was 1767.073759 seconds 2024-05-10T04:11:00.587 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-10T04:11:00.635 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-10T04:11:00.635 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T04:11:00.638 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T04:11:00.640 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T04:11:00.687 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T04:11:00.688 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T04:11:00.689 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T04:11:01.004 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-10T04:11:01.004 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:11:01.004 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T04:11:01.028 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-05-10T04:11:01.029 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T04:11:01.052 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2024-05-10T04:11:01.052 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T04:11:01.074 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-10T04:11:01.074 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T04:11:01.077 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T04:11:01.095 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T04:11:01.164 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-10T04:11:01.164 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T04:11:01.168 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T04:11:01.170 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T04:11:01.283 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-10T04:11:01.295 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-10T04:11:01.295 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T04:11:01.322 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T04:11:01.350 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T04:11:01.377 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-10T04:11:01.388 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T04:11:01.391 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T04:11:01.393 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T04:11:01.415 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-05-10T04:11:01.417 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-05-10T04:11:01.444 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2024-05-10T04:11:01.480 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T04:11:01.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:11:01.497 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T04:11:01.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:11:01.513 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T04:11:01.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:11:01.537 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-10T04:11:01.548 INFO:teuthology.task.internal:Transferring archived files... 2024-05-10T04:11:01.549 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505/remote/smithi150 2024-05-10T04:11:01.549 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T04:11:01.594 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505/remote/smithi161 2024-05-10T04:11:01.594 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T04:11:01.635 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700505/remote/smithi181 2024-05-10T04:11:01.635 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T04:11:01.675 INFO:teuthology.task.internal:Removing archive directory... 2024-05-10T04:11:01.675 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T04:11:01.679 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T04:11:01.681 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T04:11:01.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-10T04:11:01.739 INFO:teuthology.task.internal:Not uploading archives. 2024-05-10T04:11:01.739 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-10T04:11:01.801 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-10T04:11:01.802 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T04:11:01.804 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T04:11:01.806 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T04:11:01.818 INFO:teuthology.orchestra.run.smithi150.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 04:11 /home/ubuntu/cephtest 2024-05-10T04:11:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 04:11 /home/ubuntu/cephtest 2024-05-10T04:11:01.822 INFO:teuthology.orchestra.run.smithi181.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 04:11 /home/ubuntu/cephtest 2024-05-10T04:11:01.823 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-10T04:11:01.853 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-10T04:11:01.956 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iozone}} duration: 1767.0737586021423 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-10T04:11:01.956 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T04:11:02.032 INFO:teuthology.run:pass